Troubleshooting
Problem
Your application server terminated without warning or a userdump is generated but you are not sure what data to collect. The following MustGather will assist you in collecting the critical data to troubleshoot issues with an IBM WebSphere Application Server crash on the Windows platform.
Symptom
Userdump generated, Java™ Virtual Machine process disappears or terminates.
Environment
Microsoft Windows Vista (SP1 or later)*
Microsoft Windows 7
Microsoft Windows 8
Microsoft Windows Server 2008
Microsoft Windows Server 2008 R2
Microsoft Windows Server 2012
Microsoft Windows Server 2012 R2
Microsoft Windows 10
NOTE: WebSphere Liberty Profile can use these same instructions if running with an IBM SDK.
Microsoft Windows 7
Microsoft Windows 8
Microsoft Windows Server 2008
Microsoft Windows Server 2008 R2
Microsoft Windows Server 2012
Microsoft Windows Server 2012 R2
Microsoft Windows 10
NOTE: WebSphere Liberty Profile can use these same instructions if running with an IBM SDK.
Resolving The Problem
Your application server terminated without warning or a userdump is generated but you are not sure what data to collect. The following MustGather will assist you in collecting the critical data to troubleshoot issues with an IBM WebSphere Application Server crash on the Windows platform.
List of Operating System versions compatible with these instructions
Microsoft Windows Server 2008
Microsoft Windows Server 2008 R2
Microsoft Windows Server 2012
Microsoft Windows Server 2012 R2
Additionally, these environments will work as well...
Microsoft Windows Vista (SP1 or later)
Microsoft Windows 7
Microsoft Windows 8
Microsoft Windows 10
If you are trying to generate dumps manually on Windows:
Generating Javacores and Userdumps Manually For Performance, Hang or High CPU Issues
What if I'm using WebSphere Liberty Profile?
This document can be used with Liberty Profile, however locations of dump files (and server logs) will be different. Also, if you are not using an IBM SDK with your Liberty Profile, jextract will not be available.
Configuring Full Core Dumps
If you already have a dump file, proceed to the next step "Locating the Core Dump", however if it's determined that the core dump is truncated or isn't a full dump, and there were no errors running jextract, you will need to follow this section's instructions.
Starting with Windows Server 2008 and Windows Vista with Service Pack 1 (SP1), Windows Error Reporting (WER) can be configured so that full user-mode dumps are collected and stored locally after a user-mode application crashes.
Follow the below steps to configure your server to generate full dump on crash:
- For WebSphere Application Server:
- Backup the Windows registry by following the steps in How to back up and restore the registry in Windows.
- Edit the Windows registry and create the following key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps - Create a new DWORD and name it DumpType. Set the value to 2 (Decimal) then Exit.
For further reference see "Collecting User-Mode Dumps"
https://docs.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps
Assuming the dump file (core.DATESTAMP.TIMESTAMP.PID.COUNT.dmp) is complete and is not truncated, you can normally find the dump file in the profile root directory by default (certain JVM arguments and/or environment variables can alter the default location).
- <WAS_HOME>/profiles/<PROFILE_ROOT>/
You can also check the native_stderr.log, as the location of any dump due to a crash will be logged.
Additionally, locate the javacore and any snap files which will be generated in a JVM crash.
Javacores are snapshots of the JVM activity and are essential to troubleshooting these issues. These files should also be found, by default, in the profile's root directory.
Processing the Core Dump
It is critical that jextract is run against the system core. Jextract is located in <WAS_HOME>/java/jre/bin directory. Instructions for the execution of jextract can be found in this technote:
- "How to process an IBM SDK core dump with jextract"
http://www.ibm.com/support/docview.wss?uid=swg21577379
- Can I continue if jextract fails?
If jextract produces errors, continue collecting the rest of the MustGather data. Also, please take a screen shot (or copy the output in the terminal) that includes the command line and the command that was run, as well as the error given by the jextract command. Please send this screen shot to support for further investigation. - Locating the server logs:
<WAS_HOME>/profiles/<PROFILE_ROOT>/logs/<SERVER_NAME>/
Summary of all files to be uploaded
Gather the following files:
- Server logs / native logs (SystemOut.log, SystemErr.log, native_stdout.log, native_stderr.log)
- All javacore*.txt files
- All snap* files
- *.zip generated by jextract (this will contain the dump file)
You can archive these files up (although the processed core dump may want to be sent separately). Follow the instructions in the link below for sending the MustGather data to IBM support. Please note that FTP/SFTP/FTPS is the best option for transferring large files, especially the processed core dump
"Exchanging information with IBM Technical Support for problem determination"
http://www-01.ibm.com/support/docview.wss?uid=swg21153852
For a listing of all technotes, downloads, and educational materials specific to Crash, search the WebSphere Application Server support site
Related information
Submitting information to IBM support
Steps to getting support for WebSphere Application Server
MustGather: Read first for WebSphere Application Server
Troubleshooting guide for WebSphere Application Server
To diagnose or identify a problem, it is sometimes necessary to provide Technical Support with data and information from your system. In addition, Technical Support might also need to provide you with tools or utilities to be used in problem determination. You can submit files using one of following methods to help speed problem diagnosis:
- IBM Support Assistant (ISA)
- Service Request (SR)
- FTP to the Enhanced Customer Data Repository (ECuRep)
Related Information
[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Crash","Platform":[{"code":"PF033","label":"Windows"}],"Version":"9.0.0.0;8.5.5;8.5;8.0;7.0","Edition":"Base;Express;Network Deployment","Line of Business":{"code":"LOB36","label":"IBM Automation"}},{"Product":{"code":"SS7JFU","label":"WebSphere Application Server - Express"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Hangs\/performance degradation","Platform":[{"code":"PF033","label":"Windows"}],"Version":"8.5;8.0;7.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}},{"Product":{"code":"SSNVBF","label":"Runtimes for Java Technology"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Java SDK","Platform":[{"code":"PF033","label":"Windows"}],"Version":"","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]
Was this topic helpful?
Document Information
Modified date:
03 August 2020
UID
swg21053924