IBM Support

Crash on Solaris produces no core or truncated core

Troubleshooting


Problem

Document for when a IBM WebSphere Application Server crash on Solaris produces no core or truncated core.

Resolving The Problem

Follow these directions in order, unless directed by IBM support.

NOTE: The settings below require a restart of your application server JVM. If you also use a nodeagent to start your server(s), you will need to restart this as well. Changing ulimit settings additionally require the restart to occur on the same command line (or terminal) session.

Even without a core dump, there still may be a file named hs_err_pid<PID>.log in the profile root directory. This is an important file and can be used to identify some crashes without the aid of a full core dump. Not all crashes will generate this file.



1. Setting Ulimits
See also: Guidelines for setting ulimits

To set ulimits on the core and file sizes to unlimited, run these two commands as the user who starts the nodeagent and/or application server

ulimit -c unlimited
ulimit -f unlimited
* On Solaris, units are in 512 byte blocks except if "unlimited" is used as the value.

You can run ulimit -a to verify current ulimit settings.
Also, if an hs_err_pid*.log was produced, the ulimits for the current process are also listed at the very bottom of this file.





2. Configuring the Operating System for Full Core Generation

Make sure that in the config file /etc/system that the parameter sys:coredumpsize is not set to 0, as this will disable core generation.






Frequently Asked Questions (FAQ)


What happens if I do not have write permission in the profile's root directory, or the directory I am redirecting javacores, heapdumps, and system core files to?

This will result in a failure when writing these files to the system.
Also make sure that you have enough free space on your file system.

Are ulimit settings permanent?

No, they are temporary and last as long as the session is alive. Ulimits are set on a per user basis, and the settings are applied per session, such as a command-line window. If a brand new session is started, and is not spawned from the current session, the ulimits will load the defaults.



Can I set ulimit settings globally?
On Solaris, there is no "limits" config files as there are in AIX and Linux operating systems. However, the "ulimit" command can be added to each user's .profile file (in their home directory), or you can add it to the system-wide /etc/profile file to simulate a global setting.

Additionally, the ulimit setting for the file size limit can be set in the /etc/default/login file by uncommenting the line ULIMIT=0




Where can core files be generated?

Normally found in the profile's root directory, but can be in a number of alternative locations. Try searching in these locations first:
  • <WAS_HOME>/profiles/<PROFILE_NAME>
  • <WAS_HOME>/bin/
  • /tmp


If you cannot find a core file in any of these locations, search your entire machine for core* files.

Additionally, the Solaris operating system attempts to create up to two core files for each abnormal-ending process. One is a core file that uses a global core file name pattern; the other uses a per-process core file name pattern. These actual files use directory information to determine the location of the resulting core files.

The core file name pattern and directory path is configured using the coreadm command. By default, the global core file pattern is disabled and not used, and the per-process core file pattern is set to core. Therefore, by default, the operating system attempts to create a corefile named core in the process's current working directory.

Run the command coreadm to set the location of the core file. The following is a list of parameters you can set for the core file. Use this information to determine if core files are enabled and where the core file is to be written (this command saves the configuration to /etc/coreadm.conf).

global core file pattern: /var/core/core.%f.%
                init core file pattern: core
                     global core dumps: enabled
                per-process core dumps: enabled
               global setid core dumps: enabled
          per-process setid core dumps: disabled
              global core dump logging: disabled



Additional Information
For Solaris specific information for core dumps, these links may be useful:

Solaris 11: http://docs.oracle.com/cd/E23824_01/html/821-1451/tscore-7.html

Solaris 10: http://download.oracle.com/docs/cd/E19253-01/817-0403/tscore-7/index.html

Solaris 9: http://download.oracle.com/docs/cd/E19683-01/817-6959/tscrashdumps-40145/index.html

Solaris 8: http://download.oracle.com/docs/cd/E19455-01/805-7229/6j6q8svho/index.html

Solaris 7: http://download.oracle.com/docs/cd/E19620-01/806-1650/6jau1364s/index.html

Solaris 6: Not supported in any WebSphere version from 4.x on

[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Crash","Platform":[{"code":"PF027","label":"Solaris"}],"Version":"9.0;8.5;8.0;7.0","Edition":"Base;Express;Network Deployment","Line of Business":{"code":"LOB36","label":"IBM Automation"}},{"Product":{"code":"SS7JFU","label":"WebSphere Application Server - Express"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"General","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"}],"Version":"7.0;6.1;6.0;5.1","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}},{"Product":{"code":"SSNVBF","label":"Runtimes for Java Technology"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Java SDK","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
15 June 2018

UID

swg21170466