IBM Support

Managing System Dump Devices

Question & Answer


Question

What are some tips for managing system dump devices?

Answer

In a catastrophic operating system software failure, a system dump might be useful.

This document provides tips and examples to help system administrators ensure that a system dump is complete and usable for troubleshooting purposes.

Overview of system dump devices
Determining the proper size for a dump device
How to create a dedicated dump device
Tip: Do not dump to a mirrored logical volume
Tip: Dumping outside the rootvg

 


Overview of system dump devices

When an unexpected system halt occurs, the system dump facility automatically copies selected areas of kernel data to the primary dump device. These areas include kernel segment 0, and other areas registered in the Master Dump Table by kernel modules or kernel extensions.

There are two dumps devices: a primary and a secondary. To view information about the current dump configuration, use the sysdumpdev command.

# sysdumpdev -l
primary              /dev/lg_dumplv
secondary            /dev/sysdumpnull
copy directory       /var/adm/ras
forced copy flag     FALSE
always allow dump    TRUE
dump compression     ON
type of dump         fw-assisted
full memory dump     disallow

 

In this document example, the primary dump device is the logical volume /dev/lg_dumplv. The secondary dump device is a null device. The primary dump device is always in the rootvg.

Current versions of AIX ship with dump compression enabled. The "type of dump" field provides a choice between a traditional, or firmware assisted dump. A traditional dump is written by the system as it goes down. A firmware assisted dump is written as it boots and after device discovery. Full memory dumps should always be disallowed unless AIX development support requires a full dump for analysis.

When the operating system is installed, the primary dump device is automatically configured.

The default primary dump device is /dev/hd6. The default secondary dump device is /dev/sysdumpnull. If the system has 4 GB or more of memory, then the default, dedicated dump device is /dev/lg_dumplv. AIX Version 7.1 and subsequent versions extends firmware assisted dump capabilities to make it the default system dump method if it is supported by the platform.

Overview of some sysdumpdev flags:

The following flags are set by default:

forced copy flag     TRUE
always allow dump    FALSE
dump compression     ON

force copy flag: Specifies whether the system prompts you to copy this dump to external media when there is not enough space in the specified file system. The system has a set default copy directory. You can choose the one you would prefer to use. If the flag is FALSE and the system cannot copy this dump to the file system, then it discards the contents of the dump. Setting and removing the force copy flag also sets the copy directory.
Set the force copy flag:
# sysdumpdev -D /var/adm/ras
Unset the force copy flag:
# sysdumpdev -d /var/adm/ras

always allow dumpForce a dump with the key in the normal position, or on a machine without a key mode switch.

This statement refers to old AIX Power Systems that had a key switch to set a normal or service state.  If the system uses a Hardware Management Console, this setting is ignored. If the system is stand-alone, the setting is applied.

Set the always allow dump flag:

# sysdumpdev -K

 
Unset the always allow dump flag:
# sysdumpdev -k

If the primary dump device is the primary paging device, /dev/hd6, a traditional dump is written. If there is enough free space in the save area file system, the dump is copied there. The save area can be in a different file system in the rootvg volume group. AIX support recommends a dedicated dump device rather than hd6 to minimize maintenance.  Forcing the system to use a file system for a copy area causes the dump to be written twice.

copy directory Specifies a file system in the rootvg volume group where the dump will be copied upon reboot after a system dump.
Important: This setting applies only if the primary dump is the primary paging space, hd6, or if the forced copy flag is set to TRUE.  The setting is TRUE in the documented example.


Determining the proper size for a dump device

The default dump device created for system use is usually not large enough for a complete dump. To determine how large the dump device is, first determine the primary dump device.

# sysdumpdev -l
primary              /dev/lg_dumplv
secondary            /dev/sysdumpnull
copy directory       /var/adm/ras
forced copy flag     FALSE
always allow dump    TRUE
dump compression     ON
type of dump         fw-assisted
full memory dump     disallow

Next, use the lslv command to find information about this device.

# lslv lg_dumplv
LOGICAL VOLUME:     lg_dumplv              VOLUME GROUP:   rootvg
LV IDENTIFIER:      00f84bee00004c0000000179d4c6faac.11 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               sysdump                WRITE VERIFY:   off
MAX LPs:            512                    PP SIZE:        32 megabyte(s)
COPIES:             1                      SCHED POLICY:   parallel
LPs:                32                     PPs:            32
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       minimum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    32
MOUNT POINT:        N/A                    LABEL:          None
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?:     NO
INFINITE RETRY:     no                     PREFERRED READ: 0
 

NOTE: The type is sysdump. If you use mirrorvg to mirror rootvg the sysdump type keeps the dump device from being mirrored.

Next, multiply the values of LPs and PP SIZE to get the current size of the dump device in megabytes.

LPs * PP SIZE = Current size of dump device

For this example:

32 * 32 = 1024MB

To view an estimate of how large the dump device should be, use the estimate flag with sysdumpdev.  

# sysdumpdev -e
0453-041 Estimated dump size in bytes: 658841272 (628MB)

NOTE: This value is what the current running machine requires. This value changes based on the activity of the machine. It is best to run this command when the machine is under its heaviest work load.

The primary dump device should be a size that is greater than the value returned. AIX support recommends that the dump device size should be set to 1.5 times the estimate to allow for fluctuations in system memory usage.

If the dump device is a standard dump logical volume, such as lg_dumplv, then use the command extendlv to increase its size. If the dump device is primary paging space hd6, use the command chps


How to create a dedicated dump device

List the estimate of the dump size.
# sysdumpdev -e
0453-041 Estimated dump size in bytes: 658841272
List the PP size for the rootvg.
# lsvg rootvg
VOLUME GROUP:       rootvg                   VG IDENTIFIER:  00f84bee00004c0000000179d4c6faac
VG STATE:           active                   PP SIZE:        32 megabyte(s)
VG PERMISSION:      read/write               TOTAL PPs:      319 (10208 megabytes)
MAX LVs:            256                      FREE PPs:       183 (5856 megabytes)
LVs:                12                       USED PPs:       136 (4352 megabytes)
OPEN LVs:           11                       QUORUM:         2 (Enabled)
TOTAL PVs:          1                        VG DESCRIPTORS: 2
STALE PVs:          0                        STALE PPs:      0
ACTIVE PVs:         1                        AUTO ON:        no
MAX PPs per VG:     32512
MAX PPs per PV:     1016                     MAX PVs:        32
LTG size (Dynamic): 512 kilobyte(s)          AUTO SYNC:      no
HOT SPARE:          no                       BB POLICY:      relocatable
PV RESTRICTION:     none                     INFINITE RETRY: no
DISK BLOCK SIZE:    512                      CRITICAL VG:    no
FS SYNC OPTION:     no                       CRITICAL PVs:   no

 
NOTE: The PP SIZE and FREE PPs values are critical for estimating the size of the dump logical volume.
Divide the estimated size, multiplied by 1.5, by the PP size to estimate the proper number of PPs that the dump logical volume should have.
(ESTIMATED_SIZE * 1.5)/PP_SIZE = Dump Logical Volume PPs
For this example:
(658841272 * 1.5)/32 = 29.45 PPs
The PPs should be rounded up to 30.
The following example creates a permanent primary sysdump type device named lg_dumplv, which persists across reboots.
# mklv -y lg_dumplv -t sysdump rootvg 30
lg_dumplv
# sysdumpdev -Pp /dev/lg_dumplv

Tip: Do not dump to a mirrored logical volume

It is not recommended to mirror a stand-alone dump logical volume. It is a better practice to have a primary and a secondary dump device, each wholly contained on separate hdisks, rather than mirroring these devices. If the primary dump device is inaccessible, the dump program attempts to dump to the secondary device to a different storage device.  Firmware assisted dumps are only written to the primary dump device.  A secondary need not be configured.


Tip: Dumping outside the rootvg

To write a dump to a dump device outside of the rootvg:

A) Temporarily set the primary dump device to a non-rootvg logical volume.
B) Set the secondary device to a non-rootvg logical volume. 

Read more details in the sysdumpdev man page.

  • Local man page: 
    # man sysdumpdev
  • IBM Documentation 

[{"Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"","label":""}],"Platform":[{"code":"PF002","label":"AIX"}],"Version":"Version Independent"}]

Historical Number

isg1pTechnote0283

Document Information

Modified date:
26 March 2024

UID

isg3T1000169