Troubleshooting
Problem
Paging space growth might occur unexpectedly on DB2® or AIX® systems with 64K (medium) pages enabled.
Symptom
For further discussion on this topic, visit this developerWorks forum thread:
https://www.ibm.com/developerworks/community/forums/html/topic?id=ef5abd8f-2b59-4aa5-b32e-3498163692e1
Unexpected growth in paging space utilization might occur on a DB2 or AIX system where sufficient memory exists to keep DB2 application memory in RAM. This might be observed on DB2 or AIX systems using medium (64K) memory pages. While the performance effects from the paging itself should be negligible, paging space might become exhausted over time. This condition typically results in an outage, such as system hang or terminated applications.
Cause
Two AIX defects have been identified. This problem is resolved by AIX APAR IZ67445 - 64K PAGING TAKING PLACE WHEN AVAILABLE SYSTEM RAM EXISTS (affects AIX 5.3 and AIX 6.1). and AIX HIPER APAR IZ92561 - 64K KERNEL HEAP CAUSES PAGING WHEN RAM IS OTHERWISE AVAILABLE (impacts AIX 6.1 only)
Environment
This problem affects systems running the following:
- DB2 9 or later
- AIX 5.3 TL05 or later OR any level of AIX 6.1
- p5+ hardware or higher
DB2 product uses 64K memory pages for a large portion of its memory requirements in these environments.
Diagnosing The Problem
Monitor the system with the vmstat command.
Active virtual memory (avm) values show that the system is not overcommited, yet page-outs can be seen in the po column .
[112]bugdbug /home/mcornish > vmstat -t 1
System configuration: lcpu=16 mem=65536MB
<snip>
kthr memory page
----- -------------- -----------------------------------
r b avm fre re pi po fr sr cy
2 1 9286486 5691 7435 1456 0 0 9910 17488
4 2 9244576 39698 6456 3025 21 71 8466 18006
4 1 9236325 5057 5791 316 1 0 5215 12038
The "-P all" option shows that 64K pages are enabled:
[114]bugdbug /home/mcornish > vmstat -P all
System configuration: mem=65536MB
pgsz memory page
----- -------------------------- ------------------------------------
siz avm fre re pi po fr sr cy
4K 7087040 6605456 76048 0 0 0 3 11 0
64K 443136 699104 5712 0 0 0 0 0 0
Resolving The Problem
Upgrade to an AIX service level containing the APARs. The following workarounds will result in DB2 using 4K memory pages, avoiding the unexpected paging scenario. Note that performance may degrade in the range of 0-10%. The impact might be at the higher end of the range in CPU-bound systems. If the system is not a CPU-bound system, the impact is expected to be insignificant.
In order to determine if 64K pages are available on a given system, issue the command "vmstat -P all" and check if there is a row for 64K pages. If there is only one row for 4K pages, 64K pages are not available and the system is not vulnerable to this problem.
Note that APAR IZ67445 is specific to AIX 5.3 TL08, and IZ92561 is specific to AIX 6.1 TL03. Each affected AIX technology level has a unique APAR. See the following technote for details: Known issues for DB2 for Linux, UNIX and Windows on AIX 5.2, 5.3, 6.1, and 7.1.
CONFIGURING 4K PAGES FOR SHARED MEMORY:
As of DB2 Version 9.1, DB2 uses 64K (medium) pages for shared memory on AIX platforms, when available. The following command modifies DB2 to use 4K memory pages for shared memory:
db2set DB2_MEDIUM_PAGE_SUPPORT=OFF
Changes to this registry variable take effect on the next instance recycle (db2stop;db2start). Reset the registry variable to re-enable 64K pages for shared memory.
CONFIGURING 4K PAGES FOR PRIVATE MEMORY:
DB2 Version 9.5 and later also use 64K (medium) pages for private memory, when available.
In order to configure 4K pages for private memory on partitioned database environments, perform the following steps:
1. Run the following command: db2set DB2ENVLIST=LDR_CNTRL
2. Add "export LDR_CNTRL=DATAPSIZE=4K" to sqllib/userprofile
This takes effect on the next instance recycle (db2stop;db2start). To re-enable 64K pages, remove the above settings and recycle the instance.
1. Issue the command: db2set DB2ENVLIST=
2. Remove or comment out the "export LDR_CNTRL" entry in sqllib/userprofile
In order to configure DB2 to use 4K pages for private memory on a single partition system, issue the following command as root:
ldedit -bdatapsize=4K <DB2 instance home directory>/sqllib/adm/db2sysc
The command should return the following output:
ldedit: File db2sysc updated.
The instance must be stopped in order to issue this command.
Note that the pagesize modification must be repeated any time the db2sysc program is replaced. This occurs whenever the db2iupdt command is issued, for example after an upgrade, or an instance migration occurs, for example when the db2imigr command is issued.
In order to re-enable 64K pages for private memory (the default), issue the following command:
ldedit -bdatapsize=64K db2sysc
ALTERNATIVE WORKAROUND:
An alternative workaround is to increase paging space to accommodate the growth in paging space usage. Unless peak workloads cause a system to become overcommitted, configuring paging space equivalent to the amount of RAM should be sufficient to handle the worst case.
For further discussion on this topic, visit this developerWorks forum thread:
https://www.ibm.com/developerworks/community/forums/html/topic?id=ef5abd8f-2b59-4aa5-b32e-3498163692e1
Was this topic helpful?
Document Information
Modified date:
16 June 2018
UID
swg21417752