Troubleshooting
Problem
On environments where total virtual mapping limit (RLIMIT_AS, ulimit -v) is enforced, one might notice that db2pd is not able to display database-related information: $ db2pd -db test - Database TEST not activated on database partition 0. Or, this local database name cannot be found. Please check the database name.
Symptom
Db2pd is unable to display database-related information:
$ db2pd -db test -
Database TEST not activated on database partition 0. Or, this local database name cannot be found. Please check the database name.
despite the fact database is up-and-running, it is able to accept the connection and other monitoring interfaces (like snapshots) are working fine.
$ db2pd -db test -
Database TEST not activated on database partition 0. Or, this local database name cannot be found. Please check the database name.
despite the fact database is up-and-running, it is able to accept the connection and other monitoring interfaces (like snapshots) are working fine.
Cause
Db2pd utility is working by attaching (shmat) to shared memory segment of the instance (DBMS) and database in order display requested information. As the result, instance/database memory must be temporarily mapped to db2pd's address space. If in given environment the size of total memory mapped by any process is limited (RLIMIT_AS/ulimit -v), operating system will reject the request of db2pd to attach to Db2 engine's memory and the command will fail.
This will not affect Db2 engine (db2sysc) because RLIMIT_AS is internally overridden during db2start, but will be effective for other tools started from the environment of the instance owner.
This will not affect Db2 engine (db2sysc) because RLIMIT_AS is internally overridden during db2start, but will be effective for other tools started from the environment of the instance owner.
Diagnosing The Problem
Db2pd run with -debug switch, e.g.:
$ db2pd -db test - -debug db2pd.debut.out
will show an error from pdAttachToDatabase function:
db2pd.C:591:41 pdCBp->pDBList: 0x7fffea865620
timestamp: 0 1517236973
db2pd_util_fe.C:6571:10 pdAttachToDatabase enter
timestamp: 0 1517236973
db2pd_util_fe.C:6619:30 Failure: sqloConnectIfNotConnected rc = -2062614523
If run with syscall trace (strace/truss), one would see an ENOMEM error from shmat system call, e.g.
$ strace -e shmat,smget db2pd -db test -
shmget(0xc191c274, 0, 0) = 1600487425
shmat(1600487425, 0, SHM_RND) = ?
shmget(0xc191c261, 0, 0700) = 1600520195
shmat(1600520195, 0x200000000, 0) = ?
shmat(1861550126, 0x7fefd65f6000, 0) = ?
shmat(644481033, 0x7fcd48796000, 0) = -1 ENOMEM (Cannot allocate memory)
shmat(644481033, 0, 0) = -1 ENOMEM (Cannot allocate memory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
Once error like this is seen, one should verify if user running the command doesn't have a limit on the amount of memory that any given process can map (RLIMIT_AS)
$ ulimit -v
virtual memory (kbytes, -v) 112233920
If happens to be lower than the size of Db2's shared memory, ENOMEM error is expected and OS limits should be adjusted.
This is a distinct problem from db2pd not being able to attach to memory due to an address space conflict with ALSR enabled, see:
http://www-01.ibm.com/support/docview.wss?uid=swg21365583
for details.
$ db2pd -db test - -debug db2pd.debut.out
will show an error from pdAttachToDatabase function:
db2pd.C:591:41 pdCBp->pDBList: 0x7fffea865620
timestamp: 0 1517236973
db2pd_util_fe.C:6571:10 pdAttachToDatabase enter
timestamp: 0 1517236973
db2pd_util_fe.C:6619:30 Failure: sqloConnectIfNotConnected rc = -2062614523
If run with syscall trace (strace/truss), one would see an ENOMEM error from shmat system call, e.g.
$ strace -e shmat,smget db2pd -db test -
shmget(0xc191c274, 0, 0) = 1600487425
shmat(1600487425, 0, SHM_RND) = ?
shmget(0xc191c261, 0, 0700) = 1600520195
shmat(1600520195, 0x200000000, 0) = ?
shmat(1861550126, 0x7fefd65f6000, 0) = ?
shmat(644481033, 0x7fcd48796000, 0) = -1 ENOMEM (Cannot allocate memory)
shmat(644481033, 0, 0) = -1 ENOMEM (Cannot allocate memory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
Once error like this is seen, one should verify if user running the command doesn't have a limit on the amount of memory that any given process can map (RLIMIT_AS)
$ ulimit -v
virtual memory (kbytes, -v) 112233920
If happens to be lower than the size of Db2's shared memory, ENOMEM error is expected and OS limits should be adjusted.
This is a distinct problem from db2pd not being able to attach to memory due to an address space conflict with ALSR enabled, see:
http://www-01.ibm.com/support/docview.wss?uid=swg21365583
for details.
Resolving The Problem
Refer to OS documentation how to increase the limit. If is only a softlimit:
$ ulimit -Sv
112233920
and hard limit is not enforced:
$ ulimit -Hv
unlimited
One can increase the limit for current session by running:
$ ulimit -v ulimited
and check if this resolves the problem. For hard limits, an change by OS administrator will be required.
$ ulimit -Sv
112233920
and hard limit is not enforced:
$ ulimit -Hv
unlimited
One can increase the limit for current session by running:
$ ulimit -v ulimited
and check if this resolves the problem. For hard limits, an change by OS administrator will be required.
Related Information
[{"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"DB2 Tools - db2pd","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"}],"Version":"9.7;10.1;10.5;11.1","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Was this topic helpful?
Document Information
Modified date:
21 September 2020
UID
swg22013520