IBM Support

Recover from ANR0162W and SQL0290N during server start

Troubleshooting


Problem

An IBM Spectrum Protect server might fail to start with the following error messages: ANR0162W Supplemental database diagnostic information: -1:55039:-290 ([IBM][CLI Driver][DB2/NT64] SQL0290N Table space access is not allowed. SQLSTATE=55039 ). ANR3614E The server failed to connect to the database.

Symptom

The complete error message stack looks like:
An IBM Spectrum Protect server might fail to start with the following error messages:

ANR0900I Processing options file C:\Program Files\Tivoli\TSM\tsminst1\dsmserv.-
opt.
ANR4726I The ICC support module has been loaded.
ANR0990I Server restart-recovery in progress.
ANR0152I Database manager successfully started.
ANR9999D_3831306406 ReportSQLDiagInfo(dbieval.c:1572) Thread<0>: Missing
sqlState=55039, sqlCode=-290 from table. Returning rc = 9994.
ANR9999D Thread<0> issued message 9999 from:
ANR9999D Thread<0> 00007FFCD2805E45 OutDiagToCons()+155
ANR9999D Thread<0> 00007FFCD27FFD7C outDiagfExt()+10c
ANR9999D Thread<0> 00007FFCD2597970 ReportSQLDiagInfo()+2a0
ANR9999D Thread<0> 00007FFCD2596950 DbiEvalSQLOutcomeX()+4f0
ANR9999D Thread<0> 00007FFCD25B307B RdbTestConnectionToDB2X()+37b
ANR9999D Thread<0> 00007FFCD25057E4 dbiInit()+a34
ANR9999D Thread<0> 00007FFCD1EF090C StartServer()+2bc
ANR9999D Thread<0> 00007FFCD1EEC1DF admStartServer()+ff
ANR9999D Thread<0> 00007FFCD1ED23FF adsmMain()+134f
ANR9999D Thread<0> 00007FF621B5110D main()+8d dsmserv.c:241
ANR9999D Thread<0> 00007FF621B5271B __tmainCRTStartup()+10f crtexe.c:536
ANR9999D Thread<0> 00007FFCFA6C16AD BaseThreadInitThunk()+d
ANR9999D Thread<0> 00007FFCFCB954E4 RtlUserThreadStart()+34
ANR0162W Supplemental database diagnostic information: -1:55039:-290
([IBM][CLI Driver][DB2/NT64] SQL0290N Table space access is not allowed.
SQLSTATE=55039
).
ANR3614E The server failed to connect to the database.

Cause

A review of the db2diag.log file shows that an error condition puts a server database tablespace into ROLLFORWARD pending.

Diagnosing The Problem

2017-08-03-12.01.07.670000+060 I19835161F748 LEVEL: Error
PID : 19920 TID : 5620 PROC : db2syscs.exe
INSTANCE: TSMINST1 NODE : 000 DB : TSMDB1
APPHDL : 0-9 APPID: *LOCAL.TSMINST1.170803110056
AUTHID : TSMINST2 HOSTNAME: PTB3TSM01
EDUID : 5620 EDUNAME: db2agent (TSMDB1)
FUNCTION: DB2 UDB, buffer pool services, sqlbDMSDoContainerOp, probe:810
MESSAGE : ZRC=0x870F0016=-2029060074=SQLO_SHAR "File sharing violation."
DIA8519C A file sharing violation has occurred, filename was "".
DATA #1 : <preformatted>
Error checking container 1 (F:\dbs02\TSMINST1\NODE0000\TSMDB1\T0000002\C0000001.LRG) for tbsp 2. Rc = 870F0016

2017-08-03-12.01.07.670000+060 I19835911F657 LEVEL: Error
PID : 19920 TID : 5620 PROC : db2syscs.exe
INSTANCE: TSMINST1 NODE : 000 DB : TSMDB1
APPHDL : 0-9 APPID: *LOCAL.TSMINST1.170803110056
AUTHID : TSMINST2 HOSTNAME: PTB3TSM01
EDUID : 5620 EDUNAME: db2agent (TSMDB1)
FUNCTION: DB2 UDB, buffer pool services, sqlbDMSStartPool, probe:800
MESSAGE : ZRC=0x870F0016=-2029060074=SQLO_SHAR "File sharing violation."
DIA8519C A file sharing violation has occurred, filename was "".
DATA #1 : <preformatted>
Tablespace 2 (USERSPACE1)

2017-08-03-12.01.07.670000+060 E19836570F646 LEVEL: Error
PID : 19920 TID : 5620 PROC : db2syscs.exe
INSTANCE: TSMINST1 NODE : 000 DB : TSMDB1
APPHDL : 0-9 APPID: *LOCAL.TSMINST1.170803110056
AUTHID : TSMINST2 HOSTNAME: PTB3TSM01
EDUID : 5620 EDUNAME: db2agent (TSMDB1)
FUNCTION: DB2 UDB, buffer pool services, sqlbStartPools, probe:2428
MESSAGE : ADM6023I The table space "USERSPACE1" (ID "2") is in state
"0x00000000". The table space cannot be accessed. Refer to the
documentation for SQLCODE -290.

2017-08-03-12.01.07.670000+060 I19837218F567 LEVEL: Warning
PID : 19920 TID : 5620 PROC : db2syscs.exe
INSTANCE: TSMINST1 NODE : 000 DB : TSMDB1
APPHDL : 0-9 APPID: *LOCAL.TSMINST1.170803110056
AUTHID : TSMINST2 HOSTNAME: PTB3TSM01
EDUID : 5620 EDUNAME: db2agent (TSMDB1)
FUNCTION: DB2 UDB, data protection services, sqlpRecoveryLSN, probe:1260
MESSAGE : generate logAtBkp for LSN:
DATA #1 : SQLP_LSN8, PD_TYPE_SQLP_LSN8, 8 bytes
000000013DA19122

2017-08-03-12.01.07.685000+060 E19837787F615 LEVEL: Error
PID : 19920 TID : 5620 PROC : db2syscs.exe
INSTANCE: TSMINST1 NODE : 000 DB : TSMDB1
APPHDL : 0-9 APPID: *LOCAL.TSMINST1.170803110056
AUTHID : TSMINST2 HOSTNAME: PTB3TSM01
EDUID : 5620 EDUNAME: db2agent (TSMDB1)
FUNCTION: DB2 UDB, buffer pool services, sqlbStartPoolsErrorHandling, probe:39
MESSAGE : ADM6080E The table space "USERSPACE1" (ID "2"), was put OFFLINE and
in ROLLFORWARD_PENDING. Tablespace state is "0x00004080".


Later this symptom is documented in the db2diag.log file as:

2017-08-09-14.21.10.640000+060 I23248183F642 LEVEL: Warning
PID : 5288 TID : 5164 PROC : db2syscs.exe
INSTANCE: TSMINST1 NODE : 000 DB : TSMDB1
APPHDL : 0-56 APPID: *LOCAL.TSMINST1.170809132147
AUTHID : TSMINST2 HOSTNAME: PTB3TSM01
EDUID : 5164 EDUNAME: db2agent (TSMDB1)
FUNCTION: DB2 UDB, buffer pool services, sqlbIsAllowed, probe:1075
MESSAGE : ZRC=0x80020033=-2147352525=SQLB_NOT_ALLOWED_ROLLFORWARD_P
"Access not allowed. Tblspc RF Pending."
DATA #1 : <preformatted>
Tablespace 2 : state= 0x80, action= 0.

Here tablespace 2 translates to USERSPACE1 and state 0x80 translates to ROLLFORWARD_PENDING.

Resolving The Problem

Sign on to the system as instance user, on a Windows system then do:
start -> run -> db2cmd

db2stop force
db2start
db2 connect to tsmdb1
.
db2 "rollforward db tsmdb1 to end of logs tablespace(USERSPACE1) online"
.
To compile the rollforward command for the correct tablespace consult the db2diag.log file for the tablespace to proceed with.
.
If this does not work you can try to restore the database to the most current state.
.
See the related links section for instructions for DB restore.

If RESTORE DB to its most current state(rollforward) does not work do a restore DB PIT (point-in-time). For that condition follow the instructions from the point-in-time procedure (see related links section).

Related Information

[{"Product":{"code":"SSEQVQ","label":"IBM Spectrum Protect"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"Server","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Supported Versions","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
17 June 2018

UID

swg22007399