IBM Support

IC80745: DB2LOGTS DAEMON ON HADR PRIMARY COULD BE BROUGHT DOWN DUE TO LOG INFO ARRAY BEING OUT OF SYNC

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as fixed if next.

Error description

  • On HADR primary, db2logts daemon on HADR primary could be
    brought down due to log info array being out of sync.
    The issue can be hit when following events happened:
    
    1. HADR standby restarts from last byte of a log file. When this
    event happens, we can see below message in db2diag.log:
    
    2011-12-13-18.09.58.511431+540 E920335A494        LEVEL: Warning
    PID     : 9699938              TID  : 71547       PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    APPHDL  : 0-2523               APPID: *LOCAL.DB2.111213092048
    EDUID   : 71547                EDUNAME: db2shred (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager, sqlpshrEdu, probe:45320
    DATA #1 : <preformatted>
    Scan starting from the last byte of a truncated log file. Moving
    to next file.2130982527201
    
    2. Takeover happens on the standby, and it becomes the new
    primary.
    
    3. Transactions continue and new logs are generated on new
    primary. Finally, when log info array is full and DB2 trys to
    reuse one entry, the error will be hit, and following messages
    will appear in db2diag.log:
    
    2011-12-15-00.13.22.849529+540 I2986417A454       LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:64
    MESSAGE : Extent number:
    DATA #1 : Hexdump, 4 bytes
    0x070000001EFFD460 : 0000 6E1E
    ..n.
    
    2011-12-15-00.13.22.849903+540 I2986872A469       LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:65
    MESSAGE : Entry number (readEntry):
    DATA #1 : Hexdump, 8 bytes
    0x07000008E0027228 : 0000 0000 0000 0034
    .......4
    
    2011-12-15-00.13.22.850071+540 I2987342A483       LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:66
    MESSAGE : read entry extent number (readEntryExtNum):
    DATA #1 : Hexdump, 4 bytes
    0x07000008E0027230 : 0000 6D14
    ..m.
    
    2011-12-15-00.13.22.850240+540 I2987826A1084      LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:67
    DATA #1 : String, 128 bytes
    The db2logts log info array is out of sync.
    Bringing down the db2logts daemon.
    From this point on, LOG SKIPPING is DISABLED.
    
    DATA #2 : Hexdump, 112 bytes
    0x07000008E0029BA0 : 0000 6D12 0000 0000 0000 0997 EAB2 8001
    ..m.............
    0x07000008E0029BB0 : 0000 0000 0000 0000 0000 0016 0000 0000
    ................
    0x07000008E0029BC0 : 0000 0000 0000 0000 0000 0000 0165 0000
    .............e..
    0x07000008E0029BD0 : 0700 000D 0650 AF80 0700 000D 0650 B1A0
    .....P.......P..
    0x07000008E0029BE0 : 0700 000D 0650 B3C0 0700 000D 0650 B5E0
    .....P.......P..
    0x07000008E0029BF0 : 0700 000D 0650 B800 0700 000D 0650 5460
    .....P.......PT`
    0x07000008E0029C00 : 0700 000D 0650 5680 0700 000D 0650 58A0
    .....PV......PX.
    
    2011-12-15-00.13.22.850416+540 I2988911A165       LEVEL: Severe
    PID:9699938 TID:5798 NODE:000 Title: SQLP_DBCB
    Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin
    
    2011-12-15-00.13.22.907328+540 I2989077A166       LEVEL: Severe
    PID:9699938 TID:5798 NODE:000 Title: SQLPG_XHDR
    Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin
    
    2011-12-15-00.13.22.907523+540 I2989244A375       LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:0
    RETCODE : ZRC=0x00000000=0=PSM_OK "Unknown"
    

Local fix

  • NA
    

Problem summary

  • On HADR primary, db2logts daemon on HADR primary could be
    brought down due to log info array being out of sync.
    The issue can be hit when following events happened:
    
    1. HADR standby restarts from last byte of a log file. When this
    event happens, we can see below message in db2diag.log:
    
    2011-12-13-18.09.58.511431+540 E920335A494        LEVEL: Warning
    PID     : 9699938              TID  : 71547       PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    APPHDL  : 0-2523               APPID: *LOCAL.DB2.111213092048
    EDUID   : 71547                EDUNAME: db2shred (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager, sqlpshrEdu, probe:45320
    DATA #1 : <preformatted>
    Scan starting from the last byte of a truncated log file. Moving
    to next file.2130982527201
    
    2. Takeover happens on the standby, and it becomes the new
    primary.
    
    3. Transactions continue and new logs are generated on new
    primary. Finally, when log info array is full and DB2 trys to
    reuse one entry, the error will be hit, and following messages
    will appear in db2diag.log:
    
    2011-12-15-00.13.22.849529+540 I2986417A454       LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:64
    MESSAGE : Extent number:
    DATA #1 : Hexdump, 4 bytes
    0x070000001EFFD460 : 0000 6E1E
    ..n.
    
    2011-12-15-00.13.22.849903+540 I2986872A469       LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:65
    MESSAGE : Entry number (readEntry):
    DATA #1 : Hexdump, 8 bytes
    0x07000008E0027228 : 0000 0000 0000 0034
    .......4
    
    2011-12-15-00.13.22.850071+540 I2987342A483       LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:66
    MESSAGE : read entry extent number (readEntryExtNum):
    DATA #1 : Hexdump, 4 bytes
    0x07000008E0027230 : 0000 6D14
    ..m.
    
    2011-12-15-00.13.22.850240+540 I2987826A1084      LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:67
    DATA #1 : String, 128 bytes
    The db2logts log info array is out of sync.
    Bringing down the db2logts daemon.
    From this point on, LOG SKIPPING is DISABLED.
    
    DATA #2 : Hexdump, 112 bytes
    0x07000008E0029BA0 : 0000 6D12 0000 0000 0000 0997 EAB2 8001
    ..m.............
    0x07000008E0029BB0 : 0000 0000 0000 0000 0000 0016 0000 0000
    ................
    0x07000008E0029BC0 : 0000 0000 0000 0000 0000 0000 0165 0000
    .............e..
    0x07000008E0029BD0 : 0700 000D 0650 AF80 0700 000D 0650 B1A0
    .....P.......P..
    0x07000008E0029BE0 : 0700 000D 0650 B3C0 0700 000D 0650 B5E0
    .....P.......P..
    0x07000008E0029BF0 : 0700 000D 0650 B800 0700 000D 0650 5460
    .....P.......PT`
    0x07000008E0029C00 : 0700 000D 0650 5680 0700 000D 0650 58A0
    .....PV......PX.
    
    2011-12-15-00.13.22.850416+540 I2988911A165       LEVEL: Severe
    PID:9699938 TID:5798 NODE:000 Title: SQLP_DBCB
    Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin
    
    2011-12-15-00.13.22.907328+540 I2989077A166       LEVEL: Severe
    PID:9699938 TID:5798 NODE:000 Title: SQLPG_XHDR
    Dump File:/db2dump/db2inst1/9699938.5798.000.dump.bin
    
    2011-12-15-00.13.22.907523+540 I2989244A375       LEVEL: Error
    PID     : 9699938              TID  : 5798        PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000         DB   : HADRDB
    EDUID   : 5798                 EDUNAME: db2loggr (HADRDB) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpInformLogTsAboutNewLogHeader, probe:0
    RETCODE : ZRC=0x00000000=0=PSM_OK "Unknown"
    

Problem conclusion

Temporary fix

Comments

APAR Information

  • APAR number

    IC80745

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    970

  • Status

    CLOSED FIN

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2012-01-10

  • Closed date

    2013-04-19

  • Last modified date

    2013-04-19

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

    IC85207

Fix information

Applicable component levels

  • R970 PSY

       UP



Document information

More support for: DB2 for Linux, UNIX and Windows

Software version: 9.7

Reference #: IC80745

Modified date: 19 April 2013