IBM Support

IC78019: HADR STANDBY DATABASE CAN BE BROUGHT DOWN DUE TO A BADPAGE ERROR AFTER SUCCESSFUL REINTEGRATION

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • If the following steps happen in order, hadr standby database
    could be brought down.
    1. HADR is in peer state
    
    
    2. A forced takeover is executed on standby while active
    transactions exists on table <tab1> on primary and the below
    message is logged into the diaglog:
    
    2011-08-08-01.09.27.277225-420 I65530E321         LEVEL: Warning
    PID     : 20672                TID  : 47731157748560PROC :
    db2hadrs (HADRDB)
    INSTANCE: db2inst1                NODE : 000
    
    FUNCTION: DB2 UDB, High Availability Disaster Recovery,
    hdrEduAcceptEvent, probe:20214
    MESSAGE : Time out waiting for primary end of log
    
    3. New primary inserts rows or updates rows on table <tab1>
    
    
    4. Old Primary reintegrates as new standby successfully.
    
    
    5. New standby will shut down when it tries to replay the data
    changes on table tab1 with BADPAGE error.
    Diag messages similar to these will appear in the db2diag.log:
    
    2011-08-08-01.11.23.845061-420 I80277E563         LEVEL: Severe
    PID     : 19982                TID  : 47731157748560PROC :
    db2redow (HADRDB)
    INSTANCE: db2inst1                NODE : 000         DB   :
    HADRDB
    APPHDL  : 0-68                 APPID: *LOCAL.DB2.110808081043
    FUNCTION: DB2 UDB, data management, sqldRedoUpsert, probe:1884
    MESSAGE : Free Space does not match during redo!
    DATA #1 : Hexdump, 26 bytes
    0x0000000228115658 : 01A2 0200 0400 0000 0D00 2E0B 0500 8000
    ................
    0x0000000228115668 : 0000 3A0B 0000 0D00 0100
    ..:.......
    
    2011-08-08-01.11.23.845277-420 I80841E448         LEVEL: Severe
    PID     : 19982                TID  : 47731157748560PROC :
    db2redow (HADRDB)
    INSTANCE: db2inst1                NODE : 000         DB   :
    HADRDB
    APPHDL  : 0-68                 APPID: *LOCAL.DB2.110808081043
    FUNCTION: DB2 UDB, data management, sqldRedoUpsert, probe:1887
    MESSAGE : Space used:
    DATA #1 : Hexdump, 4 bytes
    0x00007FFF6D0D0088 : 1100 0000
    ....
    
    2011-08-08-01.11.23.845351-420 I81290E464         LEVEL: Severe
    PID     : 19982                TID  : 47731157748560PROC :
    db2redow (HADRDB)
    INSTANCE: db2inst1                NODE : 000         DB   :
    HADRDB
    APPHDL  : 0-68                 APPID: *LOCAL.DB2.110808081043
    FUNCTION: DB2 UDB, data management, sqldRedoUpsert, probe:1892
    RETCODE : ZRC=0x87040001=-2029780991=SQLD_BADPAGE "Bad Data
    Page"
              DIA8500C A data file error has occurred, record id is
    "".
    
    2011-08-08-01.11.23.899138-420 E98543E372         LEVEL: Severe
    PID     : 19982                TID  : 47731157748560PROC :
    db2redow (HADRDB)
    INSTANCE: db2inst1                NODE : 000         DB   :
    HADRDB
    APPHDL  : 0-68                 APPID: *LOCAL.DB2.110808081043
    FUNCTION: DB2 UDB, base sys utilities, sqleMarkDBad, probe:10
    MESSAGE : ADM7518C  "HADRDB  " marked bad.
    
    2011-08-08-01.11.23.899237-420 I98916E385         LEVEL: Severe
    PID     : 19982                TID  : 47731157748560PROC :
    db2redow (HADRDB)
    INSTANCE: db2inst1                NODE : 000         DB   :
    HADRDB
    APPHDL  : 0-68                 APPID: *LOCAL.DB2.110808081043
    FUNCTION: DB2 UDB, base sys utilities, sqleMarkDBad, probe:210
    MESSAGE : Database logging stopped due to mark db bad.
    
    2011-08-08-01.11.23.911552-420 I100194E458        LEVEL: Severe
    PID     : 19982                TID  : 47731157748560PROC :
    db2redow (HADRDB)
    INSTANCE: db2inst1                NODE : 000         DB   :
    HADRDB
    APPHDL  : 0-68                 APPID: *LOCAL.DB2.110808081043
    FUNCTION: DB2 UDB, data management, sqldRedo, probe:6291
    RETCODE : ZRC=0x87040001=-2029780991=SQLD_BADPAGE "Bad Data
    Page"
              DIA8500C A data file error has occurred, record id is
    "".
    
    2011-08-08-01.11.23.911683-420 I100653E457        LEVEL: Severe
    PID     : 19982                TID  : 47731157748560PROC :
    db2redow (HADRDB)
    INSTANCE: db2inst1                NODE : 000         DB   :
    HADRDB
    APPHDL  : 0-68                 APPID: *LOCAL.DB2.110808081043
    FUNCTION: DB2 UDB, data management, sqldmrdo, probe:783
    RETCODE : ZRC=0x87040001=-2029780991=SQLD_BADPAGE "Bad Data
    Page"
              DIA8500C A data file error has occurred, record id is
    "".
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * DB2 HADR users on all platforms                              *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * Without the fix, customer could be exposed to the problem in *
    * the error description                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Upgrade to v95fp9                                            *
    ****************************************************************
    

Problem conclusion

  • After upgarding to v95fp9, reintegration will be successful, and
    customer will not hit the error
    

Temporary fix

Comments

APAR Information

  • APAR number

    IC78019

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    950

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2011-08-10

  • Closed date

    2012-03-08

  • Last modified date

    2012-03-08

  • APAR is sysrouted FROM one or more of the following:

    IC77975

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

  • R950 PSY

       UP



Document information

More support for: DB2 for Linux, UNIX and Windows

Software version: 9.5

Reference #: IC78019

Modified date: 08 March 2012