IBM Support

IC70482: Instance abend and possible crash recovery failure on V9.7 Fix Pack 2 using circular logging.

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • This problem may manifest itself in numerous ways.
    As a first symptom , the instance will abend in  a variety of
    possible locations.
    Some known function in the stack traces are :
    
    1.
    
    sqlp_get_log_filepath_for_lsn
    
    db2diag.log entry :
    
    2010-07-01-01.13.49.945856+540 I14975442A499      LEVEL: Warning
    PID    : 1781890              TID  : 13624      PROC : db2sysc
    4
    INSTANCE: db2inst1            NODE : 004        DB  : DBNAME
    APPHDL  : 0-157                APPID: *N0.db2inst1.100729011146
    AUTHID  : DB2INST1
    EDUID  : 13624                EDUNAME: db2lload 4
    FUNCTION: DB2 UDB, RAS/PD component,
    pdEDUIsInDB2KernelOperation, probe:600
    DATA #1 : String, 33 bytes
    sqlp_get_log_filepath_for_lsn__F
    DATA #2 : String, 4 bytes
    sqlp
    
    2.
    
    sqlpgwlp
    sqlpgasn2
    
    
    2010-07-01-09.56.58.161004-240 I59812464E384      LEVEL:
    Warning
    PID    : 31052                TID  : 479683       PROC :
    db2sysc 0
    INSTANCE: db2inst1            NODE : 000
    EDUID  : 75                  EDUNAME: db2loggw (SAMPLE) 0
    FUNCTION: DB2 UDB, RAS/PD component,
    pdEDUIsInDB2KernelOperation, probe:600
    DATA #1 : String, 12 bytes
    _Z8sqlpgwlp
    DATA #2 : String, 4 bytes
    sqlp
    
    3.
    
    sqlpshrBuildRecord
    
    sqlpshrEdu
    
    
    
    2010-07-01-08.20.01.832248+540 I15631649A503      LEVEL: Warning
    PID    : 835836              TID  : 13880      PROC : db2sysc
    
    INSTANCE: db2inst1            NODE : 000        DB  : SAMPLE
    APPHDL  : 2-51                APPID: *N0.db2inst1.100802232000
    AUTHID  : DB2INST1
    EDUID  : 13880                EDUNAME: db2shred (SAMPLE)
    FUNCTION: DB2 UDB, RAS/PD component,
    pdEDUIsInDB2KernelOperation, probe:600
    DATA #1 : String, 27 bytes
    @109@sqlpshrBuildRecord__F
    DATA #2 : String, 4 bytes
    sqlp
    
    -
    
    sqlpshrBuildRecord
    
    sqlpshrEdu
    
    other traps are possible as well.
    
    
    A db2diag.log message which also occurs when the problem happens
    is the following :
    
    2010-07-01-09.46.35.712009+540 I13910007A467      LEVEL: Error
    PID    : 1417464              TID  : 3600        PROC : db2sysc
    4
    INSTANCE: db2inst1            NODE : 004        DB  : SAMPLE
    EDUID  : DB2INST1                EDUNAME: db2loggr (SAMPLE) 4
    FUNCTION: DB2 UDB, data protection services, sqlpgspr, probe:550
    DATA #1 : String, 117 bytes
    Reclaimed too much log space. Recalculating LogBytesInUse.
    Old LogBytesInUse: 74753276 New LogBytesInUse: 8124749448
    
    and following this :
    
    2010-07-01-07.06.17.705235+000 E34053A598         LEVEL: Severe
    PID     : 417804               TID  : 5142        PROC : db2sysc
    0
    INSTANCE: db2cbp               NODE : 000
    EDUID   : 5142                 EDUNAME: db2loggw (SAMPLE) 0
    FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk,
    probe:1010
    MESSAGE : ZRC=0x85100009=-2062548983=SQLP_NOSPACE
              "Log File has reached its saturation point"
              DIA8309C Log file was full.
    DATA #1 : <preformatted>
    Error getting next log file to write to. Filecount 20, active
    20, inactive 40, tailindex 18446744073709551596 currentRecord 16
    
    Note that this exhaust the available transaction log space (
    SQLP_NOSPACE error ) and will bring the database down.
    An eyecatcher is the very high tailindex value which , when
    converted to hex, is close to 0xFFFF FFFF FFFF FFFF.
    
    
    
    When subsequently crash recovery is attempted, this may fail
    with the following message during the undo phase :
    
    
    2010-12-20-09.34.11.751803+000 I5359A415          LEVEL: Info
    PID     : 12093                TID  : 13          PROC : db2sysc
    0
    INSTANCE: db2inst3             NODE : 000         DB   : SAMPLE
    EDUID   : 13                   EDUNAME: db2loggr (SAMPLE) 0
    FUNCTION: DB2 UDB, recovery manager, sqlpgSwitchFromRedoToUndo,
    probe:806
    DATA #1 : <preformatted>
    TailIndex 1  firstLso 39333401  nextLso 39349705  logGapSize 0
    
    
    2010-07-01-09.34.11.962973+000 I7301A532          LEVEL: Severe
    PID     : 12093                TID  : 14          PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000
    EDUID   : 14                   EDUNAME: db2loggw (SAMPLE) 0
    FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk,
    probe:909
    MESSAGE : ZRC=0x8610000D=-2045771763=SQLP_BADLOG "Log File
    cannot be used"
              DIA8414C Logging can not continue due to an error.
    DATA #1 : <preformatted>
    TailPage 0 does not match pagePso 39349778 and firstLso 39007321
    

Local fix

  • One of the following methods may be used to attempt to work
    around the issue:
    1) Restore database from a backup image
    2) Reset log control file(SQLOGCTL.LFH) and bypass Crash
    recovery. However, db2 should be rebuild after resetting log
    control file.
    
    2) Contact DB2 Support for assistance. Transaction log may be
    patched with internal tool.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * All                                                          *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * This APAR only applies to V9.7 FP2 database that use         *
    * circular logging.                                            *
    *                                                              *
    * When the database is  circular logging(sqlpgicl),            *
    * thefirstLso in FCB                                           *
    * array are all initialized to                                 *
    * baselso.2010-07-29-10.13.49.945856+540 I14975442A499         *
    * LEVEL:WarningPID    : 1781890              TID  : 13624      *
    * PROC :db2sysc 4INSTANCE: db2inst1            NODE : 004      *
    * DB  : XXXXXXAPPHDL  : 0-157                                  *
    * APPID:*N0.db2inst1.100729011146AUTHID  : XXXXXEDUID  : 13624 *
    * EDUNAME: db2lload 4FUNCTION: DB2 UDB, RAS/PD                 *
    * component,pdEDUIsInDB2KernelOperation, probe:600DATA #1 :    *
    * String, 33 bytessqlp_get_log_filepath_for_lsn__FDATA #2 :    *
    * String, 4 bytessqlpStacks<StackTrace>------Function +        *
    * Offset------sqlp_get_log_filepath_for_lsnsqluRegisterLoadEndca
    * + 0x260</StackTrace>                                         *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * (1) Upgrade to V9.7 Fix Pack 3, or,                          *
    * (2) Use archiving log                                        *
    ****************************************************************
    

Problem conclusion

  • Problem was first fixed in Version 9.7 Fix Pack 3.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IC70482

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    970

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    YesSpecatt / Pervasive

  • Submitted date

    2010-08-10

  • Closed date

    2010-09-23

  • Last modified date

    2011-10-05

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

    IC76438 IC77551

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

  • R970 PSN

       UP



Document information

More support for: DB2 for Linux, UNIX and Windows

Software version: 9.7

Reference #: IC70482

Modified date: 05 October 2011