IBM Support

IC76438: Instance abend and possible crash recovery failure on V9.7 Fix Pack 2 using circular logging.

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • This problem may manifest itself in numerous ways.
    As a first symptom , the instance will abend in  a variety of
    possible locations.
    Some known function in the stack traces are :
    
    1.
    
    sqlp_get_log_filepath_for_lsn
    
    db2diag.log entry :
    
    2010-07-01-01.13.49.945856+540 I14975442A499      LEVEL: Warning
    PID    : 1781890              TID  : 13624      PROC : db2sysc
    4
    INSTANCE: db2inst1            NODE : 004        DB  : DBNAME
    APPHDL  : 0-157                APPID: *N0.db2inst1.100729011146
    AUTHID  : DB2INST1
    EDUID  : 13624                EDUNAME: db2lload 4
    FUNCTION: DB2 UDB, RAS/PD component,
    pdEDUIsInDB2KernelOperation, probe:600
    DATA #1 : String, 33 bytes
    sqlp_get_log_filepath_for_lsn__F
    DATA #2 : String, 4 bytes
    sqlp
    
    2.
    
    sqlpgwlp
    sqlpgasn2
    
    
    2010-07-01-09.56.58.161004-240 I59812464E384      LEVEL:
    Warning
    PID    : 31052                TID  : 479683       PROC :
    db2sysc 0
    INSTANCE: db2inst1            NODE : 000
    EDUID  : 75                  EDUNAME: db2loggw (SAMPLE) 0
    FUNCTION: DB2 UDB, RAS/PD component,
    pdEDUIsInDB2KernelOperation, probe:600
    DATA #1 : String, 12 bytes
    _Z8sqlpgwlp
    DATA #2 : String, 4 bytes
    sqlp
    
    3.
    
    sqlpshrBuildRecord
    
    sqlpshrEdu
    
    
    
    2010-07-01-08.20.01.832248+540 I15631649A503      LEVEL: Warning
    PID    : 835836              TID  : 13880      PROC : db2sysc
    
    INSTANCE: db2inst1            NODE : 000        DB  : SAMPLE
    APPHDL  : 2-51                APPID: *N0.db2inst1.100802232000
    AUTHID  : DB2INST1
    EDUID  : 13880                EDUNAME: db2shred (SAMPLE)
    FUNCTION: DB2 UDB, RAS/PD component,
    pdEDUIsInDB2KernelOperation, probe:600
    DATA #1 : String, 27 bytes
    @109@sqlpshrBuildRecord__F
    DATA #2 : String, 4 bytes
    sqlp
    
    -
    
    sqlpshrBuildRecord
    
    sqlpshrEdu
    
    other traps are possible as well.
    
    
    A db2diag.log message which also occurs when the problem happens
    is the following :
    
    2010-07-01-09.46.35.712009+540 I13910007A467      LEVEL: Error
    PID    : 1417464              TID  : 3600        PROC : db2sysc
    4
    INSTANCE: db2inst1            NODE : 004        DB  : SAMPLE
    EDUID  : DB2INST1                EDUNAME: db2loggr (SAMPLE) 4
    FUNCTION: DB2 UDB, data protection services, sqlpgspr, probe:550
    DATA #1 : String, 117 bytes
    Reclaimed too much log space. Recalculating LogBytesInUse.
    Old LogBytesInUse: 74753276 New LogBytesInUse: 8124749448
    
    and following this :
    
    2010-07-01-07.06.17.705235+000 E34053A598         LEVEL: Severe
    PID     : 417804               TID  : 5142        PROC : db2sysc
    0
    INSTANCE: db2cbp               NODE : 000
    EDUID   : 5142                 EDUNAME: db2loggw (SAMPLE) 0
    FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk,
    probe:1010
    MESSAGE : ZRC=0x85100009=-2062548983=SQLP_NOSPACE
              "Log File has reached its saturation point"
              DIA8309C Log file was full.
    DATA #1 : <preformatted>
    Error getting next log file to write to. Filecount 20, active
    20, inactive 40, tailindex 18446744073709551596 currentRecord 16
    
    Note that this exhaust the available transaction log space (
    SQLP_NOSPACE error ) and will bring the database down.
    An eyecatcher is the very high tailindex value which , when
    converted to hex, is close to 0xFFFF FFFF FFFF FFFF.
    
    
    
    When subsequently crash recovery is attempted, this may fail
    with the following message during the undo phase :
    
    
    2010-12-20-09.34.11.751803+000 I5359A415          LEVEL: Info
    PID     : 12093                TID  : 13          PROC : db2sysc
    0
    INSTANCE: db2inst3             NODE : 000         DB   : SAMPLE
    EDUID   : 13                   EDUNAME: db2loggr (SAMPLE) 0
    FUNCTION: DB2 UDB, recovery manager, sqlpgSwitchFromRedoToUndo,
    probe:806
    DATA #1 : <preformatted>
    TailIndex 1  firstLso 39333401  nextLso 39349705  logGapSize 0
    
    
    2010-07-01-09.34.11.962973+000 I7301A532          LEVEL: Severe
    PID     : 12093                TID  : 14          PROC : db2sysc
    0
    INSTANCE: db2inst1             NODE : 000
    EDUID   : 14                   EDUNAME: db2loggw (SAMPLE) 0
    FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk,
    probe:909
    MESSAGE : ZRC=0x8610000D=-2045771763=SQLP_BADLOG "Log File
    cannot be used"
              DIA8414C Logging can not continue due to an error.
    DATA #1 : <preformatted>
    TailPage 0 does not match pagePso 39349778 and firstLso 39007321
    

Local fix

  • One of the following methods may be used to attempt to work
    around the issue:
    
    1. Install Fixpack 3a:
    
    https://www-304.ibm.com/support/docview.wss?uid=swg24028306
    
    2. Configure the database to use archive logging, then restart
       the database. For example:
    
       db2 terminate
       db2 deactivate db dbname
       db2 update db cfg for dbname using logarchmeth1 <...>
       db2 activate db dbname
    
    3. Restore the database from a backup image.
    
    4. Contact DB2 Support for assistance.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * All                                                          *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * This APAR only applies to V9.7 FP2 database that use         *
    * circular logging.                                            *
    *                                                              *
    * When the database is  circular logging(sqlpgicl),            *
    * thefirstLso in FCB                                           *
    * array are all initialized to                                 *
    * baselso.2010-07-29-10.13.49.945856+540 I14975442A499         *
    * LEVEL:WarningPID    : 1781890              TID  : 13624      *
    * PROC :db2sysc 4INSTANCE: db2inst1            NODE : 004      *
    * DB  : XXXXXXAPPHDL  : 0-157                                  *
    * APPID:*N0.db2inst1.100729011146AUTHID  : XXXXXEDUID  : 13624 *
    * EDUNAME: db2lload 4FUNCTION: DB2 UDB, RAS/PD                 *
    * component,pdEDUIsInDB2KernelOperation, probe:600DATA #1 :    *
    * String, 33 bytessqlp_get_log_filepath_for_lsn__FDATA #2 :    *
    * String, 4 bytessqlpStacks<StackTrace>------Function +        *
    * Offset------sqlp_get_log_filepath_for_lsnsqluRegisterLoadEndca
    * + 0x260</StackTrace>                                         *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * (1) Upgrade to V9.7 Fix Pack 3, or,                          *
    * (2) Use archiving log                                        *
    ****************************************************************
    

Problem conclusion

  • Problem was first fixed in Version 9.7 Fix Pack 3.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IC76438

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    980

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Pervasive

  • Submitted date

    2011-05-17

  • Closed date

    2011-05-17

  • Last modified date

    2011-05-17

  • APAR is sysrouted FROM one or more of the following:

    IC70482

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

  • R970 PSN

       UP



Document information

More support for: DB2 for Linux, UNIX and Windows

Software version: 9.8

Reference #: IC76438

Modified date: 17 May 2011