IBM Support

IC78384: INSTANCE MAY GO DOWN AT SQLOFREEMEMCHUNKS AFTER A SEVERE SQLERMASTERTHREADREQ MESSAGE.

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • Prior to the instance down of this problem, the message
    "marking fmp as unstable after new thread creation failure:"
    at sqlerMasterThreadReq is logged and fmpRowUseCount indicates
    1 as db2diag.log as below:
    -----
    2011-06-02-19.22.51.266000+540 I11423990F453      LEVEL: Severe
    PID     : 2136                 TID  : 6228        PROC :
    db2syscs.exe
    INSTANCE: TOOL                 NODE : 000         DB   : TOOLSDB
    APPHDL  : 0-49740              APPID: *LOCAL.TOOL.110602102634
    AUTHID  : SYSTEM
    EDUID   : 6228                 EDUNAME: db2agent (TOOLSDB)
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerMasterThreadReq,
    probe:10
    RETCODE : ZRC=0xFFFFFBEE=-1042
    
    2011-06-02-19.22.51.266000+540 E11424445F3409     LEVEL: Severe
    PID     : 2136                 TID  : 6228        PROC :
    db2syscs.exe
    INSTANCE: TOOL                 NODE : 000         DB   : TOOLSDB
    APPHDL  : 0-49740              APPID: *LOCAL.TOOL.110602102634
    AUTHID  : SYSTEM
    EDUID   : 6228                 EDUNAME: db2agent (TOOLSDB)
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerMasterThreadReq,
    probe:910
    DATA #1 : String, 58 bytes
    marking fmp as unstable after new thread creation failure:
    DATA #2 : String, 8 bytes
    Fmp TID:
    DATA #3 : Hexdump, 4 bytes
    0x000007FFFDE3CC90 : 0000 0000
    ....
    DATA #4 : String, 8 bytes
    Fmp Row:
    DATA #5 : sqlerFmpRow, PD_SQLER_TYPE_FMP_ROW, 496 bytes
     fmpPid: 5396
     fmpPoolList Ptr: 0x0000000000000000    fmpForcedList Ptr:
    0x0000000000000000
     nextFmpCB Ptr: 0x000007fffdc0f1a0      prevFmpCB Ptr:
    0x0000000000000000
     fmpIPCList Ptr: 0x000007fffde3cb00
     stateFlags: 0x00000012 numFmp32Attaches: 0
     numActiveThreads: 0    numPoolThreads: 0
     fmpCodePage: 0 fmpRowUseCount: 1
     active: 0x01   rowLoaderValidate: 0x00
     ipcLatch:
    0x000007FFFDE3C004 : 8000 AB01
    ....
     rowLatch:
    0x000007FFFDE3C008 : 8000 AC01
    ....
     fmpAgentList:
    -----
    
    Then later timing, this instance goes down with below message:
    -----
    2011-06-02-20.47.51.453000+540 E11740971F1136     LEVEL: Severe
    PID     : 2136                 TID  : 6480        PROC :
    db2syscs.exe
    INSTANCE: TOOL                 NODE : 000         DB   : TOOLSDB
    APPHDL  : 0-53112              APPID: *LOCAL.TOOL.110602114844
    AUTHID  : SYSTEM
    EDUID   : 6480                 EDUNAME: db2agent (TOOLSDB)
    FUNCTION: DB2 UDB, SQO Memory Management,
    sqloDiagnoseFreeBlockFailure, probe:999
    MESSAGE : Memory validation failure, diagnostic file dumped.
    DATA #1 : String, 28 bytes
    Corrupt pool free tree node.
    DATA #2 : File name, 29 bytes
    2136.6480.mem_diagnostics.txt
    CALLSTCK:
      [0] 0x0000000180116CDA pdLog + 0x36A
      [1] 0x0000000180051873 sqloDirectIOIsRegistryEnable + 0x3DB
      [2] 0x000000018007C95D sqloxltc_app + 0x5EFF
      [3] 0x00000001800519C3 sqloDirectIOIsRegistryEnable + 0x52B
      [4] 0x000000018004A7DA sqlogmblkEx + 0x52A
      [5] 0x0000000001F7676E sqlerGetFMPIPC + 0x10A
      [6] 0x0000000001CBC386 sqlccGetIPCs + 0xAE6
      [7] 0x0000000001F4BA99 sqlerAllocFmpIPC + 0x13B
      [8] 0x0000000001F4A8E7 sqlerAddFmpToPool + 0x2D7
      [9] 0x0000000001F4981A sqlerGetFmpFromPool + 0x3C0
    -----
    
    The associated trap file has a similar trace back as below:
    ---
    <sqloFreeMemChunks> <NotFound:-1>
    <sqloDirectIOIsRegistryEnable> <NotFound:-1>
    <sqloxltc_app> <NotFound:-1>
    <sqloDirectIOIsRegistryEnable> <NotFound:-1>
    <sqlogmblkEx> <NotFound:-1>
    <sqlerGetFMPIPC> <sqlerFmpCommon.C:391>
    <sqlccGetIPCs> <sqlccipc_srv.C:1185>
    <sqlerAllocFmpIPC> <sqlerFmpEngine.C:6593>
    <sqlerGetFmpThreadEntry> <sqlerFmpEngine.C:4609>
    <sqlerGetFmpFromPool> <sqlerFmpEngine.C:1773>
    <sqlerInvokeFencedRoutine> <sqlerFmpEngine.C:818>
    ---
    

Local fix

  • N/A
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * DB2 UDB Version 9.7.                                         *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error description field for more information.            *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Upgrade to Version 9.7 FixPack 7.                            *
    ****************************************************************
    

Problem conclusion

  • Problem was first fixed in DB2 UDB Version 9.7 FixPack 7.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IC78384

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    970

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2011-08-30

  • Closed date

    2012-10-24

  • Last modified date

    2012-10-24

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

    IC84171 IC88604 IC95717

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

  • R970 PSN

       UP



Document information

More support for: DB2 for Linux, UNIX and Windows

Software version: 9.7

Reference #: IC78384

Modified date: 24 October 2012