IBM Support

IT15233: DBIRELEASEALL CRASH WHEN TWO THREADS ARE ENDING THE SAME TRANSACTION AT THE SAME TIME.

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • During a very small timing window, a crash can occur when more
    then one thread is attempting to end a transaction at the same
    time.  In this documented case, two threads are ending the same
    transaction at the same time.
    
    Key core stack:
    DbiParticipate@AF31_6(??, ??, ??, ??) at 0x10019cfa8
    DbiEvalSQLOutcomeX(??, ??, ??, ??, ??, ??, ??, ??) at
    0x1000cba60
    RdbCloseConnection(??, ??, ??) at 0x1000c8b7c
    DbiReleaseAll(??) at 0x1000ca2e0
    RdbRestart(??) at 0x1000cde18
    DbiEvalSQLOutcomeX(??, ??, ??, ??, ??, ??, ??, ??) at
    0x1000cb5b8
    
    Tivoli Storage Manager Versions Affected: 6.3 and 7.1
    IBM Spectrum Protect Versions Affected: 7.1.3, 7.1.4, 7.1.5,
    and 7.1.6
    
    Customer/L2 Diagnostics:
    
    Example of messages found before the crash in the Actlog:
     ANR0171I dbiconn.c(1739): Error detected on 0:957,
      database in evaluation mode. (SESSION: 365661)
     ANR0171I dbiconn.c(1739): Error detected on 0:2006,
      database in evaluation mode. (SESSION: 365832)
     ANR9999D_0645605689 tbOpenX(tbtbl.c:5030) Thread<845259>:
      Failure participating on transaction. (SESSION: 365837)
     ANR9999D Thread<845259> issued message 9999 from:
      (SESSION: 365837)
     ANR9999D Thread<845259>  0x000000010001ca7c StdPutText
      (SESSION: 365837)
     ANR9999D Thread<845259>  0x000000010001d514 OutDiagToCons
      (SESSION: 365837)
     ANR9999D Thread<845259>  0x00000001000090bc outDiagfExt
      (SESSION: 365837)
     ANR9999D Thread<845259>  0x000000010009e28c tbOpenX
      (SESSION: 365837)
     ANR9999D Thread<845259>  0x0000000100030f74
      admElBuildClientVectors  (SESSION: 365837)
     ANR9999D Thread<845259>  0x0000000100545e88
      smExecuteSession  (SESSION: 365837)
     ANR9999D Thread<845259>  0x0000000100078c7c
      psSessionThread  (SESSION: 365837)
     ANR9999D Thread<845259>  0x000000010000c264 StartThread
    ...
    ANR0162W Supplemental database diagnostic information:
     -1:08001:-30081 ([IBM][CLI Driver] SQL30081N  A
     communication error has been detected. Communication
     protocol being used: "TCP/IP".  Communication API being
     used: "SOCKETS".  Location where the error was detected:
     "127.0.0.1".  Communication function detecting the error:
     "recv".  Protocol specific error code(s): "73", "*", "0".
      SQLSTATE=08001). (SESSION: 364452)
    ...
     ANR0171I dbiconn.c(1739): Error detected on 0:3392,
    database in evaluation mode. (SESSION: 403996)
     Context report (SESSION: 403910)
     Thread AcceptorThread (89) is a parent thread
    related to: 929581 (SESSION: 403910)
     (929621) Thread AcceptorThread (91) is a parent thread
    related to: 929621 (SESSION: 403939)
     (91) Failed attempt #1 to get TMV->mutex (SESSION: 403939)
    **** down time ***
     ANR7811I Using instance directory /home/tsminst
    
    Full core stacks that have been reported:
    
    Segmentation fault in DbiParticipate@AF31_6 at 0x10019cfa8
    ($t929809)
    0x10019cfa8 (DbiParticipate@AF31_6+0x28) eba40079         ldu
    r29,0x78(r4)
    (dbx) where
    DbiParticipate@AF31_6(??, ??, ??, ??) at 0x10019cfa8
    DbiEvalSQLOutcomeX(??, ??, ??, ??, ??, ??, ??, ??) at
    0x1000cba60
    RdbCloseConnection(??, ??, ??) at 0x1000c8b7c
    DbiReleaseAll(??) at 0x1000ca2e0
    RdbRestart(??) at 0x1000cde18
    DbiEvalSQLOutcomeX(??, ??, ??, ??, ??, ??, ??, ??) at
    0x1000cb5b8
    RdbCreateConnection(??, ??) at 0x1000c7914
    DbiGetConnectionTracked(??, ??, ??, ??) at 0x1000c6338
    dbitxn.AllocTxnDesc(??) at 0x10019cca4
    DbiParticipate@AF31_6(??, ??, ??, ??) at 0x10019d040
    tbOpenX(??, ??, ??, ??, ??) at 0x10009d910
    admElBuildClientVectors(??, ??) at 0x100030f70
    smExecuteSession(??, ??, ??, ??, ??, ??, ??, ??) at 0x100545e84
    psSessionThread(??) at 0x100078fd8
    StartThread(??) at 0x10000c260
    
    and
    
    Segmentation fault in DbiEvalSQLOutcomeX at 0x1000caf84
    ($t1294918)
    0x1000caf84 (DbiEvalSQLOutcomeX+0x204) 90980058         stw
    r4,0x58(r24)
    (dbx)
    (dbx) where
    DbiEvalSQLOutcomeX(??, ??, ??, ??, ??, ??, ??, ??) at
    0x1000caf84
    RdbCloseConnection(??, ??, ??) at 0x1000c8b7c
    DbiReleaseAll(??) at 0x1000ca2e0
    RdbDeactivateDatabase(??, ??) at 0x10019ff44
    RdbRestart(??) at 0x1000cd90c
    DbiEvalSQLOutcomeX(??, ??, ??, ??, ??, ??, ??, ??) at
    0x1000cb5b8
    RdbCreateConnection(??, ??) at 0x1000c7914
    DbiGetConnectionTracked(??, ??, ??, ??) at 0x1000c6338
    dbitxn.AllocTxnDesc(??) at 0x10019cca4
    DbiParticipate@AF31_6(??, ??, ??, ??) at 0x10019d040
    tbOpenX(??, ??, ??, ??, ??) at 0x10009d910
    AsUpdateMountCount(??) at 0x100830f04
    OpenActiveVol(??, ??, ??, ??, ??) at 0x10086fb90
    AsOpenVol(??, ??, ??, ??, ??, ??) at 0x10086e654
    AsAcquireOutputVol(??, ??, ??, ??, ??, ??) at 0x100857b04
    AllocNewVol(??, ??, ??, ??) at 0x1004898c0
    AsAllocSeg(??, ??, ??) at 0x100485f30
    DoAllocSeg(??, ??) at 0x1007ca10c
    AuxAsSink(??) at 0x1007c8808
    SsAuxSinkThread(??) at 0x1007c8524
    StartThread(??) at 0x10000c260
    
    New core stack seeing this problem:
    _mutex_lock
    pkAcquireMutexTracked
    DbiParticipate@AF32_6
    DbiEvalSQLOutcomeX
    RdbCloseConnection
    DbiReleaseAll
    RdbRestart
    DbiEvalSQLOutcomeX
    dbiEndTxn
    DoEndFuncCallbacks
    tmAbortX
    SmNodeSession
    HandleNodeSession
    smExecuteSession
    psSessionThread
    StartThread
    
    or
    
    pth_mutex._mutex_lock
    pkmon.pkAcquireMutexTracked
    dbitxn.DbiParticipate@AF32_6
    dbieval.DbiEvalSQLOutcomeX
    dbiconn.RdbCloseConnection
    dbiconn.DbiReleaseAll
    rdbinst.RdbRestart
    dbieval.DbiEvalSQLOutcomeX
    dbitxn.dbiEndTxn
    tmtxn.DoEndFuncCallbacks
    tmtxn.tmAbortX
    smnode.SmNodeSession
    smexec.HandleNodeSession
    smexec.smExecuteSession
    tcpcomm.psSessionThread
    pkthread.StartThread
    
    Initial Impact: High
    Additional Keywords: core dump outage
    

Local fix

  • Get to the fix level when available
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * All Tivoli Storage Manager Server users.                     *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See ERROR DESCRIPTION.                                       *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Apply fixing levels when available. This problem is          *
    * currently projected to be fixed in levels 6.3.6.100 and      *
    * 7.1.7. Note that this is subject to change at the discretion *
    * of IBM.                                                      *
    ****************************************************************
    

Problem conclusion

  • This problem was fixed.
    Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT15233

  • Reported component name

    TSM SERVER

  • Reported component ID

    5698ISMSV

  • Reported release

    63A

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2016-05-11

  • Closed date

    2016-06-22

  • Last modified date

    2016-12-06

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    TSM SERVER

  • Fixed component ID

    5698ISMSV

Applicable component levels

  • R71A PSY

       UP

  • R71H PSY

       UP

  • R71L PSY

       UP

  • R71S PSY

       UP

  • R71W PSY

       UP



Document information

More support for: Tivoli Storage Manager

Software version: 63A

Reference #: IT15233

Modified date: 06 December 2016