IBM Support

PM56082: THREAD RACE RESULTS IN OUT-OF-ORDER ISALIVE CALLBACK AND JVM STOPPED WITH PANICJVM IN HAMANAGER.

Fixes are available

7.0.0.23: WebSphere Application Server V7.0 Fix Pack 23
8.0.0.4: WebSphere Application Server V8.0 Fix Pack 4
7.0.0.25: WebSphere Application Server V7.0 Fix Pack 25
8.0.0.5: WebSphere Application Server V8.0 Fix Pack 5
7.0.0.27: WebSphere Application Server V7.0 Fix Pack 27
8.0.0.6: WebSphere Application Server V8.0 Fix Pack 6
7.0.0.29: WebSphere Application Server V7.0 Fix Pack 29
8.0.0.7: WebSphere Application Server V8.0 Fix Pack 7
6.1.0.47: WebSphere Application Server V6.1 Fix Pack 47
8.0.0.8: WebSphere Application Server V8.0 Fix Pack 8
7.0.0.31: WebSphere Application Server V7.0 Fix Pack 31
7.0.0.27: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.33: WebSphere Application Server V7.0 Fix Pack 33
8.0.0.9: WebSphere Application Server V8.0 Fix Pack 9
7.0.0.35: WebSphere Application Server V7.0 Fix Pack 35
8.0.0.10: WebSphere Application Server V8.0 Fix Pack 10
7.0.0.37: WebSphere Application Server V7.0 Fix Pack 37
8.0.0.11: WebSphere Application Server V8.0 Fix Pack 11
7.0.0.39: WebSphere Application Server V7.0 Fix Pack 39
8.0.0.12: WebSphere Application Server V8.0 Fix Pack 12
7.0.0.41: WebSphere Application Server V7.0 Fix Pack 41
8.0.0.13: WebSphere Application Server V8.0 Fix Pack 13
7.0.0.43: WebSphere Application Server V7.0 Fix Pack 43
8.0.0.14: WebSphere Application Server V8.0 Fix Pack 14
7.0.0.45: WebSphere Application Server V7.0 Fix Pack 45
8.0.0.15: WebSphere Application Server V8.0 Fix Pack 15
6.1.0.45: Java SDK 1.5 SR14 Cumulative Fix for WebSphere Application Server
6.1.0.47: Java SDK 1.5 SR16 Cumulative Fix for WebSphere Application Server
7.0.0.23: Java SDK 1.6 SR10 FP1 Cumulative Fix for WebSphere
7.0.0.25: Java SDK 1.6 SR11 Cumulative Fix for WebSphere Application Server
7.0.0.27: Java SDK 1.6 SR12 Cumulative Fix for WebSphere Application Server
7.0.0.29: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.45: Java SDK 1.6 SR16 FP60 Cumulative Fix for WebSphere Application Server
7.0.0.31: Java SDK 1.6 SR15 Cumulative Fix for WebSphere Application Server
7.0.0.35: Java SDK 1.6 SR16 FP1 Cumulative Fix for WebSphere Application Server
7.0.0.37: Java SDK 1.6 SR16 FP3 Cumulative Fix for WebSphere Application Server
7.0.0.39: Java SDK 1.6 SR16 FP7 Cumulative Fix for WebSphere Application Server
7.0.0.41: Java SDK 1.6 SR16 FP20 Cumulative Fix for WebSphere Application Server
7.0.0.43: Java SDK 1.6 SR16 FP41 Cumulative Fix for WebSphere Application Server

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • Due to Thread race condition, which results in
    out-of-order isAlive callback. In the end JVM stopped
    with panicJVM. The following stack could be observed.
    
    [12/01/11 17:33:54:791 JST] 0000001e SystemErr     R
    at com.ibm.ws.hamanager.runtime.RuntimeProviderImpl
       .panicJVM(RuntimeProviderImpl.java:91)
    [12/01/11 17:33:54:791 JST] 0000001e SystemErr     R
    at com.ibm.ws.hamanager.coordinator.impl.JVMControllerImpl
       .panicJVM(JVMControllerImpl.java:56)
    [12/01/11 17:33:54:791 JST] 0000001e SystemErr     R
    at com.ibm.ws.hamanager.impl.HAGroupImpl
       .doIsAlive(HAGroupImpl.java:866)
    [12/01/11 17:33:54:791 JST] 0000001e SystemErr     R
    at com.ibm.ws.hamanager.impl.HAGroupImpl$HAGroupUserCallback
       .doCallback(HAGroupImpl.java:1364)
    [12/01/11 17:33:54:791 JST] 0000001e SystemErr     R
    at com.ibm.ws.hamanager.impl.Worker.run(Worker.java:64)
    [12/01/11 17:33:54:791 JST] 0000001e SystemErr     R
    at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1550)
    [12/01/11 17:33:54:791 JST] 0000001e SystemOut     O
    java.lang.RuntimeException: emergencyShutdown called:
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server                                      *
    ****************************************************************
    * PROBLEM DESCRIPTION: An Application Server stops             *
    *                      unexpectedly with a "panic JVM"         *
    *                      message in the SystemOut log            *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    WebSphere Application Server supports a number of clustered,
    highly available singleton services, such as as a J2C adapter
    or an SIBus messaging engine. The HA Manager will elect one
    member of the cluster to host the active singleton service.
    When the HA Manager elects a cluster member, it sends an
    "activate" command to the singleton service on the elected
    cluster member. After sending the activate command, the HA
    Manager will then monitor the health of the singleton service
    by periodically sending an "isAlive" check. If the singletonn
    service responds that it is not alive, the HA Manager will
    panic the JVM so the singleton service can fail over to
    another cluster member.
    Due to a race condition, it is possible for the isAlive check
    to be processed by the singleton before the activate command
    has reached it, causing an erroneous panic of the application
    server.
    

Problem conclusion

  • Changed the way the HA Manager isAlive is dispatched to
    guarantee that the activate command will always reach the
    singleton service before the isAlive.
    
    The fix for this APAR is currently targeted for inclusion in
    fix packs 6.1.0.45, 7.0.0.23 and 8.0.0.4.  Please refer to
    the Recommended Updates page for delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PM56082

  • Reported component name

    WEBS APP SERV N

  • Reported component ID

    5724H8800

  • Reported release

    700

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2012-01-16

  • Closed date

    2012-03-15

  • Last modified date

    2012-03-15

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBS APP SERV N

  • Fixed component ID

    5724H8800

Applicable component levels

  • R61A PSY

       UP

  • R61H PSY

       UP

  • R61I PSY

       UP

  • R61P PSY

       UP

  • R61S PSY

       UP

  • R61W PSY

       UP

  • R61Z PSY

       UP

  • R700 PSY

       UP

  • R800 PSY

       UP



Document information

More support for: WebSphere Application Server
General

Software version: 7.0

Reference #: PM56082

Modified date: 15 March 2012