IBM Support

PH04833: JAVA BATCH SCHEDULER IN WEBSPHERE V8.5.5 RUNNING OUTOFMEMORY

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

APAR status

  • Closed as program error.

Error description

  • In a multi-JVM  Websphere v8.5.5 environment, java batch
    scheduler runs OutOfMemory
    
    Systemout log shows the following hung threads:
    
    
    UTLS0008W: The return of alarm thread "Deferrable Alarm : 1"
    (0000009b)
    to the alarm thread pool has been delayed for 10266
    milliseconds. This
    may be preventing normal alarm function within the application
    server.
    The alarm listener stack trace is as follows:
    
    at java.lang.ClassLoader.loadClass(ClassLoader.java:749)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:731)
    
    ...
    WSVR0605W: Thread "SoapConnectorThreadPool : 23" (000000d2) has
    been
    active for 662105 milliseconds and may be hung. There is/are 1
    thread(s) in total in the server that may be hung.
    at com.ibm.db2.jcc.am.jc.l(jc.java:2007)
    at com.ibm.db2.jcc.am.jc.e(jc.java:1648)
    at com.ibm.db2.jcc.am.kc.K(kc.java:1303)
    at com.ibm.db2.jcc.am.ResultSet.getLongX(ResultSet.java:818)
    at com.ibm.db2.jcc.am.ResultSet.getLong(ResultSet.java:798)
    at com.ibm.db2.jcc.am.ResultSet.getLong(ResultSet.java:1626)
    at
    com.ibm.ws.rsadapter.jdbc.WSJdbcResultSet.getLong(WSJdbcResult
    Set.java:1594)
    at
    com.ibm.ws.batch.JobUsageStoreImpl.getPrimaryKeys(JobUsageStore
    Impl.java:434)
    at
    com.ibm.ws.batch.JobUsageStoreImpl.getAllPrimayKeys(JobUsageStor
    e
    Impl.java:620)
    ..
    DCSV9416E: An internal error occurred. Exception is
    com.ibm.ws.dcs.
    common.exception.DCSTransportLayerException: HeartbeatProcessor
    at
    com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.common.LogAdapter.
       handle9416(LogAdapter.java:257)
    at
    com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.common.LogAdapter.
       handleNls(LogAdapter.java:170)
    at
    com.ibm.ws.dcs.vri.transportAdapter.rmmImpl.common.LogAdapter.
       onLogEvent(LogAdapter.java:131)
    at com.ibm.rmm.util.RmmLogger.baseLog(RmmLogger.java:245)
    at com.ibm.rmm.ptl.tchan.receiver.HeartbeatProcessor.run
       (HeartbeatProcessor.java:449)
    Caused by: java.lang.OutOfMemoryError: Java heap space
    at
    com.ibm.rmm.ptl.tchan.receiver.HeartbeatProcessor.run(Heartbeat
    Processor.java:371)
    
    .
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server Java Batch                           *
    ****************************************************************
    * PROBLEM DESCRIPTION: When the option to store Job usage      *
    *                      metrics in the job scheduler database   *
    *                      is enabled, if there are already a      *
    *                      very large number of entries in the     *
    *                      job usage table, an OutOfMemory         *
    *                      exception can occur.                    *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    Currently when a job usage entry needs to be added or updated,
    the entire contents of the job usage table are loaded by the
    runtime code.  After which they are searched and updated
    accordingly.  When the job usage table has a large number of
    entries, this can result in an OutOfMemory exception.  This
    condition can occur if older jobs are not periodically purged
    and the table size grows large.
    

Problem conclusion

  • A code update has been made to more efficiently process the
    required job usage updates to be made in the job scheduler
    database.
    
    The fix for this APAR is currently targeted for inclusion in
    fix pack 9.0.5.0 and 8.5.5.16.  Please refer to the Recommended
    Updates page for delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PH04833

  • Reported component name

    WEBS APP SERV N

  • Reported component ID

    5724H8800

  • Reported release

    850

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-11-02

  • Closed date

    2019-05-14

  • Last modified date

    2019-05-14

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBS APP SERV N

  • Fixed component ID

    5724H8800

Applicable component levels



Document information

More support for: WebSphere Application Server
General

Software version: 850

Reference #: PH04833

Modified date: 14 May 2019