PK25804: AFTER RUNNING WEBSPHERE PORTAL SERVER FOR A COUPLE OF HOURS, IT EXPERIENCES A DEADLOCK IN DYNACACHE

6.1.0.1: WebSphere Application Server V6.1.0 Fix Pack 1 for HP-UX
6.1.0.1: WebSphere Application Server V6.1.0 Fix Pack 1 for AIX
6.1.0.1: WebSphere Application Server V6.1.0 Fix Pack 1 for Windows
6.1.0.1: WebSphere Application Server V6.1.0 Fix Pack 1 for Linux
6.1.0.1: WebSphere Application Server V6.1.0 Fix Pack 1 for i5/OS
PK25804; 6.0.2.9: after running WebSphere portal server for a couple of hours
6.1.0.1: WebSphere Application Server V6.1.0 Fix Pack 1 for Solaris

APAR status

• Closed as program error.

Error description

• Deadlock in Dynamic cache when processing the batch operations
  
   WebSphere Version(Base & ND) 6.0.2.9.
  
    After running portal for a couple of hours portal experience
  dead lock in dynacache code.This issue is seen while Stress test
  of Portal application on WebSphere Version 6.0.2.9.
  

Local fix

• Test fix was provided.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: WebSphere Application Sever version 6 users  *
  *                 using DistributedMap.                        *
  ****************************************************************
  * PROBLEM DESCRIPTION: Deadlock in Dynamic cache when          *
  *                      performing a batch operation.           *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  The deadlock occurs under the following scenarios:
  1. One thread: process clear command, it locks Cache A. It
  processes the batch operations. When it tries to trim the
  Cache B, it locks Cache B.
  2. Another thread: process clear command, it locks Cache B. It
  processes the batch operations. When it tries to trim Cache A,
  it locks Cache A.
  

Problem conclusion

• The problem is fixed by removing trim cache logic from the
  batch operations. The trim cache operations will be performed
  after a wakeup thread removes the entries whose time limit has
  expired. This way will prevent the deadlock problem on cache
  instances.
  The code is modified to move the process of trim cache for
  each cache instances in BatchUpdate daemon to TimeLimit
  daemon.
  The TimeLimit daemon is triggered when it is time to check
  whether the entries in cache buffer expire or not.
  After the code checks the expiration, it starts trimming each
  cache instance if buffer size of the cache instance is over
  the cache size.
  
  The fix for this APAR is currently targeted for inclusion in
  fixpack 6.0.2.13 and 6.1.0.1.
  Please refer to the recommended updates page for delivery
  information:
  http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
  

Temporary fix

• The code is modified to move the process of trim cache for
  each cache instances in BatchUpdate daemon to TimeLimit
  daemon. The deadlock occurs under the following scenarios:
  1. One thread: process clear command, it locks Cache A. It
  processes the batch operations. When it tries to trim the
  Cache B, it locks Cache B.
  2. Another thread: process clear command, it locks Cache B. It
  processes the batch operations. When it tries to trim Cache A,
  it locks Cache A.
  The TimeLimit daemon is triggered when it is time to check
  whether the entries in cache buffer expire or not.
  After the code checks the expiration, it starts trimming each
  cache instance if buffer size of the cache instance is over
  the cache size.
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Modules/Macros

• DYNACACH
  

Fix information

• Fixed component name

  WEBS APP SERV N

• Fixed component ID

  5724H8800

Applicable component levels

• R60A PSY

     UP

• R60H PSY

     UP

• R60I PSY

     UP

• R60P PSY

     UP

• R60S PSY

     UP

• R60W PSY

     UP

• R60Z PSY

     UP

• R61A PSY

     UP

• R61H PSY

     UP

• R61I PSY

     UP

• R61P PSY

     UP

• R61S PSY

     UP

• R61W PSY

     UP

• R61Z PSY

     UP