JR43775: CMVC 224030 - TO PREVENT CONNECTION DEADLOCK IN ECKEY.GETNEXTVALUE

7.0.0-WS-WCServer-FP009

APAR status

• Closed as program error.

Error description

• Most WebSphere Commerce application server threads use a single
  JDBC connection for each database transaction.  Occasionally a
  thread requires a second connection for a second, concurrent,
  transaction.  For example, the BusinessAuditStorage
  getAndMarkBusinessEvent EJB method requires a new database
  transaction, and a second database connection, as does the
  ServerJDBCHelper increment EJB method, used by the KeyManager
  class.
  
  When a JDBC connection is obtained from the pool, it remains
  associated with the database transaction until the database
  transaction ends, at which point it becomes available for use by
  other database transactions.  When the pool becomes empty, a
  thread that requires a JDBC connection may have to wait until
  one becomes available.  However, if all threads are waiting for
  a second connection, they may never obtain one, since no thread
  is able to end its first transaction, thus making a JDBC
  connection available, until its second transaction ends.  The
  result can be a "connection deadlock", shown in the SystemOut
  logs as a connection timeout exception.
  
  A second kind of "connection deadlock" can also occur when a
  thread holding a Java lock blocks other threads while waiting
  for a JDBC connection, since the blocked threads may themselves
  hold all the JDBC connections in the pool, leaving none
  available for the waiting thread.
  

Local fix

• JDBC "connection deadlocks" can be avoided by defining more than
  enough JDBC connections in each connection pool, so that a
  thread never has to wait for a JDBC connection to become
  available.
  

Problem summary

• USERS AFFECTED:
  All WebSphere Commerce customers.
  
  PROBLEM ABSTRACT:
  WebSphere Commerce application server threads may hang or
  timeout if the JDBC connection pool is empty.
  
  BUSINESS IMPACT:
  There can be database scalability problems (sometimes referred
  to as a "connection storm") when too many JDBC connections are
  defined in the connection pools and a sudden increase in
  database activity occurs, causing the database server to spend
  too much time creating new connections to handle the increased
  load, rather than serving existing connections more quickly.
  
  RECOMMENDATION:
  

Problem conclusion

• Another way to avoid "connection deadlocks" is to define a
  Reserve Pool of JDBC connections, and to pre-allocate JDBC
  connections to threads.  Connections in the Reserve Pool are
  available only to threads that request a second connection.  As
  long as each thread requires at most two connections, the first
  kind of "connection deadlock" can be avoided.  The second kind
  of "connection deadlock" can also be avoided by pre-allocating a
  JDBC connection to a thread before it acquires Java locks that
  could potentially block other threads while waiting for a JDBC
  connection.
  
  By avoiding "connection deadlocks" in this manner, JDBC
  connection pools can be set to more optimal, smaller, sizes, and
  a possible "connection storm" can be avoided.
  
  Defining a WebSphere Application Server Reserve Pool
  
  For this step to work properly, you must first apply the WAS
  Feature in Service APAR PM68323 to your WebSphere Application
  Server instance.
  
  Use the WebSphere Application Server administration console to
  define a connection pool custom property.  The name of the
  custom property should be "numberOfPoolReserves" and the value
  should be "1".
  
  Pre-allocating JDBC connections to WebSphere Commerce Server
  threads
  
  For this step to work properly, you must first apply WebSphere
  Commerce APAR JR40334 and this APAR JR43775.
  
  To pre-allocate a JDBC connection for each WebSphere Commerce
  Server thread that requires a database connection, add the
  following configuration to the <InstanceProperties> element of
  the WebSphere Commerce configuration file.
  
  <com.ibm.commerce.server.TransactionManager
  preFetchJDBCConnection="true" />
  
  For more information about the WebSphere Commerce configuration
  file, see Updating the WebSphere Commerce configuration file.
  http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com
  .ibm.commerce.admin.doc/concepts/ccmxml.htm
  
  -------------------------------------------------------------
  The latest available maintenance information can be obtained
  from the Recommended Fixes for WebSphere Commerce technote:
  http://www.ibm.com/support/docview.wss?rs=3046&uid=swg21261296
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WC BUS DEV ED W

• Fixed component ID

  5724I3900

Applicable component levels

• R700 PSY

     UP