Application hangs with J2CA0045E errors and with J2CA0086W warning messages

Technote (troubleshooting)


Problem(Abstract)

An application seems to be in a hang state while trying to connect to database. The logs show several J2CA0045E errors due to ConnectionWaitTimeoutExceptions as well as other errors referring to problems with obtaining free connections from the pool.

Symptom

Before occurrences of a hang, there are repeated J2CA0086W warning messages, there could also be J2CA0294W messages logged (which indicate that a direct lookup of the jndi name was done versus a lookup using a resource reference); and the connection pool is at a maximum.

The following error indicates a connection timeout waiting for a connection to be available from the connection pool.
[8/1/11 15:57:35:290 CDT] 00000013 FreePool E J2CA0045E:
Connection not available while invoking method createOrWaitForConnection
for resource jdbc/myDB.

A warning message similar to the one below is observed in the logs before the hang occurs:

[8/1/11 15:57:05:366 CDT] 00000013 SharedPool I J2CA0086W: Shareable connection MCWrapper id 686bbdf9 Managed connection com.ibm.ws.rsadapter.spi.WSRdbManagedConnectionImpl@6156bdf9
State:STATE_TRAN_WRAPPER_INUSE

The message reference contains the following explanation of this message:

J2CA0086W: Shareable connection {0} from resource {1} was used within a local transaction containment boundary.

Explanation: Shareable connections are not allowed within a local transaction containment boundary.

User Response: Connection was made non-shareable.


Cause

This problem is most likely to happen when the application is a servlet. The J2CA0086W message indicates that a Shareable connection is used with LocalTransactionContainment (LTC). The sharing scope of a connection is a transaction. When a servlet is started, the WebContainer starts an LTC for the servlet. In this scenario, if it gets a Shareable connection from the connection pool, then the J2CA0086W could occur (it is often suppressed from the logs). When a 2nd servlet is invoked, then the LTC for the first servlet is suspended and a 2nd LTC is started. If the 2nd servlet gets a connection from the same pool, it will not share the first connection.
For a connection to be shared within a transaction it has to follow the pattern shown below and the getConnection() for the 2nd connection must be done in the same transaction:

  • get connection,
  • use connection,
  • close connection,

If this logic is followed, then a Shareable connection can be used again within a transaction. If this logic is followed for LTCs using Unshareable connections, when the close connection is called, the connection is returned to the pool and may be reused.

If this logic is not followed, then the connection will not be shared and a second (or third) connection can be allocated. (NOTE: if an Unshareable connection is reused within a transaction, then each getConnection() will cause a new connection to be used.)

For example, if the application calls the getConnection() method it gets connection1, uses connection1, and then, if it calls getConnection() again and connection1 is not ready to be reused, connection2 is obtained. Both connections remain in the shared pool and both are associated with the Local Transaction until the Local Transaction ends (is committed or rolled back, or the method ends). This can result in more connections being created than expected, which is why the connection pool that the application is using reaches the maximum connections, and the application server throws the ConnectionWaitTimeoutException. This causes a hang in the application since the pool is at a maximum, and none of the threads that have connections can complete because they are waiting to get another connection.

Another possible consequence is that while these connections are held in-use in the pool, other threads allocating connections from the same connection pool have their global transaction timed out by the backend database resulting in WTRN0006W messages.

Or, you might also see WSVR0605W messages with the java stack due to threads waiting to get a connection from the connection pool.
[10/28/10 14:36:33:885 EDT] 0000002d ThreadMonitor W WSVR0605W: Thread "WebContainer : 23" (00000054) has been active for 765689 milliseconds and may be hung. There is/are 2 thread(s) in total in the server that may be hung.
at java.lang.Object.wait(Native Method)
at com.ibm.ejs.j2c.FreePool.queueRequest(FreePool.java:408)
at com.ibm.ejs.j2c.FreePool.createOrWaitForConnection(FreePool.java:1274)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:2527)
at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:1059) at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:696)

Diagnosing the problem

This can be checked using the wsadmin command showPoolContents, refer to the technote, "How to troubleshoot J2CA0045E connection pooling problems". The symptom to look for is for the tx type to indicate LocalTransaction and for a thread to have more than one connection in a list of the Shared connections.

Resolving the problem

The following document describes how the Application Server manages connections.

Default behavior of managed connections in WebSphere Application Server

There are two solutions to this problem:

  1. The application must be modified to serially reuse the connection, using the getConnection()/use connection/close() pattern before calling getConnection() to get a new connection.
  2. Change the res-sharing-scope of the connection to be Unshareable. The recommended way to do this is by using a resource reference and looking up the resource reference versus the jndi name of the data source. See the WebSphere Application Server Information Center Creating or changing a resource reference. It is also possible to set a data source property, defaultConnectionTypeOverride, see the Tuning Connection Pools article.

Cross reference information
Segment Product Component Platform Version Edition
Application Servers Runtimes for Java Technology Java SDK

Rate this page:

(0 users)Average rating

Add comments

Document information


More support for:

WebSphere Application Server
Java 2 Connectivity (J2C)

Software version:

6.0, 6.1, 7.0, 8.0, 8.5, 8.5.5

Operating system(s):

AIX, HP-UX, IBM i, Linux, Solaris, Windows, i5/OS, z/OS

Software edition:

Base, Express, Network Deployment

Reference #:

1121449

Modified date:

2012-09-13

Translate my page

Machine Translation

Content navigation