Troubleshooting
Problem
Warehouse Proxy Agent fails to store historical data into TDW db because it has connection errors with the DB2 database used as TDW.
Symptom
WPA fails to send historical data into TDW db and short term historical (STH) data files grows
Diagnosing The Problem
<hostname>_hd_java_<nnnnnnnn>-0X.log shows connection errors like the following:
== 1 t=Thread-1255 com.ibm.db2.jcc.am.DisconnectNonTransientConnectionException: [jcc][t4][2030][11211][3.65.110] A communication error occurred during operations on the connection's underlying socket, socket input stream,
or socket output stream. Error location: Reply.fill() - socketInputStream.read (-1). Message: Connection timed out. ERRORCODE=-4499, SQLSTATE=08001
at com.ibm.db2.jcc.am.cd.a(cd.java:321)
at com.ibm.db2.jcc.t4.a.a(a.java:533)
at com.ibm.db2.jcc.t4.a.a(a.java:521)
at com.ibm.db2.jcc.t4.a.a(a.java:516)
at com.ibm.db2.jcc.t4.a.b(a.java:2092)
at com.ibm.db2.jcc.t4.a.a(a.java:2112)
at com.ibm.db2.jcc.t4.a.f(a.java:455)
at com.ibm.db2.jcc.t4.a.flush_(a.java:398)
at com.ibm.db2.jcc.am.Agent.flow(Agent.java:230)
at com.ibm.db2.jcc.am.lo.r(lo.java:3700)
at com.ibm.db2.jcc.am.lo.m(lo.java:2826)
at com.ibm.db2.jcc.am.lo.addBatch(lo.java:2761)
at com.tivoli.twh.khd.khdxjdbc.addBatch(khdxjdbc.java:1358)
Caused by: java.net.SocketException: Connection timed out
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:124)
at java.net.SocketOutputStream.write(SocketOutputStream.java:170)
at com.ibm.db2.jcc.t4.eb.b(eb.java:1685)
at com.ibm.db2.jcc.t4.eb.a(eb.java:1633)
at com.ibm.db2.jcc.t4.a.f(a.java:448)
Resolving The Problem
There are several causes which can lead to the error reported and often it's more a problem on DB2 side than ITM side. Anyway here below you can find some useful suggestions to try and solve the problem:
1) upgrade DB2 drivers. That is demonstrated in several customer's scenarios that it helps solving the problem.
2) After an investigation by DB2 team, it turned out that the symptom is caused by network error. The Jcc driver just reports the communication error by -4499. They saw the driver can connect to the target database server, but some request does not return, and the driver gets communication error at last. At this point, they recommended the affected user to consult with the network team. For example, bad MTU settings can drop TCP packets when sending large data, then causing similar symptom as reported here.
3) Finally in some cases the following suggestions can be helpful:
3.1) Set
KHD_EXPORT_THREADS=100
KHD_QUEUE_LENGTH=0
in hd.ini
3.2) Also make sure the ulimits are all set to unlimited.
3.3) Then restart the WPA and check if the time out messages in hd_java logs stops occurring.
Was this topic helpful?
Document Information
Modified date:
17 June 2018
UID
swg21962086