Knowledge Collection: WebSphere Application Server Service Integration Technology Message store problems and their solutions

Education


Abstract

This Knowledge Collection is a compilation of links to the most frequently accessed technotes and webcast replays dealing with problems related to Message stores in the WebSphere Application Server Service Integration Technology (also known as Service Integration Bus (SIB)).

Most of the problems are caused by either configuration or runtime issues or both and can be easily resolved by following the directions in these Technotes or webcasts.

Content

Directory of items within this Knowledge Collection

Technotes

Webcast Replays

Other Useful Links for Service Integration Bus

Other Knowledge Collections for WebSphere Application Server

Technotes


1. WebSphere Application Server Service Integration Bus Messaging Engine fails to start due to java.io.IOException (1628926)
You have created a Service Integration Bus and configured the messaging engine to use a filestore. The filestore is located on NFS v3 file system. After restarting WebSphere Application server, the messaging engine fails to start and java.io.IOException is written to the SystemOut.log.

2. CWSIS1522E: The messaging engines request for a database connection timed out (1626528)
Your client or application requests fail while trying to connect to WebSphere Application Server Messaging Engine even though the Messaging Engine is running

3. DSRA0080E error thrown and JVM is terminated with the message: HMGR0130I (1624733)
WebSphere Application Server (WAS) Message Engine (ME) was started and all of sudden your application is unable to send the messages to the BUS. In the SystemOut.log initially the error DSRA0080E is reported from the message data store component followed by J2CA0056I. Each of these messages is repeated several times. This sequence is then followed by message codes CWSIS1546I and CWSIS1538I, which indicate that the lock on the database has been lost and repeated attempts are being made to recover the lock. Finally the application server JVM is terminated by HA manager to avoid any possible data loss. Message HMGR0130I appears before it terminates the JVM. The application server is terminated in a orderly way.

4. CWSIS1524E: Data source, jdbc/ not found (1624732)
While starting the Messaging Engine in a stadalone server or during the failover of the Messaging Engine in a WebSphere Application Server (WAS) cluster you see this generic error: CWSIS1524E

5. WebSphere Application Server Service Integration Bus Messaging engine fails to start due to java.lang.ClassNotFoundException (1617348)
Websphere Application Server (WAS) is migrated from WAS v6.1 to WAS v7.0. After the migration ,messaging engine starts up successfully. Later when reverse migration is performed from WAS 7.0 to WAS 6.1 the messaging engine fails to start and CWSIS0001E errors are written to the SystemOut.log

6. WebSphere Application Server Messaging Engine disconnected abruptly (1617039)
Your messages are not passing from the internal Messaging bus to any other module. This results in messages collecting in a queue. When servers containing a WebSphere Application Server (WAS) Messaging Engine (ME) are restarted, messages process normally thereafter.

7. Service Integration Bus messaging engine startup fails with CWSIS1544E error (1613295)
On restarting Websphere Application Server, Service Integration Bus, the messaging engine fails to start and throws CWSIS1544E error.

8. Service Integration Bus messaging engine startup fails with CWSOM1041E Exception (1611211)
You have restarted WebSphere Application Server. After the restart, the messaging engine fails to start due to CWSOM1041E Exception

9. WebSphere Application Server Service Integration Bus messaging engine startup fails with java.sql.SQLException: ORA-01031:insufficient privileges (1611118)
On restarting Websphere Application Server, the Service Integration Bus messaging engine fails to start due to java.sql.SQLException: ORA-01031:insufficient privileges

10. WebSphere Application Server Service Integration Bus messaging engine fails to start due to CWSIS1561E exception (1611160)
On restarting WebSphere Application Server, the Service Integration Bus messaging engine fails to start and the following error is logged in the SystemOut.log: CWSIS1561E:The messaging engine's unique id does not match that found in the file store. ME_UUID=A85B64156006973D, ME_UUID(FS)=AAF19AC8C805638A

11. WebSphere Application Server Service Integration Bus messaging engine throws java.io.StreamCorruptedException (1610720)
You have a log rotation script that makes use of the copytruncate utility in UNIX to rotate the WebSphere Application Server Service Integration Bus messaging engine filestore files. You executed the script while the messaging engine was running and encountered java.io.StreamCorruptedException

12. WebSphere Application Server Service Integration Bus messaging engine startup fails with ORA-28000 error (1610676)
After restarting WebSphere Application Server (WAS), the messaging engine fails to start and throws the following error: CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException:CWSIS1501E: The data source has produced an unexpected exception: java.sql.SQLException: ORA-28000: the account is locked DSRA0010E: SQL State = 99999, Error Code = 28,000

13. Unable to start WebSphere Application Server Message Engine (1610323)
You are unable to start WebSphere Application Server (WAS) message engine(ME) either with JVM restart or manually with WAS console.

14. Unable to access data store due to security reasons while starting Messaging Engine (1610262)
WebSphere Application Server (WAS) Messaging Engine is not starting and the following errors are thrown in the SystemOut.log: CWSIS1501E, SQL30082N with database security error codes 08001 and -30,082. CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data source has produced an unexpected exception: java.sql.SQLException: SQL30082N Attempt to establish connection failed with security reason "24" ("USERNAME AND/OR PASSWORD INVALID"). SQLSTATE=08001 DSRA0010E: SQL State = 08001, Error Code = -30,082

15. Messaging Engine does not start. No errors or exceptions in the SystemOut.log related to the problem (1609567)
Your WebSphere Application Sever Messaging Engine does not start and you do not see any errors or exceptions in SystemOut.log; not even messages indicating an attempt to start the Messaging Engine.

16. Messaging Engine does not start. Exception: CWSIS1524E, specified data source not found. (1609726)
WebSphere Application Server (WAS) Messaging Engine (ME) does not start. Exception: CWSIS1524E, specified data source not found.

17. Messaging Engine starting problem with the error message: CWSOM1042E (1609496)
Your WebSphere Application Server Messaging Engine is not starting. You see the following errors in SystemOut.log: CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: Unexpected exception caught starting persistent message store: CWSOM1042E: ObjectStore=AbstractObjectStore(/ /*)/ (ObjectStore) was asked to allocate space for ManagedObject=ManagedObject(null/null)/Constructed/ (ManagedObject) when it was full. CWSIS1574E: The file stores permanent store file is full. CWSOM0003I: ObjectManager using logFile / /Log has shut down. CWSID0035E: Messaging engine cannot be started; detected error reported during com.ibm.ws.si

18. CWSOM1017E: Unable to access log file (file store location) while starting Messaging Engine (1609458)
WebSphere Application Server (WAS) Messaging Engine fails to start. The following errors occurred in the SystemOut.log file: CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: Unexpected exception caught starting persistent message store: CWSOM1017E:ObjectManagerState=ObjectManagerState( )/Stopped ManagedObject(0/0)/Ready/ (ObjectManagerState) caught exception= (Exception) trying to locate or create log file name={2}(String). CWSID0035E: Messaging engine cannot be started; detected error reported during com.ibm.ws.sib.msgstore.impl.MessageStoreImpl start() CWSID0027E:Messaging engine cannot be restarted because a serious error has been r

19. The messaging engine's instance unique id (INC_UUID) does not match that found in the data store in clustering fail over environment (1608951)
You are seeing the following error message generated in the SystemOut.log during a WebSphere Application Server clustering fail over scenario for your Messaging Engine. CWSIS1535E: The messaging engine's unique id does not match that found in the data store. ME_UUID=ME1, INC_UUID=INC1, ME_UUID(DB)=ME1, INC_UUID(DB)=INC2 (ME_UUID is same, where as INC_UUID is different in this case)

20. CWSIS1519E error occur when Messaging Engine failed to obtain lock on failover in clustering environment (1608885)
Your WebSphere Application Server Message Engine (ME1) did not release a data store lock when it failed. Now a new instance of the Message Engine (ME2) can not obtain the lock and issues message CWSIS1519E: Messaging engine MyCluster.000-MyBus cannot obtain the lock on its data store after fail over to other server in a clustering environment.

21. CWSIS1535E The messaging engine's unique id (ME_UUID) does not match that found in the data store. (1608913)
When trying to start a WebSphere Application Server Messaging Engine, it fails to start and in the SystemOut.log you see this error: CWSIS1535E: The messaging engine's unique id does not match that found in the data store.

22. PM13911; 7.0.0.7: Problems using SQL Server as a data store (4027230)
Cannot configure Microsoft SQL Server as a data store when using a JDBC 2.0 driver, or without manually creating the schema

23. Messaging stops with the CWSIS1501E message in log files when you use Service Integration Bus messaging and DB2 Version 9.5 for the message store (1381507)
When you use Service Integration Bus messaging to send and receive large messages and use IBM DB2 Version 9.5 for the message store, file system usage by the database increases. The additional space is not reused. Thus, the database system might reach its resource limits. If the resource limit is exceeded, further messaging cannot occur. Therefore, the following message is written to the FFDC log file and system output log files: CWSIS1501E: The data source has produced an unexpected exception: com.ibm.db2.jcc.b.SqlException: Unable to allocate new pages in table space "USERSPACE1"

24. The default messaging provider for WebSphere Application Server reports SQL0964C errors indicating that DB2 Version 9.5 database logs are full (1377796)
The logs of a DB2 Version 9.5 database become full when it is used as the data store for a messaging engine in a service integration bus, and automatic statistics collection is enabled in the database.

25. Messaging Engine fails to start (1407516)
Messaging Engine failed to start with the following error messages in the logs: CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data source has produced an unexpected exception: com.ibm.db2.jcc.b.SqlException: DB2 SQL error: SQLCODE: -552, SQLSTATE: 42502, SQLERRMC: WPSADMIN;CREATE SCHEMA... CWSID0035E: Messaging engine XYZ cannot be started; detected error reported during com.ibm.ws.sib.msgstore.impl.MessageStoreImpl start() 00000028 SibMessage CWSID0027E: Messaging engine XYZ cannot be restarted because a serious error has been reported.

The following technotes address issues in WebSphere Application Server 6.1

1. PK61176; 6.1.0.17: Problems after a rollback occurs within a xa_prepare call (4019239)
An FFDC is thrown showing an XidUnknownException from XidManager.rollback() on the application server hosting a Messaging Engine.

2. PM10591; 6.1.0.29: filestore never recovers from ObjectStoreFullException (4026719)
If the filestore becomes full (CWSOM1042E), no more messages can ever be sent even if all existing messages are consumed.





Webcast Replays


1. Webcast replay: Service Integration Bus and Stuck Messages (7036467)
This IBM WebSphere Support Technical Exchange is designed to address Service Integration Bus Messaging Engine start-up problems and solutions - specifically common Messaging Engine startup failures and how to identify a proper Messaging Engine startup. Solutions to common startup failures are also presented.

2. Webcast replay: WebSphere Application Server - Messaging Engine startup problems (7013517)
This IBM WebSphere Support Technical Exchange is designed to address Service Integration Bus Messaging Engine start-up problems and solutions - specifically common Messaging Engine startup failures and how to identify a proper Messaging Engine startup. Solutions to common startup failures are also presented.

3. Webcast replay: WebSphere Application Server - Service Integration Bus Messaging Engine Data Store Connectivity Problems and Solutions (7020333)
This WebSphere Support Technical Exchange is designed to describe how Service Integration Bus Messaging Engines connect to data stores and how to recover from unexpected disconnections from data stores.

4. Webcast replay: WebSphere Application Server - Message Store Overview (7016476)
This WebSphere Support Technical Exchange discusses SIB Message Stores, possible Message Store configurations, how Message Stores work, and common Message Store problems and solutions in WebSphere Application Server.

5. Webcast replay: WebSphere Default Messaging Provider: Concept and Architecture Overview (7019098)
This WebSphere Support Technical Exchange is designed to present an overview of the concepts, architecture of the Service Integration Bus, an implementation of the Default Messaging Provider in WebSphere Application Server V6 and V7.

6. Ask the Experts Replay: Service Integration Bus Scalability Best Practices (7021025)
The Service Integration Bus messaging provider in WebSphere Application Server can scale from a single server up to large environments with multiple clusters. This Ask the Experts session discusses best practices for exploiting the scalability of the Service Integration Bus to improve performance.

7. Webcast replay: SSL Configuration of the Java EE Application Client and the WebSphere Application Server V7 Service Integration Bus (7019792)
This WebSphere Support Technical Exchange is designed to provide a step-by-step walkthru of the SSL configuration for the Java EE Application Client and WebSphere Application Server V7 Service Integration Bus.

8. Webcast replay: External messaging clients for the Service Integration Bus in WebSphere Application Server (7016041)
This WebSphere Support Technical Exchange is designed to offer detailed explanation of the various external messaging clients available for the Service Integration Bus in WebSphere Application Server V6.x and V7.



Other useful links for Service Integration Bus


1. Tuning the WebSphere Service Integration Bus Heartbeat Properties (1422114)
What are the properties 'com.ibm.ws.sib.jfapchannel.HEARTBEAT_INTERVAL' and 'com.ibm.ws.sib.jfapchannel.HEARTBEAT_TIMEOUT' and how do you tune them?

2. IBM Service Integration Bus Destination Handler, Version 1.1 (4021439)
A tool to view, move, copy, delete, save and restore messages in the default messaging provider for IBM WebSphere Application Server versions 7.0, 6.1, and 6.0.

3. How to setup a Service Integration Bus to send an MQRFH2 back through WebSphere MQ (1396144)
If a WebSphere Application Server application is configured to accept an MQRFH2 in a Request/Reply message scenario, how do you send an MQRFH2 header back to the ReplyToQueue if the same path may be used for messages where the MQRFH2 cannot be sent back?

4. Current transaction errors occur when you start the server (1237714)
If you restart a server while performing messaging with a service integration bus, you might see the following error message in the system out log file (SystemOut.log) on distributed platforms, or in output of the appropriate job log on z/OS: CWSIP0671E: The destination destination_name on messaging engine messaging_engine_name failed to be initialized for maintaining order. Several instances of this message might occur during server restart.

5. Troubleshooting Service integration in WebSphere Application Server V7.0
Troubleshooting Service integration in WebSphere Application Server V8.0
Troubleshooting Service integration in WebSphere Application Server V8.5
These pages in the Information Centers provide a starting point for finding information about service integration.

5. Messaging engine troubleshooting tips in V7.0
Messaging engine troubleshooting tips in V8.0
Messaging engine troubleshooting tips in V8.5
Use this set of specific tips in the Information Centers to help you troubleshoot problems with service integration messaging engines.





Other Knowledge Collections for WebSphere Application Server


1. Knowledge Collection: WebSphere Application Server Messaging Engine startup problems and their solutions(7036148)

2. Knowledge Collection: Migration planning for WebSphere Application Server (7008724)

3. Knowledge Collection: Migrating to (or from) WebSphere Application Server V8.0 (7008725)

4. Knowledge Collection: Migrating from other Application Servers to WebSphere Application Server (7008729)

5. Knowledge Collection: IBM Support Newsletters (7011428)

6. Knowledge Collection: Java Message Service (JMS) for WebSphere Application Server (7020347)

7. Knowledge Collection: Plug-in for WebSphere Application Server (7021301)

8. Knowledge Collection: Out of Memory OOM for WebSphere Application Server (7021664)

9. Knowledge Collection: Migrating to WebSphere Application Server V8.5 (7008727)

10. Knowledge Collection: Migrating from WebSphere Application Server V6.1 (7008728)

11. Knowledge Collection: Migrating to (or from) WebSphere Application Server V7.0 (7013842)


Definition of a Knowledge Collection
A Knowledge Collection is a focused compilation of links to documents that share a common theme. Knowledge Collections are navigation aids that organize content to help users quickly find relevant information. Knowledge Collections are not designed to be an all-inclusive list of all documents dealing with the specific theme.

Original publication date

2013/6/17

Cross reference information
Segment Product Component Platform Version Edition
Business Integration WebSphere MQ Service Integration Technologies /SIB
Business Integration IBM Business Monitor
Business Integration WebSphere Enterprise Service Bus
Business Integration WebSphere Lombardi Edition
Business Integration WebSphere Partner Gateway Advanced Edition
Business Integration WebSphere Partner Gateway Enterprise Edition
Business Integration WebSphere Process Server
Business Integration WebSphere Service Registry and Repository
Messaging Applications IBM Connections Content Edition

Product Alias/Synonym

WebSphere Application Server WAS SIB SIBUS SI BUS

Rate this page:

(0 users)Average rating

Document information


More support for:

WebSphere Application Server
Service Integration Technology

Software version:

6.1, 7.0, 8.0, 8.5, 8.5.5

Operating system(s):

AIX, HP-UX, IBM i, Linux, Solaris, Windows

Software edition:

Advanced, Base, Express, Network Deployment, Single Server

Reference #:

7038698

Modified date:

2013-06-17

Translate my page

Machine Translation

Content navigation