IBM Support

WebSphere eXtreme Scale monitoring console does not connect to the catalog service domain

Troubleshooting


Problem

The monitoring might not be able to establish a connection to a catalog service domain even after the catalog server processes are available. This occurs when the monitoring console process was started before the catalog processes.  This problem occurs when you run WebSphere eXtreme Scale with WebSphere Application Server.

Symptom

The monitoring console displays a connection error.  No data grid information or statistics are available.

The following stack trace can be seen in the monitoring console log (OBJECT_GRID_HOME/console/logs/tr.log):

StatsRetrieve W   The connection attempt to the catalog service domain failed. Verify that the catalog service is running. The following exception caused this error to occur: [com.ibm.websphere.objectgrid.ConnectException: java.io.IOException:Connection close: Read failed.  Possible end of stream encountered.
    at com.ibm.ws.objectgrid.ObjectGridManagerImpl.connect(ObjectGridManagerImpl.java:1258)
    at com.ibm.ws.xs.stats.client.routing.StatsRetrieverRouter.resetContext(StatsRetrieverRouter.java:183)
    at com.ibm.ws.xs.stats.client.routing.StatsRetrieverRouter.getInstance(StatsRetrieverRouter.java:159)
    at com.ibm.ws.xs.app.DomainConnectionUpdater.updateConnectionStatus(DomainConnectionUpdater.java:57)
    at com.ibm.ws.xs.app.DomainConnectionUpdater$updateConnectionStatus.call(Unknown Source)
    at constat._getStat(constat.groovy:6)
    at constat$_getStat.callCurrent(Unknown Source)
    at constat.onRetrieve(constat.groovy:21)
    at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
    at java.lang.reflect.Method.invoke(Method.java:611)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:88)
    at zero.core.groovysupport.GroovySupport.invokeMethodOnScript(GroovySupport.java:386)
    at zero.core.groovysupport.GroovySupport.invokeGroovy(GroovySupport.java:337)
    at zero.core.groovysupport.GroovySupport.invokeMethodThenScript(GroovySupport.java:176)
    at zero.core.interpreter.GroovyInterpreter.invoke(GroovyInterpreter.java:54)
    at zero.core.events.dispatcher.EventDispatcher.invoke(EventDispatcher.java:73)
    at zero.core.events.dispatcher.OneDispatcher.invoke(OneDispatcher.java:43)
    at zero.core.events.EventEngine.dispatch(EventEngine.java:305)
    at zero.core.events.EventEngine.fire(EventEngine.java:135)
    at zero.core.resource.ResourceHandler.onGET(ResourceHandler.java:38)
    at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
    at java.lang.reflect.Method.invoke(Method.java:611)
    at zero.core.interpreter.JavaInterpreter.invoke(JavaInterpreter.java:64)
    at zero.core.events.dispatcher.EventDispatcher.invoke(EventDispatcher.java:73)
    at zero.core.events.dispatcher.OneDispatcher.invoke(OneDispatcher.java:43)
    at zero.core.events.EventEngine.dispatch(EventEngine.java:305)
    at zero.core.events.EventEngine.fire(EventEngine.java:135)
    at zero.core.handlers.DefaultRequestHandler.invoke(DefaultRequestHandler.java:148)
    at zero.core.handlers.DefaultRequestHandler.onRequest(DefaultRequestHandler.java:56)
    at zero.core.cfadapter.ChannelFrameworkAdapter.fireEvents(ChannelFrameworkAdapter.java:696)
    at zero.core.cfadapter.ChannelFrameworkAdapter.service(ChannelFrameworkAdapter.java:315)
    at zero.core.cfadapter.ZeroLink.doWork(ZeroLink.java:63)
    at zero.core.cfadapter.ZeroLink.ready(ZeroLink.java:50)
    at zero.network.internal.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:492)
    at zero.network.internal.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:421)
    at zero.network.internal.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316)
    at zero.network.internal.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:278)
    at zero.network.internal.ssl.channel.impl.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1043)
    at zero.network.internal.ssl.channel.impl.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:664)
    at zero.network.internal.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1778)
    at zero.network.internal.tcp.channel.impl.WorkQueueManager.requestComplete(WorkQueueManager.java:558)
    at zero.network.internal.tcp.channel.impl.WorkQueueManager.attemptIO(WorkQueueManager.java:608)
    at zero.network.internal.tcp.channel.impl.WorkQueueManager.workerRun(WorkQueueManager.java:985)
    at zero.network.internal.tcp.channel.impl.WorkQueueManager$Worker.run(WorkQueueManager.java:1070)
    at zero.network.vendor.runtime.ThreadPoolRepositoryManager$DefaultThreadPool$Worker.run(ThreadPoolRepositoryManager.java:719)
Caused by: com.ibm.ws.xsspi.xio.exception.ObjectGridXIOException [originating=9.42.109.46:0;exid=893]: java.io.IOException:Connection close: Read failed.  Possible end of stream encountered.
    at com.ibm.ws.xsspi.xio.dispatch.MessageInfo.getMessage(MessageInfo.java:130)
    at com.ibm.ws.xsspi.xio.dispatch.MessageInfo.getMessage(MessageInfo.java:146)
    at com.ibm.ws.xsspi.xio.actor.XIORegistry.findRemote(XIORegistry.java:598)
    at com.ibm.ws.xsspi.xio.actor.XIORegistry.findRemote(XIORegistry.java:551)
    at com.ibm.ws.objectgrid.catalog.wrapper.LocationServiceWrapperXIO.buildILocationServiceClient(LocationServiceWrapperXIO.java:121)
    at com.ibm.ws.objectgrid.catalog.wrapper.LocationServiceWrapper.resetRemote(LocationServiceWrapper.java:257)
    at com.ibm.ws.objectgrid.catalog.wrapper.LocationServiceWrapper.init(LocationServiceWrapper.java:183)
    at com.ibm.ws.objectgrid.catalog.wrapper.LocationServiceWrapper.<init>(LocationServiceWrapper.java:153)
    at com.ibm.ws.objectgrid.catalog.wrapper.LocationServiceWrapperXIO.<init>(LocationServiceWrapperXIO.java:58)
    at com.ibm.ws.objectgrid.naming.LocationServiceFactory.bootstrap(LocationServiceFactory.java:438)
    at com.ibm.ws.objectgrid.naming.LocationServiceFactory.bootstrap(LocationServiceFactory.java:152)
    at com.ibm.ws.objectgrid.ObjectGridManagerImpl.connect(ObjectGridManagerImpl.java:1231)
    ... 46 more


Caused by: java.io.IOException: java.io.IOException:Connection close: Read failed.  Possible end of stream encountered.
    at RemovedByServerSerialization.RemovedByServerSerialization(RemovedByServerSerialization:0)

Resolving The Problem

Restart the monitoring console process.

[{"Product":{"code":"SSTVLU","label":"WebSphere eXtreme Scale"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"General","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"},{"code":"PF035","label":"z\/OS"}],"Version":"8.6","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
15 June 2018

UID

swg21620512