IBM Support

MustGather: Read first for WebSphere Service Registry and Repository

Troubleshooting


Problem

Collecting diagnostic data (MustGather) aids in problem determination and saves time when resolving Problem Management Records (PMRs) for WebSphere Service Registry and Repository.

Resolving The Problem

Learn more about MustGather
Collect MustGather data
Collect MustGather data for ServiceRegistryDashboard problems
Collect MustGather data for Promotion problems
Collect MustGather data for SQL performance issues
Collect MustGather data for WSRR database on DB2 problems
Learning more about MustGather

The term "MustGather" represents the diagnostic data (system information, symptoms, log files, traces, and so on) that is required to resolve a problem. By collecting MustGather data early, even before you open a PMR, you help IBM Support quickly determine the following:
  • If symptoms match known problems (rediscovery)
  • If you have a non-defect problem that can be identified and resolved
  • If there is a defect that identifies a workaround to reduce severity
  • If locating the root cause can speed development of a code fix
 
Collect MustGather data
 
Manually collect MustGather data

Collect the following information for MustGather data:
  • WebSphere Service Registry and Repository version details
    It is important to provide details of exactly which version of WSRR is being used. This includes Fix Pack and iFix details. The simplest way to determine the exact version of WSRR is to look at the welcome page in the WSRR Web User Interface.
    Look for the section titled About your Service Registry and Repository.
    This shows the version number and build number. For example:

    8.5.0.1
    Build Number: 20140814-1023


    Both these pieces of information should be included when raising a problem relating to WSRR.
  • Business Space version details (WSRR V7.0.0 to V8.0.0 only)
If the problem relates to Business Space also collect the Business Space version information. Upload the following file:
 
WAS_HOME/BusinessSpace/bspace.versions.txt
WAS_HOME/profiles/profile_name/logs/server_name
The support teams might ask for more logs during the problem diagnosis. However, by default include the contents of this directory. It is recommended that you compress the logs directory using compression tools such as zip, or tar and gzip.

In addition to this directory, the /ffdc is often useful. This is located at the same level as the server_name directory. For example:
WAS_HOME/profiles/profile_name/logs/ffdc
Trace files are not generated by default. When trace is switched on, output is sent to the logs/server_name/trace.log. In the case where the maximum rollover log file size is reached, this file is renamed to a file with a name of the form:
logs/server_name/trace_<TIMESTAMP>.log
  • Log and trace files
    Often the most important of the user-supplied information to assist with the diagnosis of problems. All log and trace files are stored on your machine within the /profiles directory structure.
     
  • Switching trace on for WebSphere Service Registry and Repository
    The simplest way to turn on trace is by using the administrative console. To do this, start a session to the administrative console and log in. You might need to alter the filter at the bottom of the main panel in order that you can see the Troubleshooting in the left column.

  • 1. Select Troubleshooting > Logs and Trace.


    2. Select your server from the list.


    3. Change Log Detail Levels. You are presented with a free-form text box, which contains (by default) the string *=info. To trace general WebSphere Service Registry and Repository problems, this trace string should be replaced with the following string.

    Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
     
    *=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all
    4. Click OK and save these changes.


    5. Restart the server to initiate the production of trace.


    6. At this point, re-create the problem with the minimum set of steps and send the trace log files into the support group.

Note: If to re-create the problem you have to perform a number of steps or if some of the steps take a long time to complete a lot of trace can be produced and the trace files might be overwritten. The default trace log size is 20MB, and one historical file is kept, this is often not sufficient. It is recommended to use 100MB trace files and to retain 10 historical trace log files. Ensure you have adequate free disk space before changing the trace settings.
 
  • Configuration profiles
    WSRR stores all of the configuration information within a configuration profile. There can be multiple configuration profiles loaded but only one active profile. It is often useful to provide the Technical Support with the entire active configuration profile.

    To export the configuration profile navigate to the configuration perspective in the WSRR UI.

    1. Select Manage Profiles -> Configuration Profiles

    2. Check the box next to the configuration profile marked 'Active'.

    3. Press the Export button

    4. Save the zip to your local machine and send it to the support team.

     
  • Problem Description
    Provide a detailed description of the problem, including in detail, the steps needed to re-create. If the problem is related to a particular artifact like a WSDL file or an XSD then if possible provide that artifact and any of the files that it depends on both directly and indirectly.
  • Operating system and hardware architecture
    A brief description of the OS and architecture type is useful to determine which version of the Java™ SDK you are using. For example: Solaris 10 (SPARC), or Linux RHEL AS 4 (zSeries).
  • Deployment topology
    Indicate if you are using the product deployed in a cluster, and if so indicate if all the machines in the cluster are of the same OS type and version.
 

 
Gather data relating to problems with Service Registry Dashboard
The ServiceRegistrtyDashboard replaces the Business Space user interface since WSRR V8.5.

For problems where the Dashboard is not accessible or where widgets are appearing but their contents is blank, the following trace should be collected. The trace should be enabled and then the Application Server restarted before attempting to access the dashboard.

Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
 
*=info:com.ibm.cre*=all:com.ibm.sr.dashboard.debug*=all: com.ibm.sr.dashboard.cre*=all: com.ibm.sr.dashboard.client=all: com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.dashboard.noserver*=all:com.ibm.mm.proxy.connection*=all

If the Dashboard is accessible but there is a problem occurring while it is being used, the following trace should be captured. There is no need to restart the Application Server after applying this runtime trace, just enable the trace and re-create the problem.

Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
 
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.dashboard.client=all:com.ibm.sr.ui*=all

 
Gather data relating to problems with Promotion
If a problem occurs during a promotion operation there are a number of pieces of information that can help diagnose the problem. Most importantly is the PromotionProperties configuration. This is contained in the WSRR configuration profile and the easiest way to gather it is to export the entire configuration profile.

Promotion will always involve the promotion of data from one system to at least on other. Trace should be collected from ALL the systems involved in the promotion. It is important to identify these systems clearly when providing trace. Typically the source system is referred to as the "Governance Master" and each of the target systems as a "Runtime".

The following trace string should be used when gathering data for a promotion problem.

Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
 
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.promotion*=all



 
Gather data relating to SQL performance
For problems where you believe SQL queries run by WSRR are running slower than expected enable the following trace before re-creating the problem. Send the entire logs and ffdc directories for the support team to review.

Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
 
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.athene.stats.Performance=all:com.ibm.athene.util.SQLLogger=all
 

 
Gather data relating to the DB2 database
If the WSRR database is running on DB2 there is some useful data that can be collected about the database and its environment.

To aid the collection of such information, the commands and queries to be run are listed on this page so the DBA can collect most of the important information in one go.

Substitute DBNAME for the WSRR database name, USERNAME for the database user and PASSWORD for the database user's password.
  1. db2look -d DBNAME -e -o db2look.ddl -a -l -f
  2. db2 get dbm cfg > dbmcfg.out  
  3. db2 get db cfg for DBNAME > dbcfg.out
  4. db2 connect to DBNAME user USERNAME using PASSWORD
  5. db2 -tvf table.sql > table.out
  6. db2 -tvf index.sql > index.out
  7. db2 -tvf buffer.sql > buffer.out
  8. db2level > db2level.out
The files, table.sql, index.sql and buffer.sql are provided in this zip file.

Db2dc.zip


 
Gather data relating to problems with WSRR Business Space performance
The following instructions detail how to collect information relating to WSRR Business Space performance.

Enable WebSphere Application Server tracing with the following trace string:

 
*=info:com.ibm.serviceregistry.feeds.api=finer:com.ibm.sr.perf=all:com.ibm.sr.api.*=finer


Then using IE9 (or higher):
  • Start IE9
  • Go to the login page for Business Space, but do not log in yet
  • Press F12 to bring up the debugger and go to the Network tab and click the Start capturing button
  • Log in to Business Space and perform the action which has poor performance Wait for all rendering to finish
  • Click the Stop capturing button in the IE debug panel and then click the Export capture traffic button to save the NetworkData.xml file
  • Stop the WebSphere Application Server trace
Send the network capture file along with the corresponding WebSphere Application Server trace.

Additionally send output from the following:
  • From a command line on the machine running the browser run a traceroute to the WSRR server machine, for example:
tracert wsrrServer.com
  • Full specification of the machine running the browser (such as that provided by Control Panel > System)

Collect MustGather data for email notification problems
The email notification system relies on a number of configuration files as well as a correctly configured mail session in WebSphere Application Server.

For any problems with the email notification system the support team will require an export of the active configuration profile running on the WSRR instance that should be sending emails.

In addition to the profile, the following trace string should be used to gather trace showing the email notification process. Send the entire logs and ffdc directories for the support team to review.

Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
 
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.subscriptionnotifier.plugin*=all

Lastly, to rule out problems with the WebSphere Application Server mail session, enable debug mode for the WSRRMailSession.

1. Open the administrative console.

2. Click Resources > Mail > Mail sessions > WSRRMailSession.

3. Click Enable debug mode. Debugging is enabled for the WSRRMailSession only.

4. Click Apply or OK.

With the trace enabled and mail debug enabled, perform the action in WSRR that should trigger an email to be sent. Note the time that the action was performed. Remember that WSRR uses an asynchronous notification mechanism so the email might not be sent for several minutes after the action was performed; this is dictated by the interval setting of the SubscriptionNotifierPluginScheduler scheduled task.

 
Collect MustGather data for SLAAttachment API problems
For problems with the SLAAttachment API, enable the following trace before re-creating the problem. Note the SLAAttachment API is used by DataPower to retrieve policy information from WSRR. Send the entire logs and ffdc directories for the support team to review.

Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
 
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.persistence.create*=all:com.ibm.sr.management*=all:com.ibm.sr.policy*=all
 
Collect MustGather data for Access Control problems
For problems with WSRR fined grained access control, such as where a user is permitted to perform an operation that you believe they should be restricted from performing, enable the following trace and re-create the problem. Send the entire logs and ffdc directories for the support team to review.

Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console. Note that this trace string includes the WebSphere Application Server security MustGather trace.
 
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.authz*=all:com.ibm.ws.security.*=all:com.ibm.websphere.security.*=all:com.ibm.websphere.wim.*=all:com.ibm.wsspi.wim.*=all:com.ibm.ws.wim.*=all

In addition to the trace, provide an export of the current active configuration profile and a description of the users or groups who demonstrate the problem. For a user, describe which WSRR roles they belong to as well as which User Registry (typically LDAP) groups they belong to.
 
Collect MustGather data for related products

MustGather: Read first for WebSphere Application Server

MustGather: Read first for WebSphere Process Server
 


 
Submit MustGather data to IBM Technical Support

To diagnose or identify a problem, it is sometimes necessary to provide Technical Support with data and information from your system. In addition, Technical Support might also need to provide you with tools or utilities to be used in problem determination. You can submit files using one of following methods to help speed problem diagnosis:
For information about using each of these services, see Exchanging information with IBM Technical Support.
 
Related information

WebSphere Enterprise Service Bus Support page

Recommended fixes for WebSphere Enterprise Service Bus
 

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSWLGF","label":"WebSphere Service Registry and Repository"},"Component":"Documentation","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"},{"code":"PF035","label":"z\/OS"}],"Version":"8.5;8.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Product Synonym

WSRR

Document Information

Modified date:
21 March 2022

UID

swg21258610