Collect troubleshooting data for WebSphere Business Monitor V6.1, V6.2 and V7.0

Technote (troubleshooting)


Problem(Abstract)

You are having a problem with WebSphere Business Monitor V6.1, V6.2 and V7.0. You would like to know what documentation you must collect (MustGather) so that the WebSphere Business Monitor Support team can diagnose your problem. If you gather this documentation before contacting support it will expedite the troubleshooting process, and save you time.

Resolving the problem


MustGather: Read first for WebSphere Business Monitor V6.1, V6.2 and V7.0

Learn more about MustGather
Collect MustGather data
Additional diagnostic and troubleshooting information
Submit MustGather data to IBM Technical Support
Collect MustGather data for related products
Related information


Learn more about MustGather

The term "MustGather" represents the diagnostic data (system information, symptoms, log files, traces, and so on) that is required to resolve a problem. By collecting MustGather data early, even before you open a PMR, you help IBM Support to quickly determine the following:
  • If symptoms match known problems (rediscovery)
  • If you have a non-defect problem that can be identified and resolved
  • If there is a known defect and a workaround exists to provide relief

Collect MustGather data

You can choose to collect data automatically or manually. Review the following sections to determine what information should be collected for IBM support.

Automatically collect MustGather data

Automated data collectors are now available in the IBM Support Assistant Workbench to improve accuracy of complete data collection.

For WebSphere Business Monitor V6.1 and later, use one of the following IBM Support Assistance offerings:

Also, review the sections below to manually collect additional information and review suggested trace settings.

Use the Service tab of IBM Support Assistant, or if entering the output into an electronically opened PMR, use the Service Request (SR) tool. Enter "General Information -summary" output into an electronically opened PMR to eliminate waiting on the phone to provide general information to Level 1 support.

For a show-me demo of IBM Support Assistant and the preceding features, see the IBM Support Assistant Overview.

General Diagnostic Information

Collect the following general MustGather data to provide to support if not already supplied:
  1. Problem Description: Provide a detailed description of your problem. Include sample input values as needed, along with your own analysis of the problem.
    • Provide detailed instructions for how to recreate your issue.
    • Provide the names of any models, events, or metrics that you believe might be involved in the issue.
    • Provide how often the problem occurs. Include details about recent problem occurrences, including timestamps.
    • Provide any workarounds you use for the issue.
    • If you have not provided it already, supply the impact this issue is having on your company, including:
      • Any upcoming deadlines and dates
      • If the problem is seen in production, development, or testing
      • Any realized or potential affect on customers

  2. Environment Description: Provide information about your WebSphere Business Monitor environment:
    • Hardware specifications of the workstation. If you have a cluster, provide this information for all the workstations in the cluster. This information should include:
      • The number and speed of the CPUs
      • The amount of physical memory
      • The size of the hard disks
    • From the bin directory of each WebSphere Business Monitor installation, run the versionInfo command with the maintenancePackages flag:
      versionInfo -maintenancePackages
    • If this is a WebSphere Business Monitor development toolkit installation, go to Start > All Programs > IBM Installation Manager > View Installed Packages. Save the file that is opened in the Web browser.
    • Provide the versions of all databases used and information on the configuration of the database server. If using DB2, use the db2support utility to collect environment information. Be sure to delete or archive all outdated files in the DB2 diagnostic path before trying to recreate the problem and use the db2support utility. The DB2 diagnostic path is specified by the DIAGPATH parameter in the database manager configuration. After recreating the issue, run the following command:
      db2support . -d <database-name> -c -g -s
      For more information on the db2support utility, refer to the DB2 V9 Information Center.
    • Provide the version of the operating system. If you have a cluster, specify the versions of the operating systems for all the workstations in the cluster.
    • Gather the following security configuration files from the standalone server or cell deployment manager:
      <profile_root>/<profile_name>/config/cells/<cell_name>/security.xml

      <profile_root>/<profile_name>/config/cells/<cell_name>/wim/config/wimconfig.xml
      • If the problem being reported is dashboard-related, specify the version of the Web browser being used.
      • If the problem is occurring in a network deployment environment, describe the topology of your environment including machine names, cluster names, and graphical representation, if possible.

  3. Server logs: Provide log information for each of the involved servers:
    • State the exact error message received
    • Specify the log file in which the error occurs, as well as the associated time stamp
    • Gather the logs and FFDC files from each server that is involved in the issue.
      • If using the default WebSphere Business Monitor profile directory, package the following directories:
        <WBM_installPath>/profiles/<profile_name>/logs/<server_name>

        <WBM_installPath>/profiles/<profile_name>/logs/ffdc
      • If using the WebSphere Business Monitor development toolkit test server, package the following directories under WebSphere Integration Developer or Rational Application Developer:
        <WID_installPath>/pf/<profile_name>/logs/<server_name>

        <WID_installPath>/pf/<profile_name>/logs/ffdc

        [for V7.0 only]
        <WID_WTE_installPath>/runtimes/bi_v7/profiles/<profile_name/logs
        or:
        <RAD_installPath>/pf/<profile_name>/logs/<server_name>

        <RAD_installPath>/pf/<profile_name>/logs/ffdc

        [for V7.0 only]
        <RAD_WTE_installPath>/runtimes/bi_v7/profiles/<profile_name/logs
      • If using a custom profile path, package the following directories:
        <profile_root>/<profile_name>/logs/<server_name>

        <profile_root>/<profile_name>/logs/ffdc
    • Note: In the section below titled "Manually collect component-specific MustGather data," you might be asked to set the trace string to a particular value depending on the problem area if the problem is reproducible. Setting the trace string causes trace output to be written to a file named trace.log in the appropriate log directory for the server or profile, above. After setting the trace string, you must recreate the error or error conditions. For more information on tracing and logging configuration, visit the WebSphere Application Server V6.1 Information Center.

  4. Project Interchange & EAR file: For issues encountered while developing, deploying, or running a monitor model, create a Project Interchange and EAR file for specific test scenarios. If possible, simplify the scenario to isolate the problem.
    Create the Project Interchange file as follows:
    1. From within WebSphere Integration Developer, select File > Export.
    2. Select Project Interchange, and select the modules relevant to your project, then export them to a file with checkbox enabled for Include derived files. Provide the most simplified test case that exhibits your problem.
    3. Include any external dependencies, such as JAR files, that are needed to run your test scenario.
    4. Include any input data or files needed to run the test scenario.

    Create the EAR file as follows:
    1. From within WebSphere Integration Developer, select File > Export.
    2. Select EAR file and select the generated J2EE project from the EAR application drop-down menu relevant to your monitor model. Specify the destination and complete the export.

Manually collect component-specific MustGather data

If your problem falls into one of the following categories, collect the following information in addition to the preceding general information. If the problem is reproducible, set the appropriate trace string in that section prior to reproducing the problem.
  1. Installation Issues
    1. For development toolkit install problems:

      C:\Documents and Settings\All Users\Application Data\IBM\Installation Manager\logs
      C:\Documents and Settings\All Users\Application Data\IBM\Installation Manager\histories
      C:\Documents and Settings\All Users\Application Data\IBM\Installation Manager\installRegistry.xml
      C:\ProgramData\IBM\Installation Manager\logs
      [for Windows 7]
      Note: You will need to enable viewing of hidden folders to access the ProgramData directory and its subdirectories.

    2. For WebSphere Business Monitor test server install problems during a development toolkit installation:

      <RAD_installPath>\runtimes\base_v61\logs [for V6.1 or V6.2]
      <RAD_WTE_installPath>\runtimes\bi_v7\logs [for V7.0]
      or
      <WID_installPath>\runtimes\bi_v61\logs [for V6.1]
      <WID_installPath>\runtimes\bi_v62\logs [for V6.2]
      <WID_WTE_installPath>\runtimes\bi_v7\logs [for V7.0]

    3. For WebSphere Business Monitor V6.1 & V6.2 server install problems:

      Install logs:
      <WBM_installPath>\logs
      or
      <SYSTEM_TEMP>/niflogs.wbm/
      Profile related logs:
      <profile_root>\<profile_name>\logs
      .nifregistry file:
      C:\Windows\.nifregistry [for Windows]

      /opt/.ibm/.nif/.nifregistry [for Linux & UNIX platforms, root users]
      <NON_ROOT_HOME>/.ibm/.nif/.nifregistry [for Linux & UNIX platforms, non-root users]

    4. For WebSphere Business Monitor V7.0 server install problems:

      Install logs:
      <WBM_installPath>\logs
      Installation Manager logs for root installations:
      /var/ibm/InstallationManager/logs [for Linux & UNIX platforms]

      c:\Documents and Settings\All Users\Application Data\IBM\InstallationManager\logs [for Windows]

      C:\ProgramData\IBM\Installation Manager\logs [for Windows 7]
      Note: You will need to enable viewing of hidden folders to access the ProgramData directory and its subdirectories.
      Installation Manager logs for non-root installations:
      <UNIX_USER_HOME>/var/ibm/InstallationManager/logs [for Linux & UNIX platforms]

      c:\Document and Settings\<WINDOWS_USER_ID>\Application Data\IBM\InstallationManager\logs [for Windows]
      Profile related logs:
      <WBM_installPath>\logs\manageprofiles
      <profile_root>\<profile_name>\logs
      NIFStack.xml file:
      <WBM_installPath>/properties/version/nif/backup/NIFStack.xml

  2. Development Toolkit Issues (Modeling, Monitor Model Editor, Integrated Test Client, Debugger)
    1. For all problems:

      <workspace_home>\.metadata\.log

    2. For debugger problems:
      Set the trace string on the tester server to:
      *=info: com.ibm.wbimonitor.*=all
      Provide the Project Interchange files with monitor model projects and event type definitions referenced in the Event Model.

  3. Server Issues
    1. For monitor model installation and configuration problems:
      Set the trace string on the deployment manager or standalone server to:
      *=info: com.ibm.wbimonitor.lifecycle.*=all: com.ibm.wbimonitor.repository.*=all: com.ibm.wbimonitor.persistence.*=all: com.ibm.wbimonitor.schemagenerator.*=all:
      com.ibm.wbimonitor.observationmgr.*=all

      Capture the content of the following database tables in the Monitor schema:
      META_MODEL_STEP_T
      META_MODEL_T
      META_MODEL_UNVERSIONED_T

    2. For monitor model runtime problems:
      Using the Group menu instead of the Component menu in the WebSphere Application Server administrative console, set the trace where the monitor model moderator and logic are deployed for the monitor model version. The group will be listed with the name "<model name>_<version>" and so the resulting trace string would be:
      *=info: <model name>_<version>=all

    3. For failed and unrecoverable event problems:
      Set the trace string on the deployment manager and on the cluster or server to where the model is deployed:
      *=info: com.ibm.wbimonitor.observationmgr.runtime.failedevents.*=all: com.ibm.wbimonitor.errorq.*=all: com.ibm.wbimonitor.deadq.*=all: com.ibm.wbimonitor.admin.errorq.*=all: com.ibm.wbimonitor.admin.deadq.*=all
      If the issue is related to resuming an instance, also set the trace specified above under the section for monitor model runtime problems.

    4. For event processing problems for a model using queue based routing:
      Set the trace string on each CEI target server or cluster to:
      *=info: com.ibm.events.distribution.*=all

      For event processing problems on V6.2 or V7.0 for a model using queue bypass routing:
      Set the trace string on each CEI target server or cluster and on the monitor model moderator to:
      *=info: com.ibm.wbimonitor.router.*=all

      In addition, for any model regardless of routing method, set the trace specified above under the section for monitor model runtime problems.

    5. For Monitor event emitter service API problems on V6.2 or V7.0:
      Set the trace string to:
      *=info: com.ibm.wbimonitor.emitterrest.*=all

    6. For data movement services problems:
      Set the trace string to:
      *=info: com.ibm.wbimonitor.dmsservice.*=all: com.ibm.wbimonitor.repository.*=all: com.ibm.wbimonitor.scheduler.*=all

    7. For Monitor action services and alerts problems:
      Set the trace string on the deployment manager and on where action services are deployed to:
      *=info: com.ibm.wbimonitor.actionmgr.*=all: com.ibm.wbimonitor.persistence.*=all

    8. For database (retrieval and update) problems:
      Set the trace string to:
      *=info: com.ibm.wbimonitor.persistence.*=all: com.ibm.wbimonitor.repository.*=all

    9. For KPI related problems:
      Set the trace string to:
      *=info: com.ibm.wbimonitor.rest.*=all: com.ibm.wbimonitor.repository.*=all: com.ibm.wbimonitor.resources.*=all: com.ibm.wbimonitor.kpi.*=all

  4. Dashboard Issues
    1. For dashboard problems related to data retrieval and the problem persists across widgets:
      Set the trace string where REST Services is deployed to:
      *=info: com.ibm.wbimonitor.rest.*=all: com.ibm.wbimonitor.repository.*=all: com.ibm.wbimonitor.resources.*=all
      Capture the content of the following database tables in the Monitor schema, related to Monitor data security configuration:
      RAT_RESOURCE_GROUPS
      RAT_AUTH_DOMAIN
      RAT_PARTY_ROLES

    2. For Web-based dashboard problems on V6.1 or V6.1.1:
      Set the trace string where the dashboards are deployed to:
      *=info: com.ibm.wbimonitor.BusinessDashboard=all

    3. For Business Space dashboard problems on V6.1.2 or later:
      Set the trace string on the deployment manager and where Business Space is deployed to:
      *=info: com.ibm.bspace.*=all: com.ibm.wbimonitor.BusinessDashboard=all

      If on V7.0, please also add the following additional trace string to those listed above:
      com.ibm.mm.*=all

      Save the content of the debug console that appears in Business Space to a file and provide after reproducing the issue.

      NOTE: It is important that you also include the security configuration files mentioned under Environment Description of the General Diagnostic Information section above.

      Also gather the following Business Space configuration files:
      <DMGR_PROFILE_HOME>/config/cells/<CELL_NAME>/nodes/<NODE_NAME>/servers/<SERVER_NAME>/config-bspace.xml [if Business Space was configured for a single server]

      <DMGR_PROFILE_HOME>/config/cells/<CELL_NAME>/clusters/<CLUSTER_NAME>/config-bspace.xml [if Business Space was configured for a cluster]

      <WAS_ROOT>/BusinessSpace/bspace.versions.txt [for V7.0]

    4. For WebSphere Portal dashboard problems:
      Set the trace string on the WebSphere Portal server to:
      *=info: com.ibm.wbimonitor.BusinessDashboard=all: com.ibm.wbimonitor.proxy.*=all

      Also gather the following:
      <PORTAL_ROOT>\log
      <PORTAL_WAS_PROFILE_NAME>\logs

    5. For Alphablox related problems:
      If on V7.0, for any issues with Alphablox startup, set the trace string on all servers that have been deployed with the Alphablox applications (AlphabloxPlatform and AlphabloxStudio):
      *=info: com.ibm.wbimonitor.admin.command=all

      <ABX_ROOT>\Alphablox\repository\servers\AlphabloxAnalytics\logs


      NOTE: Review the Alphablox technote for Troubleshooting & Collecting Diagnostic Data for further details.

  5. Performance-related Issues
    1. For Server-related performance issues:
      • How many models and versions of each model are deployed?
      • What is the rate at which events are emitted to the WebSphere Business Monitor server (For example: how many events per hour or day)? Are there any peak periods in a day, week, or year?
      • How many events constitute a typical instance?
      • How large is the business data associated with an event?
      • Does the issue occur just at server startup or at steady state?
      • How many instances exist for each model version?
      • If on V6.2 or V7.0, what are the configured refresh rates for WebSphere Business Monitor scheduled services?
      • If the issue is specific to a monitor model or monitor model version, provide the runtime configuration settings used for each monitor model.
    2. For Dashboard-related performance issues:
      • On average, how long is the Web browser left open to view the dashboards?
      • What is the configured session timeout period?
      • What is the configured refresh rate for the widgets?
      • What is the configured refresh rate for the Alphablox cubes?
      • What is the average time it takes to log in? What is the average time it takes to navigate to a page?
      • How many cubes are present in Alphablox and are they all started?
      • What is the average number of business users? Are there any peak times?
    3. In addition, review the WebSphere Application Server MustGather for additional guidance on information to gather.
    4. For issues related to DB2 performance such as deadlocks and lock timeouts, review the DB2 MustGather for additional guidance on information to gather.
    5. For issues where the WebSphere Business Monitor server appears to have stopped consuming events or may be deadlocked, gather a thread dump from the Moderator and Model Logic targets. Review the WebSphere Application Server Information Center for additional information on how to dump threads.

  6. Mobile Device Issues
    1. Set the trace string to:
      *=info: com.ibm.wbimonitor.mobile.*=all

  7. Managed Deployment Environment (for WebSphere Business Modeler) Issues
    1. Provide the model MAR file from the WebSphere Business Modeler client workspace.
    2. Starting with V6.2.0.1, the problem determination (PD) archive file created by WebSphere Business Modeler contains information about WebSphere Business Monitor. In WebSphere Business Modeler, the PD archive file is generated when you click Request Help from IT on the "Deployment Error" message window.
    3. Set the trace string on the deployment manager or server:
      *=info: com.ibm.wbimonitor.mss.*=all
    4. Gather the following directories:

      <PROFILE_NAME>\logs
      <PROFILE_NAME>\mde.wbm
      <OS_TEMP_DIR>\mmdeploy-<RANDOM_#>


Additional diagnostic and troubleshooting information
  • Event flow issues

    If your WebSphere Business Monitor model cannot consume events, use the Event flow diagnostic and validation tool to determine where events are encountering interference and how to fix basic problems.

  • Monitoring environment restart order

    To avoid problems when restarting your monitoring environment, you must stop and restart the monitoring environment using the following procedures:

    Stopping the monitoring environment:
    1. Stop the server or cluster hosting the monitor model applications. If you have deployed the monitor model logic and moderator modules on separate clusters or servers, then stop the cluster or server that hosts the monitor model moderator module first.
    2. Stop the monitor model application(s). If you have deployed the monitor model logic and moderator on separate clusters or servers, then stop the monitor model moderator module first.
    3. Stop the server or cluster hosting the CEI server.
    4. Stop the server or cluster hosting the messaging engine.

    Starting the monitoring environment:
    1. Start the server or cluster hosting the messaging engine.
    2. Start the server or cluster hosting the CEI server.
    3. Start the monitor model application(s). If you have deployed the monitor model logic and moderator on separate clusters or servers, then start the monitor model logic module first.
    4. Start the server or cluster hosting the monitor model applications. If you have deployed the monitor model logic and moderator modules on separate clusters or servers, then start the cluster or server that hosts the monitor model logic module first.

Submit MustGather data to IBM Technical Support

To diagnose or identify a problem, it is sometimes necessary to provide Technical Support with data and information from your system. In addition, Technical Support might also need to provide you with tools or utilities to be used in problem determination. You can submit files using one of following methods to help speed problem diagnosis:
For information about using each of these services, see Exchanging information with IBM Technical Support.

Collect MustGather data for related products

MustGather: Read first for WebSphere Application Server
Collect troubleshooting data for WebSphere Business Modeler
Collect troubleshooting data for WebSphere Business Modeler Publishing Server
MustGather: Read first for WebSphere Business Services Fabric
Collect troubleshooting data for WebSphere Integration Developer
MustGather: Read first for WebSphere Process Server
MustGather: Read first for WebSphere Service Registry and Repository

Related information

Search for known problems on the WebSphere Business Monitor Support page.

See the Software Support Handbook for guidelines and references about IBM Support.

Visit the WebSphere Business Monitor Information Center for online documentation about this product.

Learn about Monitor using IBM Education Assistant which integrates narrated presentations, Show Me Demonstrations, tutorials, and resource links to help you successfully use IBM products.

Rate this page:

(0 users)Average rating

Document information


More support for:

WebSphere Business Monitor
General

Software version:

6.1, 6.1.1, 6.1.2, 6.2, 6.2.0.1, 6.2.0.2, 6.2.0.3, 7.0, 7.0.0.2, 7.0.0.3, 7.0.0.4, 7.0.0.5

Operating system(s):

AIX, HP-UX, Linux, Linux zSeries, Solaris, Windows

Reference #:

1405891

Modified date:

2009-11-27

Translate my page

Machine Translation

Content navigation