IBM Integration Bus, Version 9.0.0.5 Operating Systems: AIX, HP-Itanium, Linux, Solaris, Windows, z/OS

See information about the latest product version

Identifying problem message flows

When you need to find out whether message flows are creating an excessive number of message tree fields, then parser resource statics can provide the answer.

The statistics show the number of Fields in memory, and the approximate amount of user data-related memory that is used for the named message flow parser type. These statistics are grouped by message flow and name, and you can quickly identify when millions of fields are in memory, or gigabytes of data storage is being used.

Resource statistics

Resource statistics are collected by a broker to record performance and operating details of resources that are used by integration servers.

As a system administrator, you can use the resource statistics to ensure that your systems are using the available resources in the most efficient manner. By monitoring systems and analyzing statistical trends, you can keep system resource usage within boundaries that you consider acceptable, and help to prevent situations where system resources are overstretched and might become unavailable. Analysis of the data that is returned might require specialist skills and knowledge of each resource type.

If you detect that system resources are under pressure, you can examine the statistics that are collected by the broker to assess whether the cause of the concern is the use of those resources by processes in IBM® Integration Bus.

You must activate statistics collection because collection is not active by default. If you activate statistics, you might experience a minor degradation in operating performance of the broker or brokers for which you are collecting data. You can collect data on one or more integration servers, and all integration servers on a broker, so that you can limit the statistics gathering activity if appropriate.

Resource statistics complement the accounting and statistics data that you can collect on message flows, which are also available in the IBM Integration Explorer.

To start, stop, or check the status of resource statistics collection, use one or more of the following options:

The IBM Integration Explorer: For more information, see the Starting resource statistics collection in the IBM Integration Explorer section in the main product documentation.
See the mqsichangeresourcestats command section in the main product documentation.
See the mqsireportresourcestats command section in the main product documentation.
A CMP application: For more information, see the Working with resource statistics in a CMP application section in the main product documentation.

To view the output that is generated by statistics collection, use one or more of the following options:

The IBM Integration Explorer. Numeric data and graphs are displayed for each integration server with activated statistics collection.
An application that subscribes to a publication message, that is published by the broker every 20 seconds. The message contains the data that is collected for each integration server with activated statistics collection. The published message is available in XML format and in JSON format.
The topic for each message has the following structure:
- For XML format:
```
$SYS/Broker/broker_name/ResourceStatistics/integration_server_name
```
- For JSON format:
```
$SYS/Broker/broker_name/Statistics/JSON/Resource/integration_server_name
```
You can set up subscriptions for a specific integration server on a specific broker. For example:
- For XML format:
```
$SYS/Broker/IB9NODE/ResourceStatistics/default
```
- For JSON format:
```
$SYS/Broker/IB9NODE/Statistics/JSON/Resource/default
```
You can also use wildcards in the subscriptions to broaden the scope of what is returned. For example, to subscribe to reports for all integration servers on all brokers, use the following values:
- For XML format:
```
$SYS/Broker/+/ResourceStatistics/#
```
- For JSON format:
```
$SYS/Broker/+/Statistics/JSON/Resource/#
```

Parser resource statistics

You can view these statistics in the IBM Integration Explorer, or you can write a program that subscribes to a publication (single XML message) that returns this data.

All message flows in an integration server create parsers to parse and write input and output messages. Use the Parsers statistics to see how much resource is being used by the message trees and bit streams that these parsers own.

A statistics summary is returned, followed by an entry for accumulation by parser type for each message flow. The rows are named in the style <Message Flow>.<Parser>. A row is shown for every parser type that is used by that message flow. Extra instances are included in the accumulated statistics for each message flow.

The following table describes the statistics that are returned for each message flow parser since the integration server was last restarted.

Table 1.
Measurements	Description
Threads	The number of message flow threads that contributed to the statistics for a message flows parser type accumulation.
ApproxMemKB	The approximate amount of user data-related memory that is used for the named message flow parser type. An estimate of storage that the fields themselves are occupying in kilobytes. This value does include data that was in a specific message tree such as names, namespaces, and values. As such, this metric is intended to give a base for comparison from one message flow to another. It is not possible to calculate the exact amount of memory that is used by a parser, and so the returned value is the lowest it could possibly be.
MaxReadKB	This metric records the largest bitstream that is passed to the parser for parsing. For most transports, the input bitstream is a contiguous buffer of bytes, which is allocated when it was read in. Therefore, if `maxReadKB` is a large number, it might be contributing to a high integration server memory use.
MaxWrittenKB	Shows the largest bit stream that is written by the parser type for the named message flow.
Fields	Shows how many cached fields this parser is associated with. This number never reduces, even if the DELETE FIELD command is used, because deleted fields are reused. These fields are retained by the parser and are used for constructing the message trees.
Reads	The number of successful parses that were completed by the named message flow parser type.
TotalReads	The `totalReads` metric reports how many times the parser was assigned a bitstream to parse. These assignments might be as a result of a node reading data from a transport and assigning it to the parser, or it might be from a CREATE with PARSE clause in ESQL for example.
FailedReads	The `failedReads` metric reports the amount of `totalReads` that encountered a parsing exception. The `successfulReads` and `failedReads` add up to the `totalReads` for that parser. Examining the `failedReads` value might give an indication that a message flow is driving error handling paths, which might use more memory than expected.
Writes	The number of successful writes that were completed by the named message flow parser type.
FailedWrites	The `failedReads` metric reports the amount of `totalReads` that encountered a parsing exception. The `successfulReads` and `failedReads` add up to the `totalReads` for that parser. Examining the `failedReads` value might give an indication that a message flow is driving error handling paths, which might use more memory than expected.

Examples of reporting parser usage

The parser resource statistics do not show the number of parsers that were created. Only the following runtime command returns detailed information on each parser that was created on each thread:

mqsireportproperties broker -e integrationserver -o ComIbmParserManager -r

where broker is the name of your broker and integrationserver is the name of your integration server.

This command shows the number of parsers that are owned by each thread, and how many fields each one has. You can search the text output from this command for the keywords totalFieldsUsed or approxMemKB to see whether there are any excessively large numbers.

Capture the output of this command to a file and search for totalParsers. The results show the total number of parsers that were created on each thread. If this result is an excessive number, then the flow needs to be analyzed to see where these parsers are being created.

Although the number of parser instances is not recorded in the resource statistics, when an excessive amount of parsers are created the totalFields count is larger than expected. Even if a parser instance is not parsed to completion, it still creates a root element. This root element contributes to the totalFields count for a flow ParserName entry. So if a message flow accidentally created 100,000 parsers for example, then there would be at least 100,000 root elements that contribute to the total fields count.

The following text shows an example output from this mqsireportproperties command:

ComlbmParserManager
 uuid='ComlbmParserManager'
 userTraceLevel='none.
 traceLevel='none.
 userTraceFilter='none.
 traceFilter='none.
 vrmfintroducedAt=.7.0.0.2.
 resourceStatsReporting0n='inactive.
 resourceStatsMeasurements='<ResourceStatsSwitches ResourceType="Parsers" version='2' vrmfIntroducedAt='7.0.0.2'>
  <MeasurementV2 name="Threads" collect="on" /><MeasurementV2 name="ApproxMemKB" collect="on" />
  <MeasurementV2 name="MaxReadKB" collect="on" /><MeasurementV2 name="MaxWrittenKB" collect="on" />
  <MeasurementV2 name="Fields" collect="on" /><MeasurementV2 name="Reads" collect="on" />
  <MeasurementV2 name="FailedReads" collect="on" /><MeasurementV2 name="Writes" collect="on" />
  <MeasurementV2 name="FailedWrites" collect="on" />
  </ResourceStatsSwitches>'
 activityLogSupported='no'
Parser-Statistics
 Threads-Parsers
  Thread
   threadld='8160'
   threadName='Thread-8160'
   totalParsers='0'
   Parsers
  Thread
   threadld='1544'
   threadName='Thread-1L544'
   totalParsers='0'
   Parsers
  Thread
   threadld='11496'
   threadName='Thread-11496'
   totalParsers='7'
   Parsers
  Parser
   name='XMLNSC
   address='Oxla6fd0c0'
   type='xminsc'
   isShared='FALSE'
   creationTime='2015-10-13 21:31:35.177812'
   lastUsedTime=2015-10-13 21:31:35.177867'
   totalTimesUsed='1'
   approxMemKB='7.98'
   fields='44'
   totalFieldsUsed='44'
  Parser
   name='DFDL'
   address='Oxla180190'
   type='dfdl'
   isShared='FALSE'
   creationTime='2015-10-13 21:31:35.177921'
   lastUsedTime='2015-10-13 21:31:35.177939'
   totalTimesUsed='1'
   approxMemKB='7.98'
   fields='14'
   totalFieldsUsed='14'
   Parsing
    totalReads='1'
    maxReadKB='0.02'
    totalReadKB='0.02'
    successfulRead='l'
    fullReads='1'
    approxMemKB='7.B8'
    totalFieldsUsed='1'
    Parsing
     totalReads='1'
     maxReadKB='0.02'
     totalReadKB='0.02'
     successfulReads='1'
    Parser
     name='DFDL'
     address='0x1Bb37770'
     type='dfdl'
     isShared='FALSE'
     creationTime='2015-10-13 21:31:35.386184'
     lastUsedTime='2015-10-13 21:31:35.386188'
     totalTimesUsed='1'
     approxMemKB='7.98'
     fields='1'
     totalFieldsUsed='1'
     Parsing
      totalReads='1'
      maxReadKB='0.02'
      totalReadKB='0.02'
      successfulRead,l' 

BIP80711: Successful command completion.

bj60040_.htm |

Last updated: 2016-08-12 11:20:23