IBM Integration Bus, Version 9.0.0.8 Operating Systems: AIX, HP-Itanium, Linux, Solaris, Windows, z/OS

See information about the latest product version

Message flow aggregation

Aggregation is the generation and fan-out of related requests that are derived from a single input message, and the fan-in of the corresponding replies to produce a single aggregated reply message.

The initial request that is received by the message flow, representing a collection of related request items, is split into the appropriate number of individual requests to satisfy the subtasks of the initial request. This process is known as fan-out, and it is provided by a message flow that includes aggregation nodes.

Replies from the subtasks are combined and merged into a single reply, which is returned to the original requester (or another target application) to indicate completion of the processing. This process is known as fan-in, and it is also provided by a message flow that includes aggregation nodes.

A message aggregation is initiated by a message flow that contains the AggregateControl node followed by an AggregateRequest node. The responses are aggregated back together using a flow that contains the AggregateReply node. The aggregation nodes work correctly only for transports that use a request/reply model; for example, WebSphere® MQ Enterprise Transport.

IBM® Integration Bus provides three message flow nodes that support aggregation:

When you include these nodes in your message flows, the multiple fan-out requests are issued in sequence from one thread in the integration server process.

If you use WebSphere MQ Enterprise Transport, the responses that are received by the fan-in flow must be valid reply messages that contain the reply identifier. You must set the reply identifier to the value of the message identifier field (MsgId) in the request message's message descriptor (MQMD), then store the reply identifier in the correlation identifier field (CorrelId) of the MQMD. If the CorrelId is set to MQCI_NONE, the AggregateReply node issues an error because the reply message is not valid, and cannot be matched to a request message.

You can also use these aggregation nodes to issue requests to applications outside the broker environment. Messages can be sent asynchronously to external applications or services; the responses are retrieved from those applications, and the responses are combined to provide a single response to the original request message.

These nodes can help to improve response time because slow requests can be performed in parallel, and they do not need to follow each other sequentially. If the subtasks can be processed independently, and they do not need to be handled as part of a single unit of work, they can be processed by separate message flows.

You can design and configure a message flow that provides a similar function without using the aggregation nodes, by issuing the subtask requests to another application (for example, using the HTTPRequest node), and recording the results of each request in the local environment. After each subtask has completed, merge the results from the LocalEnvironment in a Compute node, and create the combined response message for propagating to the target application. However, all the subtasks are performed sequentially, and they do not provide the performance benefits of parallel operation that you can achieve if you use the aggregation nodes.

Examples of aggregation flows that use the aggregation nodes are provided in the following samples: The Aggregation sample demonstrates a simple four-way aggregation, and the Airline Reservations sample simulates requests that are related to an airline reservation service, and illustrates the techniques that are associated with aggregation flows.

You can view information about samples only when you use the product documentation that is integrated with the IBM Integration Toolkit or the online product documentation. You can run samples only when you use the product documentation that is integrated with the IBM Integration Toolkit.

The aggregation nodes store state for aggregations on WebSphere MQ queues. By default, the following storage queues are used: However, you can create alternative storage queues and use an aggregation configurable service to specify which queues are to be used by the node. For more information, see Configuring the storage of events for aggregation nodes.

By default, the timeout on the AggregateControl node is set to 0. If you do not specify a timeout (or if you leave it set to 0), aggregation requests that WebSphere MQ stores are never deleted unless all reply messages are returned. This situation might lead to a gradual build up of messages on the internal queues. Set the timeout to a value greater than zero to ensure that requests are removed and that queues do not fill up with redundant requests. It is good practice to set the timeout value to a large value, for example, 86400 seconds (24 hours), so that the queues clear old aggregation messages even if timeouts are not required or expected.

You can set the timeout either by setting the Timeout property on the AggregateControl node, or by using an Aggregation configurable service and specifying the timeoutSeconds property. For more information, see Setting timeout values for aggregation.

The aggregation nodes use WebSphere MQ message expiry to manage timeout of messages. For message expiry to work, the aggregation nodes must browse the message queues. The aggregation nodes browse the queues automatically to ensure that expired messages are processed.

z/OS platformOn z/OS®, you can configure WebSphere MQ to run a scavenger process that browses the queues instead of the aggregation nodes. To enable the scavenger, set the broker's queue manager property EXPRYINT to a value of 5 seconds. If you do not set EXPRYINT, or set it to a value higher than 10 seconds, the aggregation nodes revert to browsing the aggregation queues automatically.


ac00660_.htm | Last updated Friday, 21 July 2017