Clustering: Application isolation using multiple cluster transmission queues
You can isolate the message flows between queue managers in a cluster. You can place messages being transported by different cluster-sender channels onto different cluster transmission queues. You can use the approach in a single cluster or with overlapping clusters. The topic provides examples and some best practices to guide you in choosing an approach to use.
When you deploy an application, you have a choice over which IBM® WebSphere® MQ resources it shares with other
applications and which resources it does not share. There are a number
of types of resources that can be shared, the main ones being the
server itself, the queue manager, channels, and queues. You can choose
to configure applications with fewer shared resources; allocating
separate queues, channels, queue managers, or even servers to individual
applications. If you do so, the overall system configuration becomes
bigger and more complex. Using IBM WebSphere MQ clusters
reduces the complexity of managing more servers, queue managers, queues,
and channels, but it introduces another shared resource, the cluster
transmission queue, SYSTEM.CLUSTER.TRANSMIT.QUEUE
.
Figure 1 is a slice out of a large IBM WebSphere MQ deployment that illustrates the
significance of sharing SYSTEM.CLUSTER.TRANSMIT.QUEUE
.
In the diagram, the application, Client App
, is connected
to the queue manager QM2
in cluster CL1
.
A message from Client App
is processed by the application, Server
App
. The message is retrieved by Server App
from
the cluster queue Q1
on the queue manager QM3
in CLUSTER2
.
Because the client and server applications are not in the same cluster,
the message is transferred by the gateway queue manager QM1
.
The normal way to configure a cluster gateway is to make the gateway
queue manager a member of all the clusters. On the gateway queue manager
are defined clustered alias queues for cluster queues in all the clusters.
The clustered queue aliases are available in all the clusters. Messages
put to the cluster queue aliases are routed via the gateway queue
manager to their correct destination. The gateway queue manager puts
messages sent to the clustered alias queues onto the common SYSTEM.CLUSTER.TRANSMIT.QUEUE
on QM1
.
The hub and spoke architecture requires all messages between clusters
to pass through the gateway queue manager. The result is that all
messages flow through the single cluster transmission queue on QM1
, SYSTEM.CLUSTER.TRANSMIT.QUEUE
.
From a performance perspective, a single queue is not a problem. A common transmission queue generally does not represent a performance bottleneck. Message throughput on the gateway is largely determined by the performance of the channels that connect to it. Throughput is not generally affected by the number of queues, or the number of messages on the queues that use the channels.
- You cannot isolate the flow of messages to one destination from
the flow of messages to another destination. You cannot separate the
storage of messages before they are forwarded, even if the destinations
are in different clusters on different queue managers.
If one cluster destination becomes unavailable, messages for that destination build-up in the single transmission queue, and eventually the messages fill it up. Once the transmission queue is full, it stops messages from being placed onto the transmission queue for any cluster destination.
- It is not easy to monitor the transfer of messages to different cluster destinations. All the messages are on the single transmission queue. Displaying the depth of the transmission queue gives you little indication whether messages are being transferred to all destinations.
In Figure 1, clients of Server
App
open the queue Q1A
. Messages are put
to SYSTEM.CLUSTER.TRANSMIT.QUEUE
on QM2
,
transferred to SYSTEM.CLUSTER.TRANSMIT.QUEUE
on QM1
,
and then transferred to Q1
on QM3
,
where they are received by the Server App
application.
The message from Client App
passes through system
cluster transmission queues on QM2
and QM1
.
In Figure 1, the objective is to isolate
the message flow on the gateway queue manager from the client application,
so that its messages are not stored on SYSTEM.CLUSTER.TRANSMIT.QUEUE
.
You can isolate flows on any of the other clustered queue managers.
You can also isolate flows in the other direction, back to the client.
To keep the descriptions of the solutions brief, the descriptions
consider only a single flow from the client application.
Solutions to isolating cluster message traffic on a cluster gateway queue manager
One way to solve the problem is to use queue manager aliases, or remote queue definitions, to bridge between clusters. Create a clustered remote queue definition, a transmission queue, and a channel, to separate each message flow on the gateway queue manager; see Adding a remote queue definition to isolate messages sent from a gateway queue manager.
From Version 7.5 onwards, cluster queue managers are not limited to a single cluster transmission queue. You have two choices:
- Define additional cluster transmission queues manually, and define which cluster-sender channels transfer messages from each transmission queue; see Adding a cluster transmit queue to isolate cluster message traffic sent from a gateway queue manager.
- Allow the queue manager to create and manage additional cluster transmission queues automatically. It defines a different cluster transmission queue for each cluster-sender channel; see Changing the default to separate cluster transmission queues to isolate message traffic
You can combine manually defined cluster transmission queues
for some cluster-sender channels, with the queue manager managing
the rest. The combination of transmission queues is the approach taken
in Adding a cluster transmit queue to isolate cluster message traffic sent from a gateway queue manager. In that solution, most messages
between clusters use the common SYSTEM.CLUSTER.TRANSMIT.QUEUE
.
One application is critical, and all its message flows are isolated
from other flows by using one manually defined cluster transmission
queue.
The configuration in Adding a cluster transmit queue to isolate cluster message traffic sent from a gateway queue manageris limited. It does not separate the message traffic going to a cluster queue on the same queue manager in the same cluster as another cluster queue. You can separate the message traffic to individual queues by using the remote queue definitions that are part of distributed queueing. With clusters, using multiple cluster transmission queues, you can separate message traffic that goes to different cluster-sender channels. Multiple cluster queues in the same cluster, on the same queue manager, share a cluster-sender channel. Messages for those queues are stored on the same transmission queue, before being forwarded from the gateway queue manager. In the configuration in Adding a cluster and a cluster transmit queue to isolate cluster message traffic sent from a gateway queue manager, the limitation is side-stepped by adding another cluster and making the queue manager and cluster queue a member of the new cluster. The new queue manager might be the only queue manager in the cluster. You could add more queue managers to the cluster, and use the same cluster to isolate cluster queues on those queue managers as well.