Reference messages

Use this information to learn more about reference messages.

Note: Not supported in IBM® MQ for z/OS®.

This method allows a large object to be transferred from one node to another without storing the object on IBM MQ queues at either the source or the destination nodes. This is of particular benefit when the data exists in another form, for example, for mail applications.

To do this, you specify a message exit at both ends of a channel. For information about how to do this, see Channel message exit programs.

IBM MQ defines the format of a reference message header (MQRMH). See MQRMH for a description of this. This is recognized with a defined format name and might be followed by actual data.

To initiate transfer of a large object, an application can put a message consisting of a reference message header with no data following it. As this message leaves the node, the message exit retrieves the object in an appropriate way and appends it to the reference message. It then returns the message (now larger than before) to the sending Message Channel Agent for transmission to the receiving MCA.

Another message exit is configured at the receiving MCA. When this message exit receives one of these messages, it creates the object using the object data that was appended and passes on the reference message without it. The reference message can now be received by an application and this application knows that the object (or at least the portion of it represented by this reference message) has been created at this node.

The maximum amount of object data that a sending message exit can append to the reference message is limited by the negotiated maximum message length for the channel. The exit can return only a single message to the MCA for each message that it is passed, so the putting application can put several messages to cause one object to be transferred. Each message must identify the logical length and offset of the object that is to be appended to it. However, in cases where it is not possible to know the total size of the object or the maximum size allowed by the channel, design the sending message exit so that the putting application just puts a single message, and the exit itself puts the next message on the transmission queue when it has appended as much data as it can to the message it has been passed.

Before using this method of dealing with large messages, consider the following points:
  • The MCA and the message exit run under an IBM MQ user ID. The message exit (and therefore, the user ID) needs to access the object to either retrieve it at the sending end or create it at the receiving end; this might only be feasible in cases where the object is universally accessible. This raises a security issue.
  • If the reference message with bulk data appended to it must travel through several queue managers before reaching its destination, the bulk data is present on IBM MQ queues at the intervening nodes. However, no special support or exits need to be provided in these cases.
  • Designing your message exit is made difficult if rerouting or dead-letter queuing is allowed. In these cases, the portions of the object might arrive out of order.
  • When a reference message arrives at its destination, the receiving message exit creates the object. However, this is not synchronized with the MCA's unit of work, so if the batch is backed out, another reference message containing this same portion of the object will arrive in a later batch, and the message exit might attempt to re-create the same portion of the object. If the object is, for example, a series of database updates, this might be unacceptable. If so, the message exit must keep a log of which updates have been applied; this might require the use of an IBM MQ queue.
  • Depending on the characteristics of the object type, the message exits and applications might need to cooperate in maintaining use counts, so that the object can be deleted when it is no longer needed. An instance identifier might also be required; a field is provided for this in the reference message header (see MQRMH ).
  • If a reference message is put as a distribution list, the object must be retrievable for each resulting distribution list or individual destination at that node. You might need to maintain use counts. Also consider the possibility that a node might be the final node for some of the destinations in the list, but an intermediate node for others.
  • Bulk data is not typically converted. This is because conversion takes place before the message exit is invoked. For this reason, conversion must not be requested on the originating sender channel. If the reference message passes through an intermediate node, the bulk data is converted when sent from the intermediate node, if requested.
  • Reference messages cannot be segmented.

Using the MQRMH and MQMD structures

See MQRMH and MQMD for a description of the fields in the reference message header and the message descriptor.

In the MQMD structure, set the Format field to MQFMT_REF_MSG_HEADER. The MQHREF format, when requested on MQGET, is converted automatically by IBM MQ along with any bulk data that follows.

Here is an example of the use of the DataLogicalOffset and DataLogicalLength fields of the MQRMH:

A putting application might put a reference message with:
  • No physical data
  • DataLogicalLength = 0 (this message represents the entire object)
  • DataLogicalOffset = 0.
Assuming that the object is 70 000 bytes long, the sending message exit sends the first 40 000 bytes along the channel in a reference message containing:
  • 40 000 bytes of physical data following the MQRMH
  • DataLogicalLength = 40000
  • DataLogicalOffset = 0 (from the start of the object).
It then places another message on the transmission queue containing:
  • No physical data
  • DataLogicalLength = 0 (to the end of the object). You could specify a value of 30 000 here.
  • DataLogicalOffset = 40000 (starting from this point).
When this message exit is seen by the sending message exit, the remaining 30,000 bytes of data are appended, and the fields are set to:
  • 30,000 bytes of physical data following the MQRMH
  • DataLogicalLength = 30000
  • DataLogicalOffset = 40000 (starting from this point).

The MQRMHF_LAST flag is also set.

For a description of the sample programs provided for the use of reference messages, see Sample procedural programs (platforms except z/OS ).