Planning for IBM MQ Managed File Transfer

Use this topic as guidance on how you need to set up your system to run IBM® MQ Managed File Transfer .

Common configurations

There are three common IBM MQ Managed File Transfer (MFT) configurations:
  1. A single queue manager with one or more agents using local connections. This might be used to put the contents of a data set into IBM MQ queues.
  2. A single queue manager with an MFT client on a distributed machine using client bindings.
  3. Two queue managers connected by channels, and one or more agents on each machine. These agents can be client or local bindings.
MFT can use multiple queue managers:
  • One or more queue managers to transfer the data.
  • A commands queue manager that issues requests. For example, a request to start a transfer is sent to this queue manager, and the associated commands are routed to the MFT agents.
  • A coordination queue manager that manages the work.
Notes:
  1. You can use the same queue manager for transferring data, commands and coordination.
  2. This setup, although the simplest, might not be the most efficient because all the workload is on one queue manager.

If you have an existing IBM MQ Managed File Transfer configuration, your command and coordination queue manager might already exist.

If you do not have an existing IBM MQ Managed File Transfer configuration, you can use one queue manager for transferring data, commands, and coordination. Note that even if you do this, it is possible to set up multiple configurations on the same machine.

If you are using multiple queue managers you need to set up channels between the queue managers. You can either do this by using clustering or by using point-to-point connections. IBM MQ Managed File Transfer status and activity can be logged, and can be stored in either a Db2® or Oracle database.

IBM MQ Managed File Transfer is written in Java, with some shell scripts and JCL to configure and operate the program.

Important: You must be familiar with UNIX System Services (USS) in order to configure IBM MQ Managed File Transfer. For example:
  • The file directory structure, with names such as /u/userID/myfile.txt
  • USS commands, for example:
    • cd ( change directory)
    • ls (list)
    • chmod ( change the file permissions)
    • chown (change file ownership or groups which can access the file or directory)
You require the following products in USS to be able to configure and run MFT:
  1. Java, for example, in directory /java/java71_bit64_GA/J7.1_64/
  2. IBM MQ 8.0 , for example, in directory /mqm/V8R0M0
  3. If you want to use Db2 for status and history, you need to install Db2 JDBC libraries, for example, in directory /db2/db2v10/jdbc/libs.

Product registration

At startup IBM MQ Managed File Transfer checks the registration in sys1.parmlib concatenation. The following code is an example of how you register MFT: 

PRODUCT OWNER('IBM CORP')
NAME('WS MQ FILE TRANS')
ID(5655-MFT)
VERSION(*) RELEASE(*) MOD(*)
FEATURENAME('WS MQ FILE TRANS')
STATE(ENABLED)

Disk space

You will need 100 MB of DASD for PDSEs and a minimum of 50 MB in USS. If you used trace to diagnose problems, you need additional disk space in USS, for example 50 MB.

Security

You need to identify which user IDs are going to be used for MFT configuration and for MFT operation.

You need to identify the files or queues you transfer, and which user IDs are going to be submitting transfer requests to MFT.

When you customize the agents and logger, you specify the group of users that is allowed to run MFT services, or do MFT administration.

You should set up this group before you start customizing MFT. As MFT uses IBM MQ queues, if you have security enabled in the queue manager, MFT requires access to the following resources
Table 1.
Name Access required
QUEUE.SYSTEM.FTE.EVENT.agent_name Update
QUEUE.SYSTEM.FTE.COMMAND.agent_name Update
CONTEXT.SYSTEM.FTE.COMMAND.agent_name Update
QUEUE.SYSTEM.FTE.STATE.agent_name Update
QUEUE.SYSTEM.FTE.DATA.agent_name Update
QUEUE.SYSTEM.FTE.REPLY.agent_name Update
QUEUE.SYSTEM.FTE.AUTHAGT1.agent_name Update
QUEUE.SYSTEM.FTE.AUTHTRN1.agent_name Update
QUEUE.SYSTEM.FTE.AUTHOPS1.agent_name Update
QUEUE.SYSTEM.FTE.AUTHSCH1.agent_name Update
QUEUE.SYSTEM.FTE.AUTHMON1.agent_name Update
QUEUE.SYSTEM.FTE.AUTHADM1.agent_name Update
Table 2.
Name Access required
SYSTEM.FTE.AUTHAGT1.agent_name Update
SYSTEM.FTE.AUTHTRN1.agent_name Update
SYSTEM.FTE.AUTHOPS1.agent_name Update
SYSTEM.FTE.AUTHSCH1.agent_name Update
SYSTEM.FTE.AUTHMON1.agent_name Update

You can use user sandboxing to determine which parts of the file system the user who requests the transfer can access.

To enable user sandboxing, add the userSandboxes=true statement to the agent.properties file for the agent that you want to restrict, and add appropriate values to the MQ_DATA_PATH/mqft/config/coordination_qmgr_name/agents/agent_name/UserSandboxes.xml file.

See Working with user sandboxes for further information.

This user ID is configured in UserSandboxes.xml files.

This XML file has information like user ID, or user ID* and a list of resource that can be used (included), or cannot be used (excluded). You need to define specific user IDs that can access which resources: for example:
Table 3.
User ID Access Include or Exclude Resource
Admin* Read Include /home/user/**
Admin* Read Exclude /home/user/private/**
Sysprog Read Include /home/user/**
Admin* Read Include Application.reply.queue
Notes:
  1. If type=queue is specified, the resource is either a queue name, or queue@qmgr.
  2. If the resource begins with //, the resource is a dataset; otherwise the resource is a file in USS.
  3. The user ID is the user ID from the MQMD structure, so this might not reflect the user ID that actually puts the message.
  4. For requests on the local queue manager you can use MQADMIN CONTEXT.* to limit which users can set this value.
  5. For requests coming in over a remote queue manager, you have to assume that the distributed queue managers have security enabled to prevent unauthorized setting of the user ID in the MQMD structure.
  6. A user ID of SYSPROG1 on a Linux® machine, is the same user ID SYSPROG1 for the security checking on z/OS®.

How many agents do I need?

The agents do the work in transferring data, and when you make a request to transfer data you specify the name of an agent.

By default an agent can process 25 send and 25 receive requests concurrently. You can configure these processes. See IBM MQ Managed File Transfer configuration options on z/OS for more information.

If the agent is busy then work is queued. The time taken to process a request depends on multiple factors, for example, the amount of data to be sent, the network bandwidth, and the delay on the network.

You might want to have multiple agents to process work in parallel.

You can also control which resources an agent can access, so you might want some agents to work with a limited subset of data.

If you want to process requests with different priority you can use multiple agents and use workload manager to set the priority of the jobs.

Running the agents

Typically the agents are long running processes. The processes can be submitted as jobs that run in batch, or as started tasks.