5655-D45 IBM Batchpipes for OS/390 Version 2 Release 1
IBM U.S. Sales Manual
Revised: April 18, 2000.
|Table of contents||Document options|
|IBM U.S. Product Life Cycle Dates|
|Program Number||VRM||Announced||Available||Marketing Withdrawn||Service Discontinued||Replaced By|
Back to top
Batchpipes(R) for OS/390(TM) is a separately ordered software product that can dramatically shorten business-critical batch processing. The BatchPipePlex uses the coupling facility to pipe data between jobs running on different systems in a Parallel Sysplex(TM) environment. The BatchPipeWorks component provides over 100 functions that can be used to operate on data as it passes through a pipe.
The BatchPipes(R) for OS/390 solution addresses the shrinking of the daily batch window due to the demand for longer on-line availability.
BatchPipes(R) uses advanced data-in-memory techniques to pass data through processor storage buffers(pipes) directly to other jobs in the cycle. Elapsed time can be significantly reduced by passing data in memory between the jobs and through the parallelization of job execution.
Communication companies have implemented the BatchPipes(R) product to
reduce billing cycles. Retail companies have achieved elapsed time
benefits in their inventory cycles. Credit bureaus, credit card
companies , financial institutions and utility companies have improved
the run times of business critical batch workloads by implementing
Back to top
BatchPipes(R) for OS/390 is one of several software and hardware functions that IBM offers to improve the elapsed time of batch jobs that run on OS/390 and MVS/ESA. Others include:
- VIO to expanded storage
- Sequential Data Striping
Hiperbatch targets a different type of batch application from the BatchPipes product. With Hiperbatch, the target application has many readers simultaneously accessing data from an already existing dataset. The shared data all resides in processor storage and on DASD; the data persists after being read. In contrast, for BatchPipes, the target application has one writer passing data to one reader. BatchPipes holds very little of the data in processor storage at one time, and the output from the writer job does not reside on tape or DASD.
With Hiperbatch, readers run simultaneously with each other, but not with the writer; with BatchPipes(R) for OS/390, writer and reader jobs run simultaneously.
- Sequential Data Striping:
Both Sequential Data Striping and BatchPipes(R) for OS/390 can improve the processing of batch jobs that spend much of their elapsed time sequentially reading or writing large DASD files.
Sequential Data Striping can substantially improve I/O access times to DASD. For sequential data sets, BatchPipes(R) for OS/390 can eliminate I/O and enable concurrent processing.
Back to top
- Shorten business cycles that rely on batch applications
- Alleviate nightly batch window overruns
- Improve resource utilization in a Parallel Sysplex environment
- Reduce I/S expense by reducing tape and DASD usage
- Decrease batch job elapsed time by using data-in-memory techniques
Back to top
The BatchPipes(R) for OS/390 solution can reduce the elapsed time requirements of batch jobstreams.
Batchpipes(R) for OS/390 uses a data-in-memory service to allow replacement of intermediate basic sequential access method (BSAM) and queued sequential access method (QSAM) data sets in batch job streams with data-in-memory buffers or "pipe" between jobs. Significant elapsed time reductions are possible for sets of jobs that execute serially and can use BatchchPipes(R) for OS/390 to execute in parallel on the same system, or in a parallel sysplex environment. By using the coupling facility BatchPipes(R) for OS/390 will enable parallelism across multiple systems. This provides more options to distribute the batch workload and improve resource utilization.
BatchPipePlex provides flexible scheduling and increased resource utilization by piping data across several systems to relieve the constraints on a single system.
BatchPipeWorks functions can result in reduced processor cycles, virtual storage constraints and elapsed time.
This product does not have date dependencies and is Year 2000 Ready.
Back to top
|Operating Environment||Hardware Requirements||Software Requirements|
BatchPipes(R) for OS/390 is a subsystem that provides a way for one job within a batch jobstream to pass data to another job without going to DASD or tape. Through simple JCL changes, customers can connect jobs together by specifying a dataset name and pointing to the BatchPipes subsystem. Data is passed through a data-in-memory buffer, or "pipe", from a writer job to a reader job. Secured access to the subsystem and to the data can be maintained through standard security (SAF) interface calls.
Jobs and/or steps within a job that once ran serially can run in parallel because data records are available to the next job immediately after being written. That is, the whole sequential file does not have to be written and then closed before the next job can access the data. A reader job can read a record or a block from the pipe as soon as a writer job writes one. BatchPipes(R) for OS/390 keeps only a small number of records or blocks in processor memory at a time. When a pipe is full, the writer waits; when a reader empties the pipe, the reader waits.
Several jobs or jobsteps that today require serial processing can be logically connected together through BatchPipes(R) for OS/390 to run in parallel. Data can "flow" from one job to the next job in the set of jobs without external I/O. Significant elapsed time improvements for a complex set of jobs can be achieved by running several jobs in parallel on a tightly coupled multi-processor.
Many batch processes require copies of data on tape or DASD for audit records or other business purposes, or for later use by other applications. Very often, multiple applications need to process the same file. With the BatchPipes(R) for OS/390 BatchPipeWorks(TM) component, a set of functions supplied with BatchPipes(R) for OS/390, these actions and many other data customization operations can be accomplished without change to existing applications.
There are several different ways to invoke the functions of BatchPipeWorks within a batch process. One way to invoke the functions provided is through the JCL specifications for a BatchPipes(R) for OS/390 job. These functions can be either executed as the data is written to a pipe or as it is read from a pipe. An alternative method to invoke BatchPipeWorks functions is to execute them as separate jobs. Invoking them as separate jobs increases the opportunity for parallel execution within the BatchPipes(R) for OS/390 environment. Finally, BatchPipeWorks functions can be used to filter or pre-process data as it is being read from a physical dataset by an application. This method can also be used to post-process data as it is being written to a physical dataset by an application.
The BatchPipes(R) for OS/390 BatchPipePlex component provides the capability to pipe data between system images as configured in a Parallel Sysplex environment. BatchPipePlex pipes are called cross-system pipes and the subsystem runs in cross-system mode. BatchPipes data-in-memory support exists as it has in the previous releases. BatchPipePlex provides operational and monitoring capability, and abend detection and error propagation. There are variations in commands, command responses, monitor displays, messages, and SMF records to handle cross-system functions. Since there are multiple subsystems involved in piping for cross-system pipes, a subsystem failure will be handled through BatchPipes error detection and abend propagation. BatchPipePlex supports operator initiated Coupling Facility (CF) structure rebuild.
In order to activate cross-system piping, there are prerequisite software and hardware considerations. A BatchPipePlex can be created by satisfying the following conditions:
- A sysplex running OS/390 or MVS/ESA SP Version 5 in one or more system images running in logical partitions or on dedicated processors must be configured. The system images must be initialized in multi- system mode rather than in local mode or a monoplex. Sysplex capability requires a sysplex timer.
- Parallel Sysplex capability requires at least one CF connected to all systems which are included in the BatchPipePlex.
- A CF structure must be defined for the BatchPipePlex. The structure name must be of the form SYSASFPssname. Where ssname is the 1-4 character name for the BatchPipes subsystem to be used for all BatchPipes subsystems in the BatchPipePlex. Each system image in the BatchPipePlex would have a started-task procedure to start the ssnamed BatchPipePlex subsystem. Note BatchPipes still requires all pipe subsystems on the same system image to have unique names.
Whenever a BatchPipes(R) for OS/390 subsystem is started it will determine whether it is to be part of a BatchPipePlex or whether it is to run as a local mode subsystem. If the above requirements are met, the BatchPipes subsystem will run in cross-system mode and will be part of a BatchPipePlex. For example, if the name of the CF structure for BatchPipePlex is SYSASFPBPX1, then the set of BatchPipes subsystems named BPX1 started on the system images will represent a BatchPipePlex and any data piped between jobs specifying that subsystem will have at least the BatchPipes control information in the CF. If two jobs are on the same system, piped data will remain on the processor where the jobs are initiated and control information will be in the CF. If a third job connects to the same data-in-memory pipe but from a different system image, the data-in-memory pipe data will be migrated to the CF for the duration of that pipe's use. In this case, as with all 'true' cross-system pipes, data and control information will be in the CF. If the BatchPipes(R) for OS/390 BatchPipes subsystem cannot meet the requirements for a BatchPipePlex, the subsystem will run in local mode with local mode pipes, i.e., control data will be in the local processor memory and the pipes will be data-in-memory pipes.
If a BatchPipePlex is a partial covering BatchPipePlex, i.e., the set of cross-system BatchPipePlex subsystems having the same name but not having a subsystem initiated on every system image in the Parallel Sysplex cluster, then the user will still have to take action to assure jobs are converted and initiated on the systems where the BatchPipePlex subsystems are defined. BatchPipes(R) for OS/390 BatchPlex can be used to manage cross-system and local mode pipes without JCL changes for piping within jobs.
Multiple BatchPipePlexes can be defined within a Parallel Sysplex and they can be covering BatchPipePlexes, partial covering BatchPipePlexes, or some combination thereof. An example of having two multiple covering BatchPipePlexes would be to associate each BatchPipePlex with a different CF for load balancing. An example of having a partial covering BatchPipePlex would be to have cross-system pipes occurring between systems where resources are not shared between the system images in the Parallel Sysplex cluster.
BatchPipes(R) for OS/390 supports all hardware supported by OS/390 and MVS/ESA.
A Parallel Sysplex configuration with a Coupling Facility Structure defined for BatchPipes is required for piping across images.
- Central processor complex (CPCs) that can attach to the coupling facility
- At least one coupling facility and the appropriate channels and channel cards
- At least one Sysplex Timer(R)
- Connection to shared DASD
BatchPipes(R) for OS/390 supports the following versions/releases of OS/390 and MVS/ESA:
- OS/390 Version 2 (5647-A01)
- OS/390 Version 1 (5645-001)
- MVS/ESA SP Version 5 Release 1 and subsequent releases (5655-068 or 5655-069)
Using BatchPipes(R) across images in a parallel sysplex requires OS/390 or MVS/ESA V5.
BatchPipes(R) for OS/390 Version 2.1 is compatible with the BatchPipes
component of SmartBatch for OS/390 Version 1.2
Back to top
No special planning is required for BatchPipes(R) for OS/390. However all customers should refer to the BatchPipes(R) for OS/390 Users Guide and Reference(SA22-7458).
Significant elapsed time reductions are possible for sets of jobs that
execute serially by using BatchPipes(R) for OS/390 to execute these jobs
in parallel on the same OS/390 system or Parallel Sysplex environment.
Consideration must be given to the combined requirements for; initiators,
tape drives, DASD, processor storage and processor usage.
Back to top
- BatchPipes(R) for OS/390 V2 R1 BatchPipeWorks Reference (SA22-7456)
- BatchPipes(R) for OS/390 V2 R1 BatchPipeWorks User Guide (SA22-7457)
- BatchPipes(R) for OS/390 V2 R1 Users Guide and Reference (SA22-7458)
- BatchPipes(R) for OS/390 V2 R1 Introduction (GA22-7459)
BatchPipes(R) for OS/390 publications are offered in displayable
softcopy form. The displayable manuals are optional machine-readable
material. The files are shipped on CD-ROM. To order use form number
Back to top
|Security, Auditability, and Control|
BatchPipes(R) for OS/390 relies on the security and auditability features of the operating system software.
The customer is responsible for evaluation, selection, and implementation of security features, administrative procedures, and appropriate controls in application systems and communication facilities.
(R), (TM), * Trademark or registered trademark of International Business Machines Corporation.
** Company, product, or service name may be a trademark or service
mark of others.
© IBM Corporation 2003.