DFSORT overview

This chapter introduces IBM z/OS DFSORT Licensed Program 5650-ZOS.

DFSORT is intended to run in problem state and in a user key ( that is, key 8 or higher).

DFSORT is a program you use to sort, merge, and copy information.

When you sort records, you arrange them in a particular sequence, choosing an order more useful to you than the original one.
When you merge records, you combine the contents of two or more previously sorted data sets into one.
When you copy records, you make an exact duplicate of each record in your data set.

Merging records first requires that the input data sets are identically sorted for the information you will use to merge them and that they are in the same order required for output. You can merge up to 100 different data sets at a time.

In addition to the three basic functions, you can perform other processing simultaneously:

You can control which records to keep in the final output data set of a DFSORT run by using INCLUDE and OMIT statements in your application. These statements work like filters, testing each record against criteria that you supply and retaining only the ones you want for the output data set. For example, you might choose to work only with records that have a value of “Kuala Lumpur” in the field reserved for office location. Or perhaps you want to leave out any record dated after 2001 if it also contains a value greater than 20 for the number of employees.

You can parse, edit, and reformat your records before or after other processing by using INREC and OUTREC statements. INREC and OUTREC statements support a wide variety of reformatting tasks including:

The use of fixed position/length fields or variable position/length fields. For fixed fields, you specify the starting position and length of the field directly. For variable fields, such as delimited fields, comma separated values (CSV), tab separated values, blank separated values, keyword separated fields, null-terminated strings (and many other types), you define rules that allow DFSORT to extract the relevant data into fixed parsed fields, and then use the parsed fields as you would use fixed fields.
Insertion of blanks, zeros, strings, current date, future date, past date, current time, sequence numbers, decimal constants, and the results of arithmetic expressions before, between, and after the input fields in the reformatted records.
Sophisticated conversion capabilities, such as find and replace, hexadecimal display, bit display, translation of EBCDIC letters from lowercase to uppercase or uppercase to lowercase, translation of characters from EBCDIC to ASCII and from ASCII to EBCDIC, translation of characters using the ALTSEQ translation table, conversion of numeric values from one format to another, left-justify or left-squeeze (remove leading blanks or all blanks and shift left), and right-justify or right-squeeze (remove trailing blanks or all blanks and shift right).
Sophisticated editing capabilities, such as control of the way numeric fields are presented with respect to length, leading or suppressed zeros, thousands separators, decimal points, leading and trailing positive and negative signs, and so on.
Twenty-seven pre-defined editing masks are available for commonly used numeric editing patterns, encompassing many of the numeric notations used throughout the world. In addition, a virtually unlimited number of numeric editing patterns are available via user-defined editing masks.
Transformation of SMF, TOD, and ETOD date and time values to more usable forms.
Conversion of input date fields of one type (CH, ZD, PD, 2-digit year, 4-digit year, Julian, Gregorian) to corresponding output date fields of another type or to a corresponding day of the week.
Various types of arithmetic operations for input date fields.
Selection of a character constant, hexadecimal constant, or input field from a lookup table for output, based on a character, hexadecimal, or bit string as input (that is, lookup and change).

You can create the reformatted INREC or OUTREC records in one of the following ways:

By building the entire record one item at a time.
By only overlaying specific columns.
By performing find and replace operations.
By using sophisticated conditional logic or group operations to choose how different records are reformatted.

You can sum numeric information from many records into one record with the SUM statement. For example, if you want to know the total amount of a yearly payroll, you can add the values for a field containing salaries from the records of all your employees.

You can create one or more output data sets for a sort, copy, or merge application from a single pass over one or more input data sets by using OUTFIL control statements. You can use multiple OUTFIL statements, with each statement specifying the OUTFIL processing to be performed for one or more output data sets. OUTFIL processing begins after all other processing ends (that is, after processing for exits, options, and other control statements). OUTFIL statements support a wide variety of output data set tasks, including:

Creation of multiple output data sets containing unedited or edited records from a single pass over one or more input data sets.
Creation of multiple output data sets containing different ranges or subsets of records from a single pass over one or more input data sets. In addition, records that are not selected for any subset can be saved in a separate output data set.
Conversion of variable-length record data sets to fixed-length record data sets.
Conversion of fixed-length record data sets to variable-length record data sets.
A wide variety of parsing, editing and reformatting tasks including:
- The use of fixed position/length fields or variable position/length fields. For fixed fields, you specify the starting position and length of the field directly. For variable fields, such as delimited fields, comma separated values (CSV), tab separated values, blank separated values, keyword separated fields, null-terminated strings (and many other types), you define rules that allow DFSORT to extract the relevant data into fixed parsed fields, and then use the parsed fields as you would use fixed fields.
- Insertion of blanks, zeros, strings, current date, future date, past date, current time, sequence numbers, decimal constants, and the results of arithmetic expressions before, between, and after the input fields in the reformatted records.
- Sophisticated conversion capabilities, such as find and replace, hexadecimal display, bit display, translation of EBCDIC letters from lowercase to uppercase or uppercase to lowercase, translation of characters from EBCDIC to ASCII or from ASCII to EBCDIC, translation of characters using the ALTSEQ translation table, conversion of numeric values from one format to another, left-justify or left-squeeze (remove leading blanks or all blanks and shift left), and right-justify or right-squeeze (remove trailing blanks or all blanks and shift right).
- Sophisticated editing capabilities, such as control of the way numeric fields are presented with respect to length, leading or suppressed zeros, thousands separators, decimal points, leading and trailing positive and negative signs, and so on.
  Twenty-seven pre-defined editing masks are available for commonly used numeric editing patterns, encompassing many of the numeric notations used throughout the world. In addition, a virtually unlimited number of numeric editing patterns are available via user-defined editing masks.
- Transformation of SMF, TOD, and ETOD date and time values to more usable forms.
- Conversion of input date fields of one type (CH, ZD, PD, 2-digit year, 4-digit year, Julian, Gregorian) to corresponding output date fields of another type or to a corresponding day of the week.
- Various types of arithmetic operations for input date fields.
- Selection of a character constant, hexadecimal constant, or input field from a lookup table for output, based on a character, hexadecimal, or bit string as input (that is, lookup and change).
Creation of the reformatted records in one of the following ways:
- By building the entire record one item at a time.
- By only overlaying specific columns.
- By performing find and replace operations.
- By using sophisticated conditional logic or group operations to choose how different records are reformatted.
Highly detailed three-level (report, page, and section) reports containing a variety of report elements you can specify (for example, current date, current time, edited or converted page numbers, character strings, and blank lines) or derive from the input records (for example, character fields; unedited, edited, or converted numeric input fields; edited or converted record counts; and edited or converted totals, maximums, minimums, and averages for numeric input fields).
Creation of multiple output records from each input record, with or without intervening blank output records.
Repetition and sampling of data records.
Splitting of data records in rotation among a set of output data sets.

You can perform various "join" operations on two files by one or more keys. A JOINKEYS application allows you to create joined records in a variety of ways including inner join, full outer join, left outer join, right outer join and unpaired combinations. The two input files can be of different types (fixed, variable, VSAM, and so on) and have keys in different locations. The records from the two input files can be processed in a variety of ways before and after they are joined.

You can control DFSORT functions with other control statements by specifying alternate collating sequences, invoking user exit routines, overriding installation defaults, and so on.

You can direct DFSORT to pass control during run time to routines you design and write yourself. For example, you can write user exit routines to summarize, insert, delete, shorten, or otherwise alter records during processing. However, keep in mind that the extensive editing capabilities provided by the INCLUDE, OMIT, INREC, OUTREC, SUM, and OUTFIL statements can eliminate the need to write user exit routines. You can write your own routines to correct I/O errors that DFSORT does not handle, or to perform any necessary abnormal end-of-task operation before DFSORT terminates.

You can write an EFS (Extended Function Support) program to intercept DFSORT control statements and PARM options for modification prior to use by DFSORT or to provide alternate sequence support for user-defined data.

You can define and use a symbol for any field, constant, or output column that is recognized in a DFSORT control statement or ICETOOL operator. This makes it easy to create and reuse collections of symbols (that is, mappings) representing information associated with various record layouts. You can use system symbols (for example, &JOBNAME.) in your symbol constants. You can use SET and PROC symbols in your symbol constants. See Using symbols for fields and constants.