Feedback


 

5746-SM3 IBM DFSORT/VSE


EMEA                                                                          


                            PROGRAM PRODUCTS                            May 98


Table of Contents

5746-SM3 IBM DFSORT/VSE

  • Abstract
  • Product Positioning
  • Highlights
  • Description
  • Technical Description
  • Operating Environment
  • Hardware Requirements
  • Software Requirements
  • Planning Information
  • Publications
  • Security, Auditability, and Control

  • 5746-SM3 IBM DFSORT/VSE


    Abstract

    DFSORT/VSE is IBM's high-performance sort, merge, and copy product for VSE/ESA operating systems. It reduces computing costs through efficient use of machine resources, providing a fast and easy way to manage and organize your data.

    This release of DFSORT/VSE offers enhanced performance and improved productivity. These improvements can help reduce elapsed time, CPU time, and SIO counts.

    Performance Enhancements

    The following areas have been improved:

    • Dataspace sorting
    • Getvis sorting
    • Partition sorting
    • Copying and merging
    • Input/output processing techniques
    • Work file processing
    • Use of the ECKD command set for input, output, and work files

    Productivity Improvements

    The following improvements have been made:

    • Year 2000 formats for handling two-digit year data with special indicators and two-digit binary years

    • OUTREC features that help you design your output records

    • INCLUDE and OMIT features to increase the number of selection criteria you supply

    • Online message explanations for easier diagnosis and correction of errors

    • ZDPRINT option to make your summed positive zoned decimal fields printable

    For more information, visit the DFSORT/VSE Web site:

    • http://www.ibm.com/storage/dfsortvse/

    Product Positioning

    The support of improved functionality and performance enables DFSORT/VSE Version 3 Release 4 to compete favorably with other sort products.


    Highlights

    This high-performance sort, merge, and copy product for VSE/ESA operating systems:

    • Provides data processing and reporting functions

    • Enhances your ability to manage large amounts of data

    • Helps to reduce your computing costs through efficient use of machine resources

    • Includes performance and productivity improvements that make DFSORT/VSE faster and more versatile than ever

    Description

    Dataspace Sorting

    DFSORT/VSE's enhanced dataspace sorting can provide significant CPU time, elapsed time, and SIO count improvements. By using data space as an extension of partition virtual storage, DFSORT/VSE sorts larger pieces of data at one time, requiring less physical I/O and less external work space. With fewer sorts requiring work space, more sorts can be processed entirely in virtual storage resulting in improved sort performance.

    Getvis Sorting

    DFSORT/VSE's enhanced getvis sorting can provide significant CPU time, elapsed time, and SIO count improvements. Getvis sorting is similar to dataspace sorting except that it uses the partition GETVIS area for virtual storage instead of a data space.

    Partition Sorting

    DFSORT/VSE's enhanced partition sorting can provide improvements in CPU time, elapsed time, and SIO counts when used with external work files.

    Copy and Merge Applications

    DFSORT/VSE's enhanced copying and merging can provide improvements in CPU time, elapsed time, and SIO counts.

    Improved Input/Output Processing Techniques

    DFSORT/VSE's enhanced input/output processing techniques can provide improvements in CPU time, elapsed time, and SIO counts for:

    • Non-VSAM input and output files
    • Variable-length record SAM output files

    ECKD command set use

    DFSORT/VSE will now use the ECKD command set for input, output and work files, reducing elapsed time.

    Work File Processing

    We've enhanced our work file processing by:

    • Closing all work files at the end of an application.

    • Using additional work file extents, if available, when end of extent is encountered regardless of whether STXIT is in effect.

    • Using all extents of an SD work file instead of only using the first extent.

    User Control of VSAM Buffers

    You can now control the number of buffers DFSORT/VSE can use for VSAM (or SAM ESDS accessed as VSAM) input and output file processing through the new installation option VSAMBSP, reducing CPU time, elapsed time and SIO counts.

    Additional Year 2000 Formats

    As you prepare for the Year 2000, DFSORT/VSE's new Y2S and Y2B formats give you even more flexibility in sorting or transforming your two-digit year dates.

    • Y2S interprets two-digit character year data according to the century window and allows special handling of indicators X'00' (binary zeros), X'40' (EBCDIC blanks), X'20' (ASCII blanks), and X'FF' (binary ones) in the year field.

    • Y2B interprets two-digit binary year data according to the century window.

    OUTREC Features

    OUTREC gives you a variety of ways to reformat your records for output. Now you have even more flexibility in designing your output records through using OUTREC's:

    • Lookup and change (p,m,lookup): OUTREC has a new and very useful feature called lookup and change. It tells DFSORT/VSE to look at the value of a field in your input record and match it to the values you have set up in a table. When a match is found, the associated string in the table is substituted for the original value. This makes it easy for you to substitute meaningful words or phrases for cryptic values (for example, FERN can be changed to FERNALL BROS.).

    • Hexadecimal display (p,m,hex and p,hex): You can use hexadecimal display to convert data to printable hexadecimal characters (EBCDIC).

    • Editing masks (p,m,f,edit): You can control the way your numeric fields are displayed. You can control the sign (for example, + or blank for positive numbers , - or () for negative numbers) and editing control (for example, commas every n digits, decimal point, leading $, and so on) making them much easier to read. You can also use one of the twenty-six pre-defined patterns provided with DFSORT/VSE or set up your own patterns with user-defined editing masks.

    • Column support (c:): You can specify the output column for any field or separator, making it easier to place information where you want it.

    INCLUDE and OMIT Features

    INCLUDE and OMIT are two of the most frequently used DFSORT/VSE control statements. You can now:

    • Increase the number of selection criteria you specify because DFSORT/VSE now allows a significantly larger number of INCLUDE and OMIT conditions.

    • Include or omit all records using the new ALL or NONE operands.

    Online Message Explanations (OME)

    You can now request that an explanation of a DFSORT/VSE message be displayed on the console of your display station just as you would for VSE/ESA messages. OME makes it easier for you to diagnose and correct errors quickly.

    ZDPRINT OPTION

    You can now use the ZDPRINT OPTION to make your summed positive zoned decimal numbers printable.

    Additional Enhancements

    You can now:

    • Use the new STXIT=MIN installation option and MINSTXIT run-time option to improve your performance when COBOL or PL/I programs invoke DFSORT/VSE and use E15/E35 user exit routines to process records. To assist you with this, we've changed the IBM-supplied default from STXIT=YES to STXIT=MIN.

    • Use the new DIAGINF installation option to request diagnostic messages and a dump regardless of the options in effect at run time.

    • Use the new NRECOUT installation and run-time option to tell DFSORT/VSE what to do when it does not write any records to the output file: terminate the run, continue processing and pass back a warning return code, or just continue processing.

    Technical Description

    Operating Environment

    Hardware Requirements

    • Any device supported by VSE/ESA(TM) for program residence
    • Any device SAM or VSAM uses for input or output that is supported by VSE/ESA(TM)

    Software Requirements

    Any currently supported or subsequent release or modification level of the following operating systems:

    • VSE/ESA Version 1 Release 3, or later (5750-ACD)
    • VSE/ESA Version 2 (5690-VSE)

    Note: It is recommended that you use VSE/ESA Version 1 Release 4 as the minimum level of operating system. Service was discontinued for CICS/VSE Version 2 Release 2, a major component of VSE/ESA Version 1 Release 3, in November, 1997. Service will be discontinued for VSE/ESA Version 1 Release 3 in December, 1998.

    The minimum available partition program area requirement is 32 KB. This value is unchanged from DFSORT/VSE Version 3 Release 3.

    DFSORT/VSE requires a minimum of 370 KB in the Shared Virtual Area (SVA): 16 KB is required in the 24-bit SVA, 354 KB may reside in the 31-bit SVA.

    DFSORT/VSE can use data space as an extension of primary storage. With a larger amount of primary storage available, more files can be sorted without using work space.

    Disk storage is used as intermediate storage if there is insufficient virtual storage to support an incore sort. DFSORT/VSE Version 3 Release 4 supports all disk storage devices supported by VSE/ESA for intermediate storage.

    All storage input/output components that are now supported by VSE/ESA are supported by DFSORT/VSE.

    DFSORT/VSE is installed and maintained using the Maintain System History Program (MSHP) feature of Virtual Storage Extended/Advanced Functions (VSE/AF).


    Planning Information

    Customer Responsibilities

    Not applicable.

    Performance Considerations

    DFSORT/VSE Version 3 Release 4 enhanced support for sort, copy, and merge applications can result in significant performance improvements when compared with DFSORT/VSE Version 3 Release 3 and DOS/VS-VM/System Product Sort/Merge Version 2 Release 5, hereafter called Sort/Merge V2R5.

    The tables that follow show the maximum reductions that were observed for sort, copy, and merge applications. Tables 1 and 2 compare DFSORT/VSE Version 3 Release 4 with DFSORT/VSE Version 3 Release 3. Tables 3 and 4 compare DFSORT/VSE Version 3 Release 4 with Sort/Merge V2R5.

    As an example of how to read the tables, consider the dataspace sorting row in table 1. This row shows that when using dataspace sorting with no work files and Fixed-Length Records (FLR), elapsed time was reduced by up to 23%, CPU time up to 7%, and SIO counts up to 78%.

     
        off
    +-----------------------------------------------------------------------+
    | Table 1: DFSORT/VSE V3R4 vs V3R3 (disk input/output)                  |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Activity         | Work  | Record | Elapsed |  CPU   |  SIO   | Work  |
    |                  | Files | Format |  Time   |  Time  | Counts | File  |
    |                  |       |        |         |        |        | Space |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Dataspace Sorting|  no   |  FLR   |  -23%   |   -7%  |  -78%  |  na   |
    |                  |  no   |  VLR   |  -21%   |   -5%  |  -69%  |  na   |
    |                  |  yes  |  FLR   |  -13%   |  -23%  |  -50%  |  -0%  |
    |                  |  yes  |  VLR   |  -29%   |   -5%  |  -59%  | -29%  |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Getvis Sorting   |  no   |  FLR   |  -23%   |   -6%  |  -78%  |  na   |
    |                  |  no   |  VLR   |  -22%   |   -4%  |  -69%  |  na   |
    |                  |  yes  |  FLR   |  -11%   |   -8%  |  -50%  |  -0%  |
    |                  |  yes  |  VLR   |  -30%   |   -7%  |  -59%  | -29%  |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Partition Sorting|  no   |  FLR   |   +4%   |   -0%  |  -57%  |  na   |
    |                  |  no   |  VLR   |   -6%   |   -3%  |  -67%  |  na   |
    |                  |  yes  |  FLR   |  -10%   |   -6%  |  -18%  |  -0%  |
    |                  |  yes  |  VLR   |   -7%   |   -0%  |  -31%  |  -0%  |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Copying          |  na   |  FLR   |  -23%   |  -58%  |  -95%  |  na   |
    |                  |  na   |  VLR   |  -22%   |  -54%  |  -96%  |  na   |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Merging          |  na   |  FLR   |  -20%   |  -54%  |  -95%  |  na   |
    |                  |  na   |  VLR   |  -12%   |  -57%  |  -95%  |  na   |
    +-----------------------------------------------------------------------+
     
    
    +-----------------------------------------------------------------------+
    | Table 2: DFSORT/VSE V3R4 vs V3R3 (tape input/output)                  |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Activity         | Work  | Record | Elapsed |  CPU   |  SIO   | Work  |
    |                  | Files | Format |  Time   |  Time  | Counts | File  |
    |                  |       |        |         |        |        | Space |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Copying          |  na   |  FLR   |  -32%   |  -53%  |  -96%  |  na   |
    |                  |  na   |  VLR   |  -38%   |  -43%  |  -96%  |  na   |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Merging          |  na   |  FLR   |  -39%   |  -49%  |  -93%  |  na   |
    |                  |  na   |  VLR   |  -33%   |  -42%  |  -94%  |  na   |
    +-----------------------------------------------------------------------+
     
    
    +-----------------------------------------------------------------------+
    | Table 3: DFSORT/VSE V3R4 vs Sort/Merge V2R5 (disk input/output)       |
    |------------------+-------+--------+---------+--------+--------+-------|
    | DFSORT/VSE V3R4  | Work  | Record | Elapsed |  CPU   |  SIO   | Work  |
    | Sorting Technique| Files | Format |  Time   |  Time  | Counts | File  |
    |                  |       |        |         |        |        | Space |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Dataspace Sorting|  no   |  FLR   |  -35%   |  -10%  |  -71%  |  na   |
    |                  |  no   |  VLR   |  -36%   |  -14%  |  -72%  |  na   |
    |                  |  yes  |  FLR   |  -56%   |  -38%  |  -64%  | -38%  |
    |                  |  yes  |  VLR   |  -58%   |  -31%  |  -75%  | -35%  |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Getvis Sorting   |  no   |  FLR   |  -36%   |  -10%  |  -71%  |  na   |
    |                  |  no   |  VLR   |  -32%   |  -11%  |  -72%  |  na   |
    |                  |  yes  |  FLR   |  -56%   |  -40%  |  -64%  | -38%  |
    |                  |  yes  |  VLR   |  -58%   |  -31%  |  -75%  | -35%  |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Partition Sorting|  no   |  FLR   |   -4%   |   +5%  |  -57%  |  na   |
    |                  |  no   |  VLR   |   -5%   |   -7%  |  -67%  |  na   |
    |                  |  yes  |  FLR   |   -9%   |   -8%  |  -18%  |  -0%  |
    |                  |  yes  |  VLR   |   -7%   |   -6%  |  -23%  |  -0%  |
    +-----------------------------------------------------------------------+
    | - Sort/Merge V2R5 only has partition sorting                          |
    +-----------------------------------------------------------------------+
     
    
    +-----------------------------------------------------------------------+
    | Table 4: DFSORT/VSE V3R4 vs Sort/Merge V2R5 (disk input/output)       |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Activity         | Work  | Record | Elapsed |  CPU   |  SIO   | Work  |
    |                  | Files | Format |  Time   |  Time  | Counts | File  |
    |                  |       |        |         |        |        | Space |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Copying          |  na   |  FLR   |  -39%   |  -71%  |  -97%  |  na   |
    |                  |  na   |  VLR   |  -35%   |  -75%  |  -97%  |  na   |
    |------------------+-------+--------+---------+--------+--------+-------|
    | Merging          |  na   |  FLR   |  -36%   |  -66%  |  -97%  |  na   |
    |                  |  na   |  VLR   |  -31%   |  -74%  |  -97%  |  na   |
    +-----------------------------------------------------------------------+
     
    

    Notes:

    1. na = Not Applicable

    2. Both DFSORT/VSE and Sort/Merge V2R5 provide partition sorting. When DFSORT/VSE Version 3 Release 4 partition sorting is compared to Sort/Merge V2R5 partition sorting, only a small improvement (or slight degradation in some cases) is seen because of the limited size of the partition program area.

      DFSORT/VSE also provides dataspace sorting and getvis sorting which enable a greater percentage of sort applications to be processed without using work files. DFSORT/VSE's dataspace sorting, getvis sorting, improved I/O, and improved data processing methods result in significant reductions in elapsed time, CPU time, and SIO counts when compared with Sort/Merge V2R5.

      For best performance, we recommend that you use DFSORT/VSE's dataspace sorting or getvis sorting instead of partition sorting.

    3. Significant performance improvements were also observed when comparing DFSORT/VSE Version 3 Release 4 with Sort/Merge V2R5 using tape input and output.

    Test Environments

    Performance measurement test runs for DFSORT/VSE Version 3 Release 4, DFSORT/VSE Version 3 Release 3, and Sort/Merge V2R5 were completed using the same test environments. The test environments included:

    • Size of data space area: from 5 MB to 26 MB
    • Size of GETVIS area: from 5 MB to 26 MB
    • Size of partition program area: from 64 KB to 5376 KB
    • FLR and VLR files: from 3.1 MB to 67 MB
    • Number of records: from 10000 to 220000
    • Block size: from 6400 to 56664 bytes
    • Record length for FLR: 320 bytes
    • Average record length for VLR: 320 bytes
    • Keys were in random order and were 8 bytes in length
    • For both DFSORT/VSE releases and Sort/Merge V2R5, the SVA eligible modules were loaded in the SVA
    • VSE/ESA 2.2.1 operating system was used

    All measurements were run on an IBM 9672 Model R73 with 512 MB of central storage, in a stand-alone VSE/ESA environment. All of the input, output, and work files resided on IBM 3390-2 disk storage connected to an IBM 3990-3 storage control unit. Both volatile cache of 256 MB and non-volatile disk storage Fast Write cache of 4 MB of the 3990 Model 3 disk storage controller were activated.

    The actual performance characteristics that may be experienced by any specific user or for any specific file depends on many factors, including record length, file size, DFSORT/VSE and Sort/Merge V2R5 storage options; noticeably, in a multitasking environment, elapsed time results are application-profile and workload dependent. So, the results may differ from user to user.

    IBM does not represent nor warrant that users will experience the same changes in performance characteristics observed in these examples.


    Publications

    DFSORT Version 3 Release 4 provides publications that include the following:

    • DFSORT/VSE General Information (GC26-7039)
    • DFSORT/VSE Application Programming Guide (SC26-7040)
    • DFSORT/VSE Messages, Codes and Diagnosis (SC26-7132)
    • DFSORT/VSE Installation and Tuning Guide (SC26-7041)
    • Getting Started with DFSORT/VSE (SC26-7101)
    • DFSORT/VSE Reference Summary (SX26-6008)
    • DFSORT/VSE Licensed Program Specifications (GC26-7038)
    • DFSORT/VSE Memo to Current Users (GI10-4515)
    • DFSORT/VSE Program Directory (GI10-4513)
    • DFSORT/VSE Version 4 Library (SBOF-6130)

    Also shipped with each order is the VSE Online Product Library CD-ROM (SK2T-0060), which includes displayable soft-copy of the above publications.

    Additional copies of unlicensed publications are available for a fee immediately after availability. These copies can be ordered from your IBM representative, through the system library subscription service (SLSS), or by direct order.

    All of the these publications can be ordered using one bill of forms number, SBOF-6130.

    For more information, refer to Software Announcement 297-024, dated February 11, 1997.


    Security, Auditability, and Control

    This program uses the security and auditability features of the operating system.

    The customer is responsible for evaluation, selection, and implementation of security features, administrative procedures, and appropriate controls in application systems and communication facilities.

    Trademarks

    (R), (TM), * Trademark or registered trademark of International Business Machines Corporation.

    ** Company, product, or service name may be a trademark or service mark of others.