IBM Parallel Environment for AIX improves usability and performanceIBM United States Software Announcement 209-345
October 20, 2009
|Table of contents||Document options|
|At a glance|
IBM® Parallel Environment for AIX®, V5.2 provides the following enhancements to scalability and performance:
- Support for additional parallel programming APIs
- New environment variables designed to improve performance
- Enhanced flexibility in choosing the number of Message Passing Interface - Input/Output (MPI-IO) agents and their placements, which can improve MPI-IO performance
- Scaling improvements for collective communication
- Flexibility in choice for resource managers, including allocation of nodes for user space jobs to run without any resource manager
- An Eclipse plug-in that allows users to run IBM High Performance Computing Toolkit (HPC Toolkit) from within Eclipse to analyze their application performance
For ordering, contact your IBM representative, an IBM Business Partner, or IBM Americas Call Centers at 800-IBM-CALL (Reference: YE001).
Back to top
Parallel Environment for AIX is a high-function development and execution environment for parallel applications running on one or more IBM Power® Systems servers.
Parallel Environment for AIX can offer High Performance Computing (HPC) users whose parallel applications use the industry-standard Message Passing Interface (MPI) a tool for creating and using parallel applications with a familiar look and feel. IBM's MPI implementation exploits an enhanced low-level application programming interface (LAPI) designed to increase parallel application performance on clusters made up of Power Systems servers running the AIX operating system.
Version 5.2 offers:
- The ability to use the resource manager of your choice for submitting and managing batch or interactive parallel jobs. Or instead of using a separate resource manager, Parallel Environment manages the adapter resources itself, on behalf of the user application.
- An enhanced LAPI designed to increase parallel application performance.
- New environment variables for controlling the number of active I/O agents assigned to a job, which can help improve performance.
- In addition to MPI and LAPI, Parallel Environment 5.2 provides support for other parallel programming APIs.
- The use of the IBM MetaCluster Distributed Checkpoint Restart (MDCR) function to coordinate the checkpointing and restarting of jobs.
- An OpenSSH based user authentication method.
- Executable and library functions as 64-bit objects. The MPI library provides both 32-bit and 64-bit object support.
- The IBM High Performance Computing Toolkit, which includes an Eclipse plug-in that integrates the IBM HPC Toolkit tools with the Eclipse Integrated Development Environment (IDE).
Note: IBM is continuing support for IBM General Parallel File System (GPFS) for Power, V3.1 until September 30, 2010.
Back to top
- AIX 6.1, TL 6100-04-01 or AIX 5.3, TL 5300-07-10
- Parallel Environment APAR IZ59219 and IZ59288
- Any of these systems:
- IBM Power Systems servers (IBM POWER6)
- IBM BladeCenter® Power Architecture® servers (POWER6)
Back to top
|Planned availability date|
November 20, 2009
Back to top
Beginning with Parallel Environment for AIX, V5.2 :
- You can use the resource manager of your choice for submitting and managing
batch or interactive parallel jobs. Parallel Environment 5.2 includes a set
of resource management interfaces and data areas, for configuring your resource
manager to interact with the parallel operating environment.
Additionally, support is provided for running a job without a resource manager. You can launch a User Space job, using the InfiniBand interconnect, without the use of a resource manager such as LoadLeveler®. Instead of using a separate resource manager, Parallel Environment manages the adapter resources itself, on behalf of the user application.
- IBM's MPI implementation exploits an enhanced LAPI designed to increase parallel application performance on clusters made up of Power Systems servers running the AIX operating system.
- To help improve performance, new environment variables are provided for controlling the number of active I/O agents assigned to a job. Additionally, improvements have been made in the ability to run the parallel operating environment in large-scale environments.
- In addition to MPI and LAPI, Parallel Environment 5.2 provides support for other parallel programming APIs. These APIs can be used in any combination in a single parallel program. Options are also included for specifying how the parallel APIs in your application will use system resources.
- The underlying framework for the checkpoint and restart function has changed. Parallel Environment now uses the IBM MetaCluster Distributed Checkpoint Restart (MDCR) function, and its associated components, to coordinate the checkpointing and restarting of jobs. MDCR invokes the appropriate AIX Application Workload Partition (WPAR) commands on each node of a parallel job and coordinates the checkpoint and restart of those jobs.
- Parallel Environment provides an OpenSSH based user authentication method. With this release, support for the cluster based security (CTSec) method will be removed. Compatibility (AIX based) security is still supported and continues to be the default security method.
- Parallel Environment 5.2 provides its executable and library functions as 64-bit objects. The MPI library provides both 32-bit and 64-bit object support.
- The IBM High Performance Computing Toolkit includes an Eclipse plug-in that integrates the IBM HPC Toolkit tools with the Eclipse Integrated Development Environment (IDE). The IBM HPC Toolkit tools can be used to instrument an application program in order to obtain application performance measurements for hardware performance counters, MPI tracing, OpenMP profiling, and I/O profiling. This plug-in also allows users to view this performance measurement data for their application.
Accessibility by people with disabilities
A U.S. Section 508 Voluntary Product Accessibility Template (VPAT) containing details on accessibility compliance can be requested at
Back to top
Parallel Environment for AIX can offer HPC users whose parallel applications use the industry-standard MPI a tool for creating and using parallel applications with a familiar look and feel. For those who need increased performance, Parallel Environment uses LAPI, which is provided on the Parallel Environment media.
Flexibility is provided through a new API allowing a choice of resource manager for submitting and managing batch or interactive parallel jobs. Or Parallel Environment can manage the adapter resources itself, on behalf of the user application.
Back to top
Program Program number VRM name 5765-PEA 5.2.0 IBM Parallel Environment for AIX 5765-G66 3.1.0 IBM General Parallel File System for Power 5765-G67 3.1.0 IBM General Parallel File System for Power
Product identification number
Maintenance Maintenance Program PID number 1-year PID number 3-year PID number 5765-PEA 5660-PAE 5662-PAE 5661-PAE 5663-PAE 5664-PAE
Back to top
Product information is available via the Offering Information Web site
Back to top
The following publications are available from the Web:
- IBM Parallel Environment: Installation
- IBM Parallel Environment: Operation and Use
- IBM Parallel Environment: Messages
- IBM Parallel Environment: MPI Programming Guide
- IBM Parallel Environment: MPI Subroutine Reference
To access the above Parallel Environment documentation, refer to the IBM Cluster Information Center Web site, which contains the most recent Parallel Environment documentation in PDF and HTML formats
IBM HPC Toolkit documentation is downloadable by following the documentation link at the HPC Central Web site
The IBM Publications Center
The Publications Center is a worldwide central repository for IBM: product publications and marketing material with a catalog of 70,000 items. Extensive search facilities are provided. Payment options for orders are via credit card (in the U.S.) or customer number for 50 countries. A large number of publications are available online in various file formats, and they can all be downloaded by all countries, free of charge.
Back to top
Specified operating environment
- IBM Power Systems (POWER6)
- IBM BladeCenter Power Architecture servers (POWER6)
Parallel Environment is supported in a clustered environment interconnected with the InfiniBand host channel adapters and switches, running on IBM clusters.
In IP mode, Parallel Environment uses the standard UDP/IP protocol, which is available on many interconnects. Parallel Environment in IP mode is tested extensively on 1 Gb Ethernet and the InfiniBand host channel adapters.
- AIX 6.1, TL 6100-04-01 or AIX 5.3, TL 5300-07-10
- Parallel Environment APAR IZ59219 and IZ59288
If batch jobs are to be submitted under Parallel Environment, then an external scheduler is required. If using the preferred IBM TWS LoadLeveler, Version 4.1 (5765-L40) is required.
If Fortran or C/C++ programs are to be developed, you will need a supported level of the specific compiler. Refer to the Parallel Environment Installation Guide for the required levels.
If using the HPC Toolkit, the C and Fortran compiler and runtime requirements, along with other software requirements and instructions for downloading, are documented in the IBM High Performance Computing Toolkit Installation and Usage Guide provided on the CD-ROM.
If using the parallel checkpoint and restart function, the following is required:
- APAR IZ59981
- IBM General Parallel File System for AIX, V3.2.1, or later
Note: The supported node limit for parallel checkpoint restart is four nodes.
The program's specifications and specified operating environment information may be found in documentation accompanying the program, if available, such as a readme file, or other information published by IBM, such as an announcement letter. Documentation and other program content may be supplied only in the English language.
IBM Tivoli® Workload Scheduler LoadLeveler for AIX (5765-L40) can be used to submit batch parallel jobs. It can help assist with node allocation for job management.
General Parallel File System (GPFS) (5765-G66) is beneficial because MPI-IO depends on the underlying file system's ability to maintain file consistency when multiple tasks modify a file. An MPI job distributes tasks across many nodes, and these tasks commonly do large amounts of concurrent and even coordinated file I/O.
GPFS is designed to deliver high-performance parallel I/O between the file system and the tasks of the MPI job. MPI-IO, part of the Parallel Environment MPI, is designed to work correctly and efficiently with GPFS. Using MPI-IO for highest performance depends on the consistency for access from multiple nodes that only a true parallel file system like GPFS can provide.
Parallel Engineering and Scientific Subroutine Library (Parallel ESSL) (5765-F84) is a scalable mathematical subroutine library that provides outstanding performance through simple subroutine calls for engineering and scientific MPI applications on clusters of Power Systems servers and Power Architecture BladeCenter servers using the Parallel Environment MPI library for communications.
Parallel Environment 5.2 commands and applications are compatible only with AIX 5.3 and AIX 6.1, or later, and not with earlier versions of the AIX operating system.
If you are using TWS LoadLeveler as your resource manager, Parallel Environment 5.2 requires the 64-bit TWS LoadLeveler 4.1 shared library. Parallel Environment 5.2 does not work with earlier versions of TWS LoadLeveler.
All nodes in a parallel job must run the same versions of Parallel Environment, LAPI, and TWS LoadLeveler at the same maintenance levels. Parallel Environment 5.2 is only compatible with the following versions of LAPI and TWS LoadLeveler:
- LAPI 3.1.4 (shipped with AIX 6.1) or 2.4.7 (shipped with AIX 5.3)
- TWS LoadLeveler 4.1
Because Parallel Environment 5.2 is only compatible with TWS LoadLeveler 4.1,
when Parallel Environment is migrated to Version 5.2, TWS LoadLeveler must
also be migrated to V4.1.
Note: Existing Parallel Environment 5.1 customers should not attempt to use LAPI 3.1.4 or 2.4.7.
- Parallel Environment does not support interoperability between nodes running AIX and Linux® versions of Parallel Environment. Parallel jobs cannot be mixed between AIX and Linux Parallel Environment nodes.
- The supported node limit for Parallel Environment is 64 nodes. However, for clusters larger than 64 nodes, a request can be submitted through special bid.
- The checkpoint and restart function will be available with APAR IZ59981 and is only available when running AIX 6.1 (TL 6100-04), or later. The supported node limit for checkpoint and restart is four nodes. For a list of other limitations, refer to the IBM Parallel Environment: MPI Programming Guide .
- If checkpoint and restart is required with AIX 5.3, it is required to remain on Parallel Environment 5.1.
Parallel Environment 5.2 is the last release that will support the AIX V5 operating system.
IBM Parallel Environment for AIX is distributed as a single package available via CD-ROM media with the following:
- IBM International Program License Agreement in multilanguage booklet (LC23-5052-01) and its License Information (L-RHAN-7T8M23) in multiple languages (ppe.loc.license)
- Proof of Entitlement (POE)
- Product readme file that describes the program's specified operating environment and program specifications
- Documentation for the HPCT (/usr/lpp/ppe.hpct/doc/hpct_guide.pdf and /usr/lpp/ppe.hpct/doc/hpct_guide.html)
- Program filesets:
- Parallel Environment filesets (ppe.poe, ppe.man, ppe.shell)
- High Performance Computing Toolkit filesets (ppe.hpct, ppe.hpct.rte)
- Low-level API filesets (rsct.lapi.3.1.4 and rsct.lapi.2.4.7)
- Resource Manager filesets (LoadL.resmgr)
- MetaCluster filesets (mdcr.rte, mcr.rte)
Security, auditability, and control
Parallel Environment uses the security and auditability features of the AIX operating system. The customer is responsible for evaluation, selection, and implementation of security features, administrative procedures, and appropriate controls in application systems and communication facilities.
Back to top
IBM Software Services has the breadth, depth, and reach to manage your services needs. You can leverage the deep technical skills of our lab-based, software services team and the business consulting, project management, and infrastructure expertise of our IBM Global Services team. Also, we extend our IBM Software Services reach through IBM Business Partners to provide an extensive portfolio of capabilities. Together, we provide the global reach, intellectual capital, industry insight, and technology leadership to support a wide range of critical business needs.
To learn more about IBM Software Services or to contact a Software Services sales specialist, visit
Back to top
Part number or Program name PID number Charge metric Parallel Environment 5765-PEA Per processor core - small, for AIX medium, or large processor group including one year SWMA 1-year SWMA: 5660-PAE Per processor core - small, Registration medium, or large processor group and Renewal 1-year SWMA: 5661-PAE Per processor core - small, After-license medium, or large processor group 3-year SWMA: 5662-PAE Per processor core - small, Registration medium, or large processor group 3-year SWMA: 5663-PAE Per processor core - small, Renewal medium, or large processor group 3-year SWMA: 5664-PAE Per processor core - small, After-license medium, or large processor group
Withdrawn program features
The following features are withdrawn effective January 20, 2010.
5765-PEA - Parallel Environment for AIX Basic license one-time charge Feature Feature description number Small per block of 250 processors 0002 Medium per block of 250 processors 0004 Large per block of 250 processors 0006
Feature Feature description number 5660-PAE Maintenance no-charge 1-year registration Small per block of 250 processors 0010 Medium per block of 250 processors 0159 Large per block of 250 processors 0239 5660-PAE Maintenance 1-year 24 x 7 support Small per block of 250 processors 0297 Medium per block of 250 processors 0318 Large per block of 250 processors 0327 5662-PAE Maintenance 3-year registration (2-year uplift) Small per block of 250 processors 0004 Medium per block of 250 processors 0033 Large per block of 250 processors 0038 5662-PAE Maintenance 3-year 24 x 7 support Small per block of 250 processors 0019 Medium per block of 250 processors 0036 Large per block of 250 processors 0041 5661-PAE 1-year Software Maintenance after-license Small per block of 250 processors 0002 Medium per block of 250 processors 0006 Large per block of 250 processors 0010 5661-PAE After-license 1-year 24 x 7 support Small per block of 250 processors 0004 Medium per block of 250 processors 0008 Large per block of 250 processors 0012 5664-PAE 3-year Software Maintenance after-license Small per block of 250 processors 4497 Medium per block of 250 processors 0003 Large per block of 250 processors 0007 5664-PAE After-license 3-year 24 x 7 support Small per block of 250 processors 0001 Medium per block of 250 processors 0005 Large per block of 250 processors 0009
Note: These features will be removed in eConfig effective October 20, 2009.
No changes or updates from prior release.
System Program Order (SPO): An order for SPO 5692-A5L or 5692-A6P is mandatory for shipments of program media distribution. The individual licensed program orders are for registration and billing purposes only. No physical deliveries occur as a result of those orders.
The SPO (5692-A5L for AIX 5 or 5692-A6P for AIX 6) must contain the following features, as previously announced, to deliver a machine-readable copy of the product using the currently available media.
Feature Program number Program/Function name number 5692-A5L Parallel Environment for AIX, V5 2218 5692-A6P Parallel Environment for AIX, V5 2218
Expedite feature for licensed programs and SWMA programs Feature Program number Description number 5765-PEA Branch Office Expedite 3445 5660-PAE Branch Office Expedite 3445 5661-PAE Branch Office Expedite 3445 5662-PAE Branch Office Expedite 3445 5663-PAE Branch Office Expedite 3445 5664-PAE Branch Office Expedite 3445
Back to top
|Terms and conditions|
The information provided in this announcement letter is for reference and convenience purposes only. The terms and conditions that govern any transaction with IBM are contained in the applicable contract documents such as the IBM International Program License Agreement, IBM International Passport Advantage® Agreement, and the IBM Agreement for Acquisition of Software Maintenance.
IBM International Program License Agreement including the License Information document and Proof of Entitlement (PoE) govern your use of the program. PoEs are required for all authorized use.
This software license includes Software Subscription and Support (also referred to as Software Maintenance).
Agreement for Acquisition of Software Maintenance
The IBM Agreement for Acquisition of Software Maintenance (Z125-6011) agreement applies for subscription and support (also referred to as Software Maintenance) and does not require customer signatures.
These programs are licensed under the IBM Program License Agreement (IPLA) and the associated Agreement for Acquisition of Software Maintenance, which provide for support with ongoing access to releases and versions of the program. IBM includes one year of Software Subscription and Support with the initial license acquisition of each program acquired. The initial period of Software Subscription and Support can be extended by the purchase of a renewal option, if available. These programs have a one-time license charge for use of the program and an annual renewable charge for the enhanced support that includes telephone assistance (voice support for defects during normal business hours), as well as access to updates, releases, and versions of the program as long as support is in effect.
License Information form number
The program's License Information will be available for review on the IBM Software License Agreement Web site
Limited warranty applies
IBM warrants that when the program is used in the specified operating environment, it will conform to its specifications. The warranty applies only to the unmodified portion of the program. IBM does not warrant uninterrupted or error-free operation of the program or that IBM will correct all program defects. You are responsible for the results obtained from the use of the program.
IBM provides you with access to IBM databases containing information on known program defects, defect corrections, restrictions, and bypasses at no additional charge. For further information, consult the IBM Software Support Handbook found at
IBM will maintain this information for at least one year after the original licensee acquires the program (warranty period).
If for any reason you are dissatisfied with the program and you are the original licensee, you may obtain a refund of the amount you paid for it, if within 30 days of your invoice date you return the program and its PoE to the party from whom you obtained it. If you downloaded the program, you may contact the party from whom you acquired it for instructions on how to obtain the refund.
For clarification, note that for programs acquired under any of IBM's On/Off Capacity on Demand (On/Off CoD) software offerings, this term does not apply since these offerings apply to programs already acquired and in use by you.
Authorization for use on home/portable computer
You may not copy and use this program on another computer without paying additional license fees.
Volume orders (IVO)
Yes. Contact your IBM representative.
Passport Advantage applies
Software Subscription and Support applies
Yes. All distributed software licenses include Software Subscription and Support (also referred to as Software Maintenance) for a period of 12 months from the date of acquisition, providing a streamlined way to acquire IBM software and assure technical support coverage for all licenses. Extending coverage for a total of three years from date of acquisition may be elected.
While your Software Subscription and Support is in effect, IBM provides you assistance for your routine, short duration installation and usage (how-to) questions, and code-related questions. IBM provides assistance via telephone and, if available, electronic access, only to your information systems (IS) technical support personnel during the normal business hours (published prime, shift hours) of your IBM support center. (This assistance is not available to your end users.) IBM provides Severity 1 assistance 24 hours a day, every day of the year. For additional details, consult your IBM Software Support Handbook at
Software Subscription and Support does not include assistance for the design and development of applications, your use of programs in other than their specified operating environment, or failures caused by products for which IBM is not responsible under this agreement.
IBM Operational Support Services - SoftwareXcel
System i Software Maintenance applies
Variable charges apply
Educational allowance available
Yes. A 15% education allowance applies to qualified education institution customers.
Sub-capacity pricing terms and conditions
To be eligible for sub-capacity pricing, the machine on which the eligible products are installed and running must be eligible for sub-capacity pricing terms and conditions. Software pricing, at less than full machine capacity for eligible products, apply when running:
- AIX 5L V5.1, or later, on an IBM System p® 690 or equivalent partition-capable operating system and machine
- OS/400® V5R1, or later, running on an IBM System i®
- Linux running in an LPAR under AIX 5L V5.1, OS/400 V5R1, or in a partition on an equivalent partition-capable operating system and server
Sub-capacity pricing for eligible products is based on the current program pricing methodology, but the number of processors will be determined based on the sum of processors for all partitions where the program is defined (used). To obtain pricing at less than full machine capacity for eligible products, you are required to:
- Install and use, when available, IBM's license use management program, which installs with eligible IBM programs.
- Install available updates to the operating system and eligible products such that license use can be accurately managed.
- Determine if the use of sub-capacity pricing terms results in a reduced requirement for entitlements; you can reallocate the entitlement difference by distributing entitlements across a larger or different set of systems, or reserve them for future growth. There will be no refunds for these freed up entitlements. Subscription, Software Maintenance and support volumes, and entitlements for existing contracts will continue at the same levels as the acquired licenses.
Back to top
Information on charges is available at
Choose the option entitled Purchase/upgrade tools.
The licensed program (5765-PEA) and associated Software Maintenance (566x-PAE) have been previously announced and priced.
Extended support attachments:
For current prices, contact 1-888-Shop-IBM or your local IBM representative.
Back to top
To order, contact the Americas Call Centers or your local IBM representative, or your IBM Business Partner.
To identify your local IBM representative or IBM Business Partner, call 800-IBM-4YOU (426-4968).
Phone: 800-IBM-CALL (426-2255) Fax: 800-2IBM-FAX (242-6329) For IBM representative: email@example.com For IBM Business Partner: firstname.lastname@example.org Mail: IBM Teleweb Customer Support ibm.com® Sales Execution Center, Americas North 3500 Steeles Ave. East, Tower 3/4 Markham, Ontario Canada L3R 2Z1
The Americas Call Centers, our national direct marketing organization, can add your name to the mailing list for catalogs of IBM products.
Note: Shipments will begin after the planned availability date.
General Parallel File System, GPFS, POWER6 and AIX 5L are trademarks of IBM Corporation in the United States, other countries, or both.
IBM, AIX, Power, BladeCenter, Power Architecture, LoadLeveler, Tivoli, Passport Advantage, System p, OS/400, System i and ibm.com are registered trademarks of IBM Corporation in the United States, other countries, or both.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
Other company, product, and service names may be trademarks or service marks of others.
For the most current information regarding IBM products, consult your IBM representative or reseller, or visit the IBM worldwide contacts page
Back to top