IBM General Parallel File System Multiplatform V3.2 delivers superior scalability and performanceIBM United States Software Announcement 207-178
August 7, 2007
(Corrected on August 10, 2007)
|At a glance|
IBM General Parallel File System (GPFS) Multiplatform V3.2 offers:
- Administration, manageability, and monitoring improvements
- Improvements in migration and compatibility between GPFS releases
- Multicluster enhancements to support multiple active Network Shared Disk (NSD) servers per logical unit number (LUN)
- Clustered Network File System (NFS), which is a high availability solution for NFS exporting of file systems
- Improvements to performance, scaling, availability, and reliability
- Support for Red Hat Enterprise Linux 5
Back to top
GPFS Multiplatform (System x, BladeCenter® HS20, e325, and e326) is ideal for high-performance parallel file transfer and parallel I/O to single or multiple files. GPFS delivers multicluster support, scalability and performance, support for extremely large files, failure recovery, and ease of administration.
Version 3.2 enhancements:
- To facilitate migration, you can install new GPFS code one node at a time without shutting down GPFS on other nodes. Capability enhancements now allow for backward compatibility between GPFS V3.2 and V3.1 remote clusters.
- Designate up to eight NSD servers per LUN.
- GPFS for Linux offers the Clustered Network File System support, which is a high availability solution for NFS exporting of file systems.
- Use of low-level InfiniBand protocols may provide increased bandwidth, which helps to improve the price/performance of I/O servers and clients. CPU utilization may also be reduced by using the Remote Direct Memory Access (RDMA) offered by InfiniBand.
- GPFS cluster monitoring capability through the utilization of an SNMP-based management application. This requires a Linux node to be installed in order to collect the data.
- Performance enhancements include parallel defragmentation of a file system, larger pagepool support, and directory-locking improvements for concurrent file creates and deletes from multiple nodes (directory-locking improvements will be available in APAR IZ01431).
- GPFS can attempt to reconnect broken sockets.
- Mount up to 256 file systems.
- Trace improvements increase the ability to obtain accurate and reliable problem determination information.
- Support is added for backup and restore policy attributes, tighter integration with HSM, and other additional ILM policy language enhancements.
- GPFS V3.2 allows the policy code to run in parallel across all nodes in the home cluster that have the file system mounted.
- GPFS offers support for Red Hat Enterprise Linux 5.
Back to top
|Planned availability date|
October 5, 2007
Back to top
GPFS Multiplatform (System x, BladeCenter HS20, e325, and e326) delivers file system services to parallel and serial applications. It gives parallel applications simultaneous access to the same or different files from any node that has the GPFS file system mounted, while managing a high level of control over all file system operations. GPFS is particularly appropriate in an environment where the aggregate peak need for data bandwidth exceeds the capability of a distributed file system server.
GPFS V3.2 enhancements
In the past, migrating to a new release of GPFS required shutting down GPFS and upgrading all nodes before GPFS could be restarted. GPFS V3.2 supports rolling upgrades and a limited form of backward compatibility:
- Rolling upgrades enable you to install new GPFS code one node at a time without shutting down GPFS on other nodes. It is expected that all nodes will be upgraded within a short time. Some features become available on each node as soon as the node is upgraded, while other features become available as soon as all participating nodes are upgraded.
- Backward compatibility allows running with a mixture of old and new nodes. Multi-cluster environments may be able to upgrade the local cluster while still allowing mounts from remote nodes in other clusters that have not yet been upgraded.
You can designate up to eight NSD servers to simultaneously service I/O requests from different clients. Each of these NSD servers must have physical access to the same LUN. Different servers can serve I/O to different nonintersecting sets of clients for a variety of reasons, such as load balancing on the server, network partitioning by balancing the load on different networks, or workload partitioning. Multiple NSD server functions require all (peer) NSD servers to be part of the same GPFS cluster. The existing subnet functions in GPFS determine which NSD server should serve a particular client. The assumption is that nodes within a subnet are connected via high-speed networks.
GPFS for Linux offers Clustered NFS support to enable highly available NFS exporting of file systems. Some or all nodes in an existing GPFS cluster also serve NFS and are part of the NFS cluster. All of the nodes in the NFS cluster export the same file systems to the NFS clients.
- Monitoring: Every node in the NFS cluster runs an NFS monitoring utility that monitors GPFS and the NFS and networking components on the node. After failure detection, and based on customer configuration, the monitoring utility may invoke a failover.
- Failover: The automatic failover procedure transfers the NFS serving load from the failing node to another node in the NFS cluster. The failure is managed by the GPFS cluster, including lock and state recovery.
- Load balancing: The IP address is the load unit that can be moved from one node to another because of failure or load balancing needs. This solution supports a failover of all the node's load (all NFS IP addresses) as one unit to another node. However, if no locks are outstanding, individual IP addresses can be moved to other nodes for load balancing purposes.
The GPFS InfiniBand RDMA code uses RDMA for NSD client file I/O requests. RDMA transfers data directly between the NSD client memory and the NSD server memory instead of sending and receiving the data over the TCP socket. Using RDMA may improve bandwidth and decrease CPU utilization.
Monitoring is enabled with the SNMP-based management application Net-SNMP. SNMP requires a Linux node installed in order to collect the data. Monitoring involves gaining a view of the GPFS system. Monitored information can be roughly grouped into the categories of configuration, status, and performance.
- Configuration denotes the initially customized aspects of the system's current state.
- Status information is dynamic information that expresses the current health of nodes, disks, and other hardware, including any reported error conditions, disk utilization, and fragmentation.
- Performance information includes quantitative measurement of the workings of a system.
Performance enhancements include support for parallel defragmentation of a file system, larger pagepool support, and directory-locking:
- Defragmentation of a filesystem can now be run in parallel across nodes in a cluster.
- Maximum pagepool support is increased to 256 GB.
- Directory-locking improvements for concurrent file creates and deletes from multiple nodes (function will be available in APAR IZ01431).
When a socket connection breaks due to a network failure, GPFS now tries to re-establish the connection rather than immediately initiating node expulsion procedures.
Mount up to 256 file systems.
GPFS V3.2 extends Information Lifecycle Management functionality to integrate with HSM products. A single set of policies can be used to move data across different storage pools of a file system and to move data from GPFS to HSM storage and from HSM storage to GPFS storage. Additional enhancements include the ability to use policies while restoring data.
GPFS V3.2 enables the policy code to run in parallel across all nodes in the home cluster that have the file system mounted. The policy evaluation can then scale with the size of the cluster.
Tracing is improved to increase the reliability of trace data gathering from customers. It enhances the ability of getting accurate and reliable problem determination information. The new trace command mmtracectl contains the following functions:
- Turn trace on or off on next session to automatically start trace when GPFS starts
- Allow for predefined trace levels: I/O, all, def, same (do not change), and user-specified trace levels
- Enable changing size of trace buffers
- Allow a user-defined dictionary for keeping raw trace files and internal dumps
- Specify whether to perform internal dumps on abnormal GPFS shutdowns or both local and global shutdowns
Accessibility by people with disabilities
A U.S. Section 508 Voluntary Product Accessibility Template containing details on accessibility compliance can be requested at
Back to top
GPFS is ideal for several different roles in clustered computing and shared storage:
- A multicluster file system for parallel processing on compute clusters, featuring extreme scalability and throughput optimized for streaming workloads such as those common in Web 2.0, digital media, scientific, and engineering applications. Users are allowed shared access to files in either the cluster where the file system was created or other GPFS clusters. Each site in the network is managed as a separate cluster, while allowing shared file system access.
- An information lifecycle management file system, enabling multiple tiers of storage, including tape, with powerful policy-driven file placement, migration, archival, and deletion. The unique GPFS policy engine enables file migration scalable to billions of files and multiple petabytes of data.
- A scalable, shared storage file system for storage consolidation and virtualization. This includes Clustered NFS support, providing a scalable shared namespace, enabling horizontally scalable file serving. Additionally, GPFS includes many data/SAN management features, including storage hardware-agnostic snapshots/mirroring.
- A high availability file system for any business application or database that may benefit from very rapid failover of a consistent application data environment.
- A distributed file system, providing a global, cross-platform coherent namespace across an enterprise for around-the-world, around-the-clock operations.
Back to top
Refer to 207-179 , dated August 7, 2007.
Back to top
Product information is available via the Offering Information Web site
Also, visit the Passport Advantage® Web site
Back to top
No publications are shipped with this program; however, the following publications are available from the Web:
IBM General Parallel File System:
- Concepts, Planning, and Installation Guide (GA76-0413)
- Administration and Programming Reference (SA23-2221)
- Problem Determination Guide (GA76-0415)
- Data Management API Guide (GA76-0414)
- Advanced Administration Guide (SC23-5182)
You can view the manuals by selecting the GPFS Library link from the IBM Cluster Information Center at
The IBM Publications Center
The Publications Center is a worldwide central repository for IBM product
publications and marketing material with a catalog of 70,000 items.
Extensive search facilities are provided. Payment options for orders are
via credit card (in the U.S.) or customer number for 50 countries. A
large number of publications are available online in various file
formats, and they can all be downloaded by all countries, free of charge.
Back to top
Any of the following IBM servers:
- System x
- Cluster 1350
- System 325, 326, or 326M
Note: Use of GPFS on compatible non-IBM hardware is documented in the GPFS FAQ available at
Any of the following connectivity and storage:
- 100 Mb Ethernet, 1 Gb Ethernet, Myrinet (IP only), or InfiniBand (IP and optionally RDMA VERBS specifically for data transfers)
- IBM System Storage DS4000 disk (DS4100, DS4300, DS4400, DS4500)
- IBM System Storage ESS (2105-F20 or 2105-800)
Note: Refer to the GPFS FAQ available from the following Web site for an updated list of formally qualified disk subsystems and servers
- GPFS Service update 3.2.0-1
One of the following Linux distributions:
- Red Hat Enterprise Linux 4
- Red Hat Enterprise Linux 5
- SUSE Linux Enterprise Server 9
- SUSE Linux Enterprise Server 10
For service information, visit
For updated supported kernel levels, consult the FAQ available from the GPFS Library Link from the IBM Cluster Information Center at
Companion products: Related products to GPFS are:
- IBM DS and DCS disk arrays
- IBM GPFS for POWER
- Tivoli® Storage Manager
- Tivoli Provisioning Manager
- HPSS (High Performance Storage System)
- IBM System Storage SAN Volume Controller
Compatibility: GPFS V3.2 supports:
Rolling migration within a cluster:
- You may now install new GPFS code one node at a time without shutting down GPFS on other nodes.
- The availability of new features in GPFS is staged until all nodes in the cluster have been upgraded.
Backward compatibility between clusters:
- You may now upgrade the local cluster to GPFS V3.2 while still allowing remote clusters at the V3.1 level to mount local file systems.
The mmchfs -V command has new options in support of rolling migration and
- full enables all upgradeable format changes for the file system, and remote cluster running GPFS V3.1 will no longer be able to mount the file system.
- compat enables backward-compatible format changes for the file system, allowing remote clusters running GPFS V3.1 to continue to mount the file system.
Limitations: For complete and up-to-date information regarding GPFS specifications and capacities, refer to the GPFS Multiplatform (5724-N94) Sales Manual, and to the GPFS FAQ available from
For additional information, refer to usage restrictions in the Terms and conditions section of this announcement, or to the License Information document that is available on the IBM Software License Agreement Web site at
The following enhancements will not be available at planned availability, but will be available in a service update:
- Snapshots can be created in a DMAPI-enabled file system. This enhancement will permit the Data Storage Management (XDSM) API to read data files in an online GPFS snapshot for point-in-time backup and recovery. Function will be available in APAR 1Z00523.
- Further support will be added for backup and restore policy attributes. A SHOW clause will be added to the policy language to support aggregation of smaller files into larger units. File aggregation is beneficial on high-speed tapes where the marks between files become expensive. To reduce this expense, the SHOW clause will allow you to aggregate small files into larger units and reduce the number of tape marks. Function will be available in APAR 1Z00524.
- A performance enhancement will include directory-locking improvements for concurrent file creates and deletes from multiple nodes. This will be available in APAR IZ01431.
Packaging: IBM GPFS Multiplatform is distributed via electronic software distribution (ESD) or CD-ROM. A product README file and License Information is shipped with the product. Publications are available online.
Security, auditability, and control
GPFS Multiplatform V3.2 uses the security and auditability features of the Linux operating system.
The customer is responsible for evaluation, selection, and implementation of security features, administrative procedures, and appropriate controls in application systems and communication facilities.
IBM Software Services has the breadth, depth, and reach to manage your services needs. You can leverage the deep technical skills of our lab-based software services team and the business consulting, project management, and infrastructure expertise of our IBM Global Services team. Also, we extend our IBM Software Services reach through IBM Business Partners to provide an unmatched portfolio of capabilities. Together, we provide the global reach, intellectual capital, industry insight, and technology leadership to support any critical business need.
ITS Specific Service offering supports the GPFS Multiplatform product.
To learn more about IBM Software Services or to contact a Software Services sales specialist, visit
Back to top
This product is only available via Passport Advantage. It is not available as shrinkwrap.
Product group: Other Software Product Identifier Description (PID) General Parallel File System Multiplatform (5724-N94) Product category: System Software
Part number or Program name PID number Charge metric GPFS Multiplatform 5724-N94 10 processor Value Units
A processor (commonly called a CPU or core) is a functional unit within a computing device that interprets and executes instructions. A processor consists of at least an instruction control unit and one or more arithmetic or logic unit. With multicore technology, each core is considered a processor. With full capacity licensing, a Proof of Entitlement (PoE) must be acquired for all activated processors available for use on the server.
Value Unit: A Value Unit is a pricing charge metric for program license entitlements that is based upon the quantity of a specific designated measurement used for a given program. Each program has a designated measurement. The most commonly used designated measurement is a processor core. However, for select programs, there are other designated measurements such as users, client devices, and messages. The number of Value Unit entitlements required for a program depends on how the program is deployed in your environment and must be obtained from a Value Unit table. You must obtain a PoE for the calculated number of Value Unit entitlements for your implementation. The Value Unit entitlements are specific to a program and may not be exchanged, interchanged, or aggregated with Value Unit entitlements of another program.
A processor core is a functional unit within a computing device that interprets and executes instructions. A processor core consists of at least an instruction control unit and one or more arithmetic or logic unit. Not all processor cores require the same number of Value Unit entitlements. To calculate the number of Value Unit entitlements required, refer to the Value Unit Table at
With full capacity licensing, a PoE must be acquired for the appropriate number of Value Units based on all activated processor cores available for use on the server.
Part Program name/description number IBM General Parallel File System Multiplatform V3.2 BH031EN English CD Media Pack General Parallel File System 10 Value Units D61HGLL LIC+SW MAINT 12 MO General Parallel File System 10 Value Units E0414LL ANNUAL SW MAINT RNWL General Parallel File System 10 Value Units D61HHLL SW MAINT REINSTATE 12 MO
Passport Advantage customer: Media pack entitlement details
Customers with active maintenance or subscription for the products listed are entitled to receive the corresponding media pack.
Entitled maintenance offerings Description IBM General Parallel File System for Linux Multiplatform Media packs description Part number IBM General Parallel File System for Linux on BH02AIE xSeries® Multiplatform V2.3.0 IE CD Media Pack IBM General Parallel File System for Linux BH02ZIE Multiplatform V3.1 IE CD Media Pack
Withdrawal of previous Passport Advantage part numbers: The following IPLA part numbers are being replaced or obsoleted by this announcement. The effective withdrawal date is October 5, 2007.
Orders for these part numbers will not be accepted after the stated effective date of withdrawal, nor will normal marketing activities or educational support be available unless previous agreement exists between the customer and IBM.
Withdrawn from marketing information
Part Program name/description number IBM General Parallel File System for D53GZLL Linux Multiplatform Processor License + SWM 12 months IBM General Parallel File System for E016TLL Linux Multiplatform Processor Annual SWM Renewal IBM General Parallel File System for D53H0LL Linux Multiplatform Processor SWM Reinstatement 12 months
On/Off Capacity on Demand (CoD)
Part Description number IBM General Parallel File System Multiplatform D57QTLL Processor OOCoD Temporary Use Charge
Back to top
|Terms and conditions|
The information provided in this announcement letter is for reference and convenience purposes only. The terms and conditions that govern any transaction with IBM are contained in the applicable contract documents such as the IBM International Program License Agreement, IBM International Passport Advantage Agreement, and the IBM Agreement for Acquisition of Software Maintenance.
This product is only available via Passport Advantage. It is not available as shrinkwrap.
Agreement: IBM International Program License Agreement including the License Information document and PoE govern your use of the program. PoEs are required for all authorized use. Part number products only, offered outside of Passport Advantage, where applicable, are license only and do not include Software Maintenance.
License Information form number: L-KFIS-6YZPME
The program's License Information will be available for review on the IBM Software License Agreement Web site
Limited warranty: Yes
Limited warranty: IBM warrants that when the program is used in the specified operating environment, it will conform to its specifications. The warranty applies only to the unmodified portion of the program. IBM does not warrant uninterrupted or error-free operation of the program or that IBM will correct all program defects. You are responsible for the results obtained from the use of the program.
IBM provides you with access to IBM databases containing information on known program defects, defect corrections, restrictions, and bypasses at no additional charge. Consult the IBM Software Support Guide for further information at
IBM will maintain this information for at least one year after the original licensee acquires the program (warranty period).
Program technical support: Technical support of a program product will be available for a minimum of three years from the general availability date, as long as your Software Maintenance is in effect. This technical support allows you to obtain assistance (via telephone or electronic means) from IBM for product-specific, task-oriented questions regarding the installation and operation of the program product. Software Maintenance also provides you with access to updates, releases, and versions of the program. You will be notified, via announcement letter, of discontinuance of support with 12 months' notice. If you require additional technical support from IBM, including an extension of support beyond the discontinuance date, contact your IBM representative or IBM Business Partner. This extension may be available for a fee.
Money-back guarantee: If for any reason you are dissatisfied with the program and you are the original licensee, you may obtain a refund of the amount you paid for it, if within 30 days of your invoice date you return the program and its PoE to the party from whom you obtained it. If you downloaded the program, you may contact the party from whom you acquired it for instructions on how to obtain the refund.
For clarification, note that (1) for programs acquired under the IBM International Passport Advantage offering, this term applies only to your first acquisition of the program and (2) for programs acquired under any of IBM's On/Off Capacity on Demand (On/Off CoD) software offerings, this term does not apply because these offerings apply to programs already acquired and in use by you.
Authorization for use on home/portable computer: You may not copy and use this program on another computer without paying additional license fees.
Volume orders (IVO): No
Passport Advantage applies: Yes, and through the Passport Advantage Web site at
Software Maintenance applies: Yes. Software Maintenance is included with licenses purchased through Passport Advantage and Passport Advantage Express. Product upgrades and Technical Support are provided by the Software Maintenance offering as described in the Agreements. Product upgrades provide the latest versions and releases to entitled software and Technical Support provides voice and electronic access to IBM support organizations, worldwide.
IBM includes one year of Software Maintenance with each program license acquired. The initial period of Software Maintenance can be extended by the purchase of a renewal option, if available.
While your Software Maintenance is in effect, IBM provides you assistance for your routine, short duration installation and usage (how-to) questions, and code-related questions. IBM provides assistance via telephone and, if available, electronic access, only to your IS technical support personnel during the normal business hours (published prime shift hours) of your IBM support center. (This assistance is not available to your end users.) IBM provides Severity 1 assistance 24 hours a day, 7 days a week. For additional details, consult your IBM Software Support Guide at
Software Maintenance does not include assistance for the design and development of applications, your use of programs in other than their specified operating environment, or failures caused by products for which IBM is not responsible under the applicable agreements.
For additional information about the International Passport Advantage Agreement and the IBM International Passport Advantage Express Agreement, visit the Passport Advantage Web site at
IBM Operational Support Services SoftwareXcel: No
System i Software Maintenance applies: No
Variable charges apply: No
Educational allowance available: Not applicable
On/Off Capacity on Demand (CoD)
To be eligible for On/Off CoD pricing, you must be enabled for temporary capacity on the
corresponding hardware, and the required contract, Amendment for iSeries and pSeries® Temporary
Capacity On Demand Software (Z125-6907), must be signed prior to use.
Back to top
|IBM Electronic Services|
IBM has transformed its delivery of hardware and software support services to help you achieve higher system availability. Electronic Services is a Web-enabled solution that offers an exclusive, no-additional-charge enhancement to the service and support available for IBM servers. These services are designed to provide the opportunity for greater system availability with faster problem resolution and preemptive monitoring. Electronic Services comprises two separate, but complementary, elements: Electronic Services news page and Electronic Services Agent.
The Electronic Services news page is a single Internet entry point that replaces the multiple entry points traditionally used to access IBM Internet services and support. The news page enables you to gain easier access to IBM resources for assistance in resolving technical problems.
The Electronic Service Agent is no-additional-charge software that resides on your server. It monitors events and transmits system inventory information to IBM on a periodic, client-defined timetable. The Electronic Service Agent automatically reports hardware problems to IBM. Early knowledge about potential problems enables IBM to deliver proactive service that may result in higher system availability and performance. In addition, information collected through the Service Agent is made available to IBM service support representatives when they help answer your questions or diagnose problems. Installation and use of IBM Electronic Service Agent for problem reporting enables IBM to provide better support and service for your IBM server.
To learn how Electronic Services can work for you, visit
Back to top
For Passport Advantage information and charges, contact your IBM representative or authorized IBM Business Partner. Additional information is also available on the following Passport Advantage Web site
Back to top