IBM United States
Software Announcement 200-223
July 18, 2000
New Release of General Parallel File System for AIX Increases File Size and File System Size
(Corrected on August 8, 2000)
The prerequisite version of AIX was corrected.
At a Glance
General Parallel File System for AIX (GPFS) is a shared disk file system that allows access to files within an RS/6000 SP system from any node on the system. Enhancements for Release 3 include:
General Parallel File System (GPFS) for AIX® Version 1.3 is a parallel file system that supports almost all of the interfaces in today's popular UNIX® operating system standards. GPFS implements a shared disk model and allows access to files within an RS/6000® SP system from any node in the system. This provides high-performance file Input/Output (I/O) to parallel jobs running on multiple RS/6000 SP nodes or to serial jobs that are scheduled to run on RS/6000 SP nodes based on node availability.
New in Release 3
GPFS Version 1.3 provides several performance, scalability, standards compliance, and usability enhancements. These include improved performance in the areas of directory operations, strided I/O, memory-mapped file support, administration commands, and scalability. In addition, GPFS Version 1.3 offers exploitation of SP Security Services using Distributed Computing Environment (DCE) and export of GPFS file systems via Distributed File Service (DFS).
Withdrawal of Version 1.1 and Version 1.2
GPFS for AIX Version 1.1 will be withdrawn from marketing November 3, 2000. GPFS for AIX Version 1.2 will be withdrawn from marketing December 31, 2000.
General Parallel File System (GPFS) for AIX Version 1.3 is a shared disk model parallel file system that executes on all nodes of an RS/6000 SP system. The use of twin-tailed disks and backup IBM Virtual Shared Disk or Concurrent Virtual Shared Disk (PSSP 3.2 only) servers allows applications to continue to operate without interruption in the event of server or adapter failure.
In Release 3, the supported value for maximum file system size has been increased to nine terabytes (TB). Multiple levels of indirection allow file sizes up to the largest supported GPFS file system size. GPFS now internally controls the values for indirect block size and i-node size. System control of these values allows for more effective caching of i-nodes and may improve the performance of some applications.
GPFS exploits the Low-Level Application Programming Interface (LAPI) function of PSSP. Using LAPI instead of TCP/IP should improve the performance of token communication between the GPFS daemons. However, when using LAPI as the communications protocol, there is an administration limitation: a switch adapter window must be reserved on each node in the GPFS nodeset.
GPFS I/O performance and server failover are improved by their use of enhancements in the Parallel System Support Programs (PSSP) Version 3.2 IBM Virtual Shared Disk components. CPU utilization is reduced using the VSD Kernel Low-level API (KLAPI) for data protocol in supported environments and disk server failover is improved with CVSD.
GPFS file systems may now be exported via the DFS protocol, which moves GPFS data outside of the SP configuration to any network-attached system supporting DFS.
GPFS file systems can now be managed and monitored by using the Data Management Application Programming Interface (DMAPI) for GPFS. This is the GPFS implementation of the X/OPEN Data Storage Management Standard (XDSM) and allows vendors of storage management applications to provide Hierarchical Storage Management (HSM) function for GPFS file systems.
Support is now provided for mmap, munmap, and msync memory-mapped file capabilities as per the X/OPEN 4.2 standard.
GPFS programming interfaces now support both 32-bit and 64-bit applications.
Additional block sizes of 512 KB and 1024 KB are supported, which may allow reduced disk overhead while providing optimal RAID performance by matching RAID block sizes for 8+P or 16+P.
Access range "hints" tell GPFS that access to a given file will be confined to specified ranges of the file and that locks and prefetch should be confined to ranges. Also, "hints" specify completion of use of the file to allow resources to be freed quickly.
This program is not impacted by euro currency.
For scientific and technical computing usage, GPFS provides a single global file system for the SP system and is ideal for high performance parallel file transfer and parallel I/O to single or multiple files into and out of the SP. For Business Intelligence usage, GPFS provides parallel enablement for applications such as SAS, improves performance of database loading utilities, and supports parallel operation of online analysis processing using tools not enabled for parallel use.
The following class is available:
Course Course Title Number RS/6000(R) SP(TM) Implementing GPFS AU88
Visit the following Web site for locations, availability dates, and additional information:
The appropriate curriculum will be updated, as necessary, to include the enhancements described in this announcement.
Call IBM Education and Training at 800-IBM-TEACH (426-8322) for catalogs, schedules and enrollments.
Product information will be available on day of announcement through Offering Information (OITOOL) at:
No hardcopy publications are shipped with this program.
Displayable Softcopy Publications: The following publications are offered in displayable softcopy form:
These displayable manuals are part of the basic machine-readable material and are viewable in PDF and HTML formats. These publications are also available on the SP Resource Center, which is shipped with PSSP and will be available on the RS/6000 Web site at general availability of the product.
Terms and conditions for use of the machine-readable files are shipped with the files.
GPFS supports the X/OPEN 4.2 standard.
Specified Operating Environment
All applications that executed on the previous release of GPFS will execute on the new level of GPFS. File systems created under the previous release of GPFS may continue to be used under the new level of GPFS. However, once a GPFS file system has been explicitly changed to the new level by issuing the mmchfs command with the -V option, the disk images can no longer be read by a back-level file system. You will be required to recreate the file system from the backup medium and restore the content if you choose to go back after this command has been issued.
GPFS Version 1.1 will not support future releases of AIX. Customers must migrate to a more current release of GPFS after migrating their AIX level.
Performance Considerations: Communication performance may be improved between the GPFS daemons if using the Low-Level Application Programming Interface (LAPI) function of Parallel System Support Programs (PSSP) instead of Transmission Control Protocol/Internet Protocol (TCP/IP). However, when using LAPI, a switch adapter window must be reserved on each node in the GPFS nodeset. To successfully reserve an adapter window for use by GPFS, the user must stop all LoadLeveler® jobs before issuing GPFS configuration commands. If LoadLeveler for AIX (5765-D61) is not stopped, it will reserve all adapter windows for its own use and one cannot be reserved for GPFS. In addition, there will be no window available for application user space communications.
To plan for performance and recovery, customers should evaluate the number of disks attached to multiple IBM Virtual Shared Disks servers and have sufficient disks to replicate key control structures.
GPFS file systems and the set of nodes it runs on must follow these coexistence guidelines:
It is required that all nodes within a GPFS nodeset, that use a given file system, are not only at the same level of GPFS, but are also upgraded to that level at the same time.
If a system is going to have some GPFS nodesets with GPFS Version 1.2 installed and other nodesets with GPFS Version 1.3 installed, the CWS must have GPFS Version 1.3 installed.
New functions existing in GPFS Version 1.3, such as Data Management API, DFS export, larger block sizes, and multiple indirection levels, will create data structures that are not recognizable by earlier versions of GPFS. GPFS will not allow these data structures to be created, and therefore new functions may not be usable until you specifically authorize these changes by issuing the mmchfs command with the -V option.
In order to use the 64-bit versions of the GPFS programming interfaces, you must recompile your code using the appropriate 64-bit options for your compiler.
Customer Responsibilities: If your RS/6000 SP system has dedicated IBM Virtual Shared Disk server nodes, GPFS does not need to be installed on those nodes. However, if GPFS is not installed, you cannot access GPFS files from those nodes.
Installability: Any customer migrating to GPFS Version 1.3 and PSSP Version 3.2 must install PSSP Version 3.2 first. GPFS Version 1.3 supports nodesets (GPFS groups) such that two GPFS sets of nodes at different levels can coexist in the same partition while migrating. Refer to the Installation and Tuning Guide when upgrading.
Security, Auditability, and Control
General Parallel File System uses the security and auditability features of the AIX Version 4 Operating System and the RS/6000 SP.
The customer is responsible for evaluation, selection, and implementation of security features, administrative procedures, and appropriate controls in application systems and communication facilities.
GPFS Version 1.3 has one charge unit based on node installations. One-time charge (OTC) feature numbers are available in variable order quantities based on the number of nodes in your RS/6000 SP system that will be running GPFS.
General Parallel File System V1.3 is available at an upgrade price for those customers currently licensed for GPFS Version 1.1 or GPFS Version 1.2. Upgrades may be acquired up to the current level of use authorized for the qualifying programs.
Program Name: General Parallel File System for AIX, Version 1.3.
New users of General Parallel File System Version 1.3 should specify Type 5765, Model B95, asset feature number 9001, and the feature number equal to the total number of desired licenses to be installed times the quantity of installs. For example, if ordering GPFS for 12 nodes, order the feature number for Order Type Description 11 to 50 Installs with a quantity equal to 12.
Machine Type/Model: 5765-B95 OTC Medium Order Type Feature Feature Description Number Number Medium 1 to 5 Installs 3001 5214 CD-ROM 6 to 10 Installs 3002 11 to 50 Installs 3003 51 or More Installs 3004
GPFS Version 1.2 will remain orderable until December 31, 2000. Refer to Software Announcement 298-364 , dated October 5, 1998, for ordering information.
GPFS Version 1.1 will remain orderable until November 3, 2000. Refer to Software Announcement 297-318 , dated August 26, 1997, for ordering information.
Current licensees of General Parallel File System Version 1.1 or 1.2 who wish to upgrade to GPFS Version 1.3 should specify Type 5765 and Model B95 and order an upgrade equal to the current level of use authorized. For example, if currently licensed to use GPFS Version 1.1 or Version 1.2 on eight nodes, order the feature number for Order Type Description six to ten Installs with a quantity equal to eight.
Machine Type/Model: 5765-B95 OTC Medium Order Type Feature Feature Description Number Number Medium 1 to 5 Installs 3005 5214 CD-ROM 6 to 10 Installs 3006 11 to 50 Installs 3007 51 or More Installs 3008
Withdrawal of Feature Numbers
The following order feature numbers for GPFS Version 1.1 are withdrawn as of November 3, 2000:
Machine Medium Order Type Type/ Feature Description Model Number Program Package 5765-B95 V1.1 4000 1 Use-Pack 4001 5 Use-Pack 4002 10 Use-Pack 4003 50 Use-Pack 4004 4-mm Tape for V1.1 6235 8-mm Tape for V1.1 6234
The following order feature numbers codes for GPFS Version 1.2 are withdrawn as of December 31, 2000:
Machine Medium Order Type Type/ Feature Description Model Number Program Package 5765-B95 V1.2 1001 1 Use-Pack 1002 5 Use-Pack 1003 10 Use-Pack 1004 50 Use-Pack 1005 Upgrade from V1.1 to V1.2 Program Package 5765-B95 1006 1 Use-Pack 1007 CD-ROM Media for V1.2 5204
AIX/UNIX Upgrade Protection Plan
Customers who purchased upgrade protection for GPFS Version 1.2 for AIX, under Software Upgrade Protection Plan for AIX and UNIX Operating System Based Software (5692-ADV) are entitled to receive GPFS Version 1.3 at no charge. Eligible customers should add the CD media supply feature number 0558 under their existing 5692-ADV record. GPFS Version 1.3, for AIX, is offered on CD-ROM media only.
5692-ADV Eligible Media Planned Billing Supply Availability Feature Feature for Upgrade Description OTC ALC CD-ROM Protection GPFS Version 1.3 for AIX (5765-B95) Program 4922* 4923* 0558 August 4, 2000 Package 1 Use-Pack 4924* 4925* 5 Use-Pack 4926* 4927* 10 Use-Pack 4928* 4929* 50 Use-Pack 4930* 4931*
Software Subscription for AIX (5692-SSO)
Software Subscription for AIX replaces the IBM Software Upgrade Protection Plan for AIX and UNIX Operating System Based Software (5692-ADV) for new and expired contracts.
Customers who purchased protection for GPFS Version 1.2 for AIX, under Software Subscription for AIX (5692-SSO) are entitled to receive Version 1.3 at no charge. Eligible customers should add CD media supply feature number 0558 under their existing 5692-SSO record. GPFS Version 1.3 for AIX is offered on CD-ROM media only.
5692-SSO Eligible Media Planned Billing Supply Availability Feature Feature for Upgrade Description ALC CD-ROM Protection From 5765-B95 0558 August 4, 2000 GPFS V1.2 to 5765-B95 GPFS V1.3 Program 4923*, 6312*, Package 6313* 1 Use-Pack 4925*, 6314*, 6315* 5 Use-Pack 4927*, 6316*, 6317* 10 Use-Pack 4929*, 6318*, 6319* 50 Use-Pack 4931*, 6320*, 6321*
To register for future upgrades of products from Software Subscription for AIX for programs described in this announcement, specify 5692-SSO and feature number 9001 for asset registration, and the applicable billing features with quantity equal to the number of licenses in the Software Subscription for AIX contract. Use the tables below as reference to select the appropriate billing feature.
Annual Order Support Type Billing Description Number Feature GPFS Version 1.3 5692-SSO for AIX (5765-B95) 1 to 5 Installs 6484 6 to 10 Installs 6485 11 to 50 Installs 6490 51 or More Installs 6491
Withdrawal of AIX/UNIX Upgrade Protection Feature Numbers
The following feature numbers will be withdrawn effective July 18, 2000.
Order type Machine Feature Description Type/Model Number Media Supply Feature 5692-ADV 0541 CD-ROM
Withdrawal of Software Subscription for AIX Feature Numbers
The following feature numbers will be withdrawn effective July 18, 2000.
Order Type Machine Feature Description Type/Model Numbers 5765-B95 GPFS V1.2 5692-SSO 4923, 6312, 6313, 4925, 6314, 6315, 4927, 6316, 6317, 4929, 6318, 6319, 4931, 6320, 6321
Customization Options for 5765-B95: Select the appropriate feature numbers to customize your order with delivery options desired. These features can be specified on the initial or MES orders.
Example: For future updates, specify feature number 3480 to ship media updates only. If, in the future, publication updates are required, order an MES to remove feature number 3480; then, the publications will ship with the next release of the program.
Feature Description Number Initial Shipments Serial Number Only (suppresses shipment 3444 of media and documentation) Ship Media Only (suppresses initial 3470 shipment of documentation) Ship Documentation Only (suppresses 3471 initial shipment of media) Update Shipments Ship Media Updates Only (suppresses 3480 update shipment of documentation) Ship Documentation Only (suppresses 3481 update shipment of media) Suppress Updates (suppresses update 3482 shipment of media and documentation) Feature Description Number Expedite Shipments Local IBM Office Expedite 3445 (for IBM use only) Customer Expedite Process Charge 3446 ($30 charge for each product)
Expedite shipments will be processed to receive 72-hour delivery from the time IBM Software Delivery and Fulfillment (SDF) receives the order. SDF will then ship the order via overnight air transportation.
Terms and Conditions
Contact your IBM representative for charges information for this announcement.