IBM Support

Readme and Release notes for release 3.4.0.8 General Parallel File System (GPFS) 3.4.0.8 GPFS-3.4.0.8-power-Linux Readme

Fix Readme


Abstract

xxx

Content

Readme file for: GPFS Readme header
Product/Component Release: 3.4.0.8
Update Name: GPFS-3.4.0.8-power-Linux
Fix ID: GPFS-3.4.0.8-power-Linux
Publication Date: 7 October 2011
Last modified date: 7 October 2011

Installation information

Download location

Below is a list of components, platforms, and file names that apply to this Readme file.

Fix Download for Linux

Product/Component Name: Platform: Fix:
General Parallel File System (GPFS) Linux 64-bit,pSeries RHEL
Linux 64-bit,pSeries SLES
GPFS-3.4.0.8-power-Linux

Prerequisites and co-requisites

None

Known issues

  • - Problem discovered in earlier GPFS releases

    During internal testing, a rare but potentially serious problem has been discovered in GPFS. Under certain conditions, a read from a cached block in the GPFS pagepool may return incorrect data which is not detected by GPFS. The issue is corrected in GPFS 3.3.0.5 (APAR IZ70396) and GPFS 3.2.1.19 (APAR IZ72671). All prior versions of GPFS are affected.

    The issue has been discovered during internal testing, where an MPI-IO application was employed to generate a synthetic workload. IBM is not aware of any occurrences of this issue in customer environments or under any other circumstances. Since the issue is specific to accessing cached data, it does not affect applications using DirectIO (the IO mechanism that bypasses file system cache, used primarily by databases, such as DB2® or Oracle).

    This issue is limited to the following conditions:

    1. The workload consists of a mixture of writes and reads, to file offsets that do not fall on the GPFS file system block boundaries;
    2. The IO pattern is a mixture of sequential and random accesses to the same set of blocks, with the random accesses occurring on offsets not aligned on the file system block boundaries; and
    3. The active set of data blocks is small enough to fit entirely in the GPFS pagepool.

    The issue is caused by a race between an application IO thread doing a read from a partially filled block (such a block may be created by an earlier write to an odd offset within the block), and a GPFS prefetch thread trying to convert the same block into a fully filled one, by reading in the missing data, in anticipation of a future full-block read. Due to insufficient synchronization between the two threads, the application reader thread may read data that had been partially overwritten with the content found at a different offset within the same block. The issue is transient in nature: the next read from the same location will return correct data. The issue is limited to a single node; other nodes reading from the same file would be unaffected.


  • - SLES 10 for POWER
    • /etc/init.d/running-kernel that ships with pre SLES10 SP1 kernel-source rpms contains a bug that results in the wrong set of files being copied to the kernel source tree. This bug will be fixed with SLES10 SP1. If the official fix is unavailable, the following change should also address the problem:

      --- running-kernel.orig 2006-10-06 14:54:36.000000000 -0500
      +++ /etc/init.d/running-kernel 2006-10-06 14:59:58.000000000 -0500
      @@ -53,6 +53,7 @@
      arm*|sa110) arch=arm ;;
      s390x) arch=s390 ;;
      parisc64) arch=parisc ;;
      + ppc64) arch=powerpc ;;
      esac
    • The gpfs.base 3.4.0-0 rpm must be installed using the rpm --nopre flag BEFORE any updates can be applied.


    • The GPFS required level of Korn Shell for SLES 10 support is version ksh-93r-12.16 and can be obtained at the following architecture-specific link.

Installation information

  • - Installing a GPFS update for Linux on Power Systems

    Follow the steps below to install the fix package:

    1. Unzip and extract the update package (< filename >.tar.gz file) with one of the following commands:

      gzip -d -c < filename >.tar.gz | tar -xvf -

      or

      tar -xzvf < filename >.tar.gz


    2. Verify the udpate's RPM images in the directory. Normally, the list of RPM images in this directory would be similar to one of the following:

      GPFS update
      gpfs.base.< update_version >.< OS >.ppc64.update.rpm
      gpfs.docs.< update_version >.noarch.rpm
      gpfs.gpl.< update_version >.noarch.rpm
      gpfs.msg.en_US.< update_version >.noarch.rpm


      GPFS update with GPL licensed kernel module
      gpfs.base.< update_version >.< OS >.ppc64.update.rpm
      gpfs.docs.< update_version >.noarch.rpm
      gpfs.gpl.< update_version >.noarch.gpl.rpm
      gpfs.msg.en_US.< update_version >.noarch.rpm


      where
      < update_version >
      specifies the version number of the update you downloaded, for example, 3.1.0-7 .

      < OS >
      specifies the Linux operating system, for example, sles8 (SUSE Linux Enterprise
      Server 8) or sles9 (used for both SUSE Linux Enterprise Server 9 and Red Hat Enterprise Linux 4).

      For specific filenames, check the Readme for the GPFS update by clicking the "View" link for the update on the Download tab.

    3. Follow the installation and migration instructions in your GPFS Concepts, Planning and Installation Guide.
  • - Upgrading GPFS nodes

    In the below instructions, node-by-node upgrade cannot be used to migrate from GPFS 3.2 to later releases. For example, upgrading from 3.2.x.x to 3.4.y.y requires complete cluster shutdown, upgrade install on all nodes and then cluster startup.

    Upgrading GPFS may be accomplished by either upgrading one node in the cluster at a time or by upgrading all nodes in the cluster at once. When upgrading GPFS one node at a time, the below steps are performed on each node in the cluster in a sequential manner. When upgrading the entire cluster at once, GPFS must be shutdown on all nodes in the cluster prior to upgrading.

    When upgrading nodes one at a time, you may need to plan the order of nodes to upgrade. Verify that stopping each particular machine does not cause quorum to be lost or that an NSD server might be the last server for some disks. Upgrade the quorum and manager nodes first. When upgrading the quorum nodes, upgrade the cluster manager last to avoid unnecessary cluster failover and election of new cluster managers.

    1. Prior to upgrading GPFS on a node, all applications that depend on GPFS (e.g. Oracle) must be stopped. Any GPFS file systems that are NFS exported must be unexported prior to unmounting GPFS file systems. If tracing was turned on, then tracing must be turned off before shutting down GPFS as well.
       
    2. Stop GPFS on the node. Verify that the GPFS daemon has terminated and that the kernel extensions have been unloaded (mmfsenv -u ). If the command mmfsenv -u reports that it cannot unload the kernel extensions because they are "busy", then the install can proceed, but the node must be rebooted after the install. By "busy" this means that some process has a "current directory" in some GPFS filesystem directory or has an open file descriptor. The freeware program lsof can identify the process and the process can then be killed. Retry mmfsenv -u and if that succeeds then a reboot of the node can be avoided.


    3. Upgrade GPFS using the RPM command as follows:

      GPFS update
      rpm -U gpfs.base-< update_version >.< OS >.ppc64.update.rpm
      rpm -U gpfs.docs-< update_version >.noarch.rpm
      rpm -U gpfs.gpl-< update_version >.noarch.rpm
      rpm -U gpfs.msg.en_US-< update_version >.noarch.rpm


      GPFS update with GPL licensed kernel module
      rpm -U gpfs.base-< update_version >.< OS >.ppc64.update.rpm
      rpm -U gpfs.docs-< update_version >.noarch.rpm
      rpm -U gpfs.gpl-< update_version >.noarch.gpl.rpm
      rpm -U gpfs.msg.en_US-< update_version >.noarch.rpm


    4. Check the GPFS FAQ to see if any additional images or patches are required for your Linux installation:

      General Parallel File System FAQs (GPFS FAQs)
       
    5. Recompile any GPFS portability layer modules you may have previously compiled. The recompilation and installation procedure is outlined in the following file:

      /usr/lpp/mmfs/src/README

Additional information

  • - Notices

    [January 24, 2011]

    A fix introduced in GPFS 3.3.0-11 and in GPFS 3.4.0-3 changed the returned buffer size for file attributes to include additional available information, affecting the TSM incremental backup process due to the selection criteria used by TSM. As a result of this buffer size change, TSM incremental backup will treat all previously backed up files as modified, causing the dsmc incremental backup process to initiate new backups of all previously backed up files. If the file system being backed up is HSM managed, this new backup can result in recall of all files which have been previously backed up. This effect is limited to files backed up using TSM incremental backup; there are no known effects on files backed up using either GPFS mmbackup or the TSM selective backup process.

    This issue is resolved in GPFS 3.3.0-12 (APAR IZ92779) and GPFS 3.4.0-4 (APAR IZ90535). Customers using the TSM Backup/Archive client to do incremental backup (via dsmc incremental command) should not apply GPFS 3.3.0-11 or GPFS 3.4.0-3, but should wait to apply GPFS 3.3.0-12 or GPFS 3.4.0-4. Any customer using TSM incremental backup and needing fixes in GPFS 3.3.0-11 or 3.4.0-3 should apply an ifix containing the corresponding APAR before executing dsmc incremental backup using these PTF levels, to avoid the additional file backup overhead, and (in the case of HSM-managed file systems) the potential for large scale recalls caused by the backup. Please contact IBM service to obtain the ifix, or to discuss your individual situation.

    [October 26, 2010]

    The restriction below is no longer in effect. GPFS file systems with file system format version less than 9.00 as reported by mmlsfs (V2.3 and older) can now be mounted on a GPFS V3.4 cluster safely.

    [July 30, 2010]

    Restrictions: File systems that currently have file system format version less than 9.00, as reported by mmlsfs (this format version corresponds to GPFS V2.3 and older) cannot be mounted on a GPFS V3.4 cluster. This restriction will be lifted in a future GPFS update.

    [April 1, 2010]

    During internal testing, a rare but potentially serious problem has been discovered in GPFS. Under certain conditions, a read from a cached block in the GPFS pagepool may return incorrect data which is not detected by GPFS. The issue is corrected in GPFS 3.3.0.5 (APAR IZ70396) and GPFS 3.2.1.19 (APAR IZ72671). All prior versions of GPFS are affected.

    Click here for details.

    [March 31, 2010]

    Support for SLES 10 kernel beyond 2.6.16.60-0.58.1 has changed. GPFS 3.3 requires GPFS 3.3.0-5 and GPFS 3.2 requires 3.2.1-18.

    [December 17, 2009]

    Support for GPFS 3.1 has only been extended for AIX and Linux on POWER systems. Service updates will be made available for other Linux platforms, but support is not being extended.

    [November 9, 2009]

    GPFS 3.3.0-1 does not correctly operate with file systems created with GPFS V2.2 (or older). Such file systems can be identified by running "mmlsfs all -u": if "no" is shown for any file system, this file system uses the old format, and the use of GPFS 3.3.0-1 is not possible. GPFS 3.3.0-2 corrects this issue.

    [November 7, 2008]

    GPFS 3.2.1.7 contained a change that impacts TSM HSM recall process of files with stub size >0 causing hangs during recalls. To avoid this problem, the configuration parameter dmapiDataEventRetry has to be set to 'no' via command 'mmchconfig dmapiDataEventRetry=no -i '.

    [September 11, 2008]

    The 3.2.1-5 maintenance level had a data integrity problem using the mmap feature to write or update files on Linux and AIX. The 3.2.1-6 maintenance level is the recommended upgrade path from versions 3.2.0-0 through 3.2.1-4.

  • - Package information

    The update images listed below and contained in the tar image with this README are maintenance packages for GPFS. The update images are a mix of standard RPM images that can be directly applied to your system.

    The update images require a prior level of GPFS. Thus, the usefulness of this update is limited to installations that already have the GPFS product. Contact your IBM representative if you desire to purchase a fully installable product that does not require a prior level of GPFS.

    After all RPMs are installed, you have successfully updated your GPFS product.

    Update to Version:

    3.4.0-8

    Update from Version:

    3.4.0-0 through 3.4.0-7

    Update (tar file) contents:

    README
    changelog
    gpfs.base-3.4.0-8.sles.ppc64.update.rpm
    gpfs.docs-3.4.0-8.noarch.rpm
    gpfs.gpl-3.4.0-8.noarch.rpm
    gpfs.msg.en_US-3.4.0-8.noarch.rpm

  • - Changelog for GPFS 3.4.x

    Unless specifically noted otherwise, this history of problems fixed for GPFS 3.4.x applies for all supported platforms.

    Problems fixed in GPFS 3.4.0.8 [October 06, 2011]

    • Fix a race condition between creating a fileset snapshot in a node and mounting file system in another node which the file system was internal mounted.
    • Fix gpfs.base RPM to prevent errors during install when install_initd is not available.
    • Errors in an off-line hierarchical storage manager will result in errors when trying to access the data from the on-line "stub" file. The on-line operation will fail with the return code from the off-line hierarchical storage manager. Deleting an on-line "stub" file, which has a snapshot that also references the off-line data, requires that the off-line data be recalled on-line so that it may be captured in the snapshot. If the off-line error is due to a permanent data loss, then all retries will also obtain the same error resulting in the user being unable to delete the on-line file. This change allows the on-line file to be deleted when there are permanent errors in the off-line storage. The snapshot file remains on-line, but access to the lost file data will result EIO errors to indicate the data loss.This fix requires the most recent version of IBM Tivoli Storage Manager for Space Management.
    • Fix an assertion caused by expanding a quota file fragment.
    • Fix code which can cause daemon assert when mmdelfs was immediately followedby mmcrfs command with same FS name.
    • Attempt to restore a missing shadow file before resorting to rebuilding it from query data in all cases. The former behavior was inconsistent with -t full or -t incremental.
    • Invalid assertion in SFSCreateMetaFile.
    • Workaround ksh issue causing spurious command failure in RHEL6 x86_64.
    • Fix code which can cause deadlock when mmdeldisk running with a truncate operation on the helper node.
    • Fixed mmdelsnapshot not allowning non root user to delete globalsnapshots.
    • Fix a double free problem when doing restripe.
    • Prevent quorum loss at the cluster manager when quorum nodes are being added or deleted in an environment where tie-breaker disks and persistent reserve are used.
    • mmsdrserv should not result in connection failure when multiple clients query the server simultaneously.
    • Fix NULL ptr exception in aioComplete.
    • In kSFSGetattr(), the check for pcache fileset needs to account for OpenFile being compact.
    • Don't print out "GPFS Deadman Switch timer has expired" in the system error log if there are no outstanding IOs.
    • OpLock token revoke can cause deadlock on cacheObjMutex.
    • Add debug statement in exec script to detail expire errors. Restore lost progress indication for every 30mins to output Backing up files. Limit error messages about policy nonzero return status to occur only for policy errors. In full backup plus query mode preserve the restored shadow file in .mmbackupShadow*.old. Repaired message formatting for TSM failed with RC... message. Repaired message formatting indicating Audit log missing info. Elide decorating double quotes from audit log lines while comparing file names to shadow file. This fixes most common problem with quotation marks in file names when usingthe TSM 6.3.0 client (enhanced now at least print the correct path). Change exit status from tsbackup33 when all errors have been corrected to be "4" instead of $worstTSMrc.
    • Take note of any "severe" class error message from dsmc commands and count them. If a severe error occurs, or if ANS1999E message occurs indicating processing of a file list was aborted, then keep old shadow file.
    • Fixed rare deadlock during token recovery.
    • Fix mmfsck to use correct calucation of max number disk addr allowed in inode to avoid false header corruption report.
    • Fix mmapplypolicy ... --choice-algorithm=fast.
    • Prevent a node from being added to the cluster a second time under a different name.
    • Fix code to enforce that FILESET type quota entries belong to FILESET_ROOT.
    • Fix an assertion during "mmlsfs -Q" and the file system recovery has failed.
    • Fix mmstartpolicy to accept all valid low disk space event names.
    • Fix for segfault in mmapplypolicy under low or no space conditions.
    • Avoid very unlikely chance of errors after failed mmcrsnapshot command.
    • free kernel buffer upon exit from kxWinOps.
    • Handle CNFS interface on vlan tagging devices.
    • Dynamicaly update callbacks that allow restricting to a particular nodeclass when the nodeclass's member nodes change.
    • Fix problem that file is unnecessarily recalled before it is deleted. Should not recall file if there is no more snapshots.
    • Avoid very rare asserts in fileset and snapshot delete commands.
    • Fix a race condition between creating a fileset snapshot in a node and mounting file system in another node which the file system was internal mounted.
    • Fixed the broken "sync" mechanism when kernel version is greater than 2.6.31.
    • Avoid problems accessing files in snapshots while running the mmunlinkfileset and mmdelsnapshot commands.
    • Fix invalid mmlsquota command syntax error causing signal 11.
    • Work around posix_unblock_lock() prototype changes in later linux kernel.
    • This update addresses the following APARs: IV03588 IV03589 IV05690 IV06271 IV06329 IV06333 IV06438 IV06830 IV07501 IV07717.

    Problems fixed in GPFS 3.4.0.7 [July 29, 2011]

    • Reference any user handle passed to the kernel using the correct access mode and security check.
    • Fix code which can lead to assertion in a very rare case.
    • Sovled rare race condition which may lead to a kernel crash for 64bit AIX boxes when starting GPFS and unloading GPFS kernel extension concurrently.
    • Fix deadlock involving fcntl locking operations that can occur on on Linux systems with 2.6.18 based kernels under memory pressure.
    • Fix a problem in libmmantras that was causing application core dump.
    • Fixed a problem found when reading alloc sum file.
    • Fixed mmexpelnode to avoid 'Failed to locate a working cluster manager' errors.
    • Fixed bad assertion in InodeLkObj::token_revoke.
    • Fixed remount code that caused mmmount to still show success after remount failed.
    • Fix for a rare race condition when GPFS startup and shutdown race each other, resulting in a spurious assert.
    • Fix to allow policy rules files with extremely long files, such as those that may be generated by a program or a script. A viable workaround would be to just insert a few newline characters to break long lines.
    • Command mmlsquota with option -d only allows one option among -u, -g, -j.
    • Fix 'unknown opcode 0x20' warning message in trace back of linux on x86_64.
    • Add protection to filesystem name in preUnmount event callback to prevent kernel panic.
    • Call putECred on exit from kxWinOps and kxSetTimes.
    • Fix to possible deadlock situation after a node which is the primaryNSD server fails while I/O requests are being handled by that NSD server. Threads may be waiting for "Change SG desc".
    • Correct disk usage problem in recent Linux kernels with certain LANG setting.
    • Fix long waiters caused by error handling in mmpmonNodeListRequest message handler.
    • Serialize creation of grace period thread.
    • Fix log code which can cause filesystem cleanup to get stuck, which could then prevent other GPFS commands from executing.
    • Remove extra IOs when closing a sequentially written file that is larger than the writebehindThreshold.
    • Fix rare kernel memory race condition doing AIX IO when the disk max_transfer setting is smaller than the GPFS blocksize.
    • Address an issue of mmrestoreconfig in mix version cluster.
    • Added new api to restripe a very big file in chunks.
    • Fix GPFS RDMA ibv_reg_mr error 12.
    • Fix segfault in tsapolicy due to destructor being called twice.
    • Fix code that can cause mmdefragfs to never finish. This could occur only when all disks are almost full.
    • Fix UTF8 encoding check routine to return correct path name length.
    • Based on number of clients that can connect simulteneosly, if correct value is set for socketMaxListenConnections, connection failure will not be observed by clients attempting to connect at the same time.
    • Fix to mmexpelnode to avoid indefinite blockage when a network partition is present and the target node is the cluster manager.
    • Fix EA limit calculation when metablock size is smaller than 64KB.
    • Fix mmapplypolicy in a mixed AIX/Linux cluster AND using the --sort-buffer-size option.
    • Fixed windows node assert using gpfsSetAllocFileSize on a datareplicated file system.
    • Fix mmrepquota to show human readable form instead of scientific notation for negative number.
    • Fixed a problem during quorum type change and when tiebreaker type is set to callback.
    • Permit mmbackup to analyze audit log even if "Failed" messages are not detected in STDOUT from dsmc selective command. Should help in rare cases where TSM runs out of space and exits rc=12 without emitting any of the usual failure messages.
    • Fix problem in enable fastea path when some other file system error happens at the same time, which interrupted enable fast ea operation.
    • Fix exception in GetReturnAddr following a cNFS grace period.
    • Fix large inDoubt left unclaimed problem after running chgrp command.
    • Add checking code to make sure sgID and inode number passed by dmapi handle are valid.
    • Prevent intermittent not-found errors when looking up snapshot pathnames while other snapshots are being deleted.
    • Change allocation code to prevent a deadlock that could occur when rebalance disks. This problem could only occur if there are large number of disk (over 32) and most of them 100% full.
    • Fix to a problem where when a file is opened with O_DIRECT, but there are other processes with the same file opened without this flag, the writes to the file do not get immediately updated on the disk.
    • Fix AIX crash or EINVAL error when calling gpfs_get_winattrs_path.
    • HoldDaemonSeg before referencing configuration flags in the segment.
    • Install this fix is you are using mmapplypolicy ... -i pathlist -g something ...
    • Eliminate possible FSSTRUCT error when accessing snapshots.
    • Restore quota file record size shortened by mistake in 3.4 and fix possible quota file checksum errors introduced by wrong record size.
    • This update addresses the following APARs: IV01082 IV01136 IV01138 IV01793 IV02033 IV02055 IV02253 IV02277 IZ99707.

    Problems fixed in GPFS 3.4.0.6 [June 01, 2011]

    • Fix the mmchpolicy code which caused daemon assert after fail with E_NOENT error.
    • Fix a quota manager assertion where it could be caching invalid quota file inode after restripe.
    • Simplify expel cmd execution when multiple nodes including clmgr is specified in the cmd.
    • When TSM mangles path names, always leave the old shadow database in place and indicate an error by returning 12. Suggest next mmbackup be run with -q to synchronise shadow file to TSM.
    • Fix assert "!ofP->destroyOnLastClose" or apparent hang under certain workloads with concurrent directory updates from multiple nodes.
    • Fix for a rare race condition involving simultaneous mounts of different file systems that could lead to conflicting tokens being issued.
    • mmtracectl --off will now reset all trace config variables.
    • replace mhAclStore assert when stale ACL data is discovered.
    • Made the Windows code more robust in validating various GPFS handles and data structures before making GPFS internal calls. An inflight shutdown can race and cause various GPFS data structures to go invalid.
    • Modify the reporting of various file system allocation units in terms of GPFS sub-blocks. This forces the Windows Explorer to round up the disk usage for files to the nearest sub-block, which is consistent with the actual allocation size reported for files.
    • Fix a long waiter problem where the quorum is lost while the update mtime handler is already waiting for node recovery.
    • Fix for ProbeClusterThread waiting indefinitely for outbound connect during startup (problem occurs mainly on Windows nodes).
    • Add functionality to suspend write operations on a filesystem.
    • tsdbfs: avoid displaying Inode's wide address fields on narrow disk address filesystems.
    • Disallow immutable flag setting on narrow da fs.
    • Performance improvement in mmdf.
    • Persistent Reserve fix for database clusters where the nsd are directly-attached with many paths.
    • Clean up exit code reporting throughout. Add date/time stamp on exit. If tssnapdir command fails, add output to explain failure. Modify count of failed files to include skipped files in audit log. Permit failure exit if count of errors does not match count of Failed and Skipped files in audit log. Skip over pathnames in audit log if mangled. Permit shadow file update to record progress if all the failed pathnames can be elided properly even if mangled ones were skipped.
    • Wait recovery to finish before deldisk.
    • Fix long mmstartup delay on AIX after mmshutdown --force.
    • Fix code that could cause rare filesystem panic during sync.
    • Windows maximum shared segment size was increased from 256 MB to 1 GB and the maximum TM memory limit was increased from 1 GB to 128 GB.
    • Avoid extremely rare assert during unmount.
    • Replace portmap with rpcbind on SLES11 for CNFS.
    • Disable expiration time setting on narrow da file system.
    • Speed up node failure recovery for configurations with large values for maxFilesToCache.
    • Fix the Windows codepaths to call gpfsMmap specifying a valid byte-range.
    • Get rid of spurious "approaching limit for the maximum number of inodes" when a new FS manager takes over.
    • Ensure sdr server is not running on old configuration servers after rebooted.
    • Corrrectly set the list of registered RPC programs after restart portmap.
    • Fix CNFS failover problem with SLES11 or later.
    • Fix a deadlock that may occur in the presence of node failures.
    • Fix a GPL build break on RHEL 4.X when LINUX_KERNEL_VERSION=2060900.
    • Fix the allocation code which caused signal 11 under certain error condition.
    • Speed up traceback dumping on Linux by saving the symbol table the first time it is read.
    • Improve the fairness of the outbound RPC message queue, so that unlucky reply threads will not be stuck indefinitely under heavy communications loads.
    • Rare race condition causing spurious ENOSPC error when creating files.
    • Change the GPFS code for handleing the conversion of file handle to dentry to ESTALE to NFSD since ESTALE this the valid return code for file not found.
    • Change the SIGTERM and SIGINT signal handler and the methods it invokes so that all methods are "signal-safe", invoking only library calls that are Posix, specified as signal-safe. This code is still responsible for removing temporary files that tsapolicy creates.
    • There was a race/synchronization problem in the relatively new gpBS (buck slip) protocol that was introduced into rel 3.4 on 2011/02/25. This was fixed by adding code to synchronize of new flag p_beg2_sent, so that we are sure every session is bracketed by a begin... fini sequence.
    • Fix thread traceback stopping early and not translating some addresses to symbols.
    • Performance improvement in snapshot copy-on-write.
    • Fixed an invalid condition check that asserts the deamon instead of doing a proper cleanup while a filesystem manager takeover happens during quota check or fsck operation.
    • Fix problem when writing replicated data with direct-IO, where GPFS recovery after a node failure could interfere with concurrent updates to the same file from other nodes.
    • Prevent GPFS from starting when registering the pagepool to infiniband fails.
    • Make multi-threaded execution procedure local static constructors and/or destructors thread safe by adding code to avoid multiple execution.
    • Improve handling of symbolic links in mmbackup.
    • Improve error handling in mmbackup.
    • Prevent a rare FSSTRUCT error accessing large directory in a snapshot from multi-threaded application on AIX.
    • Fix a memory leak when getting extended attributes of a file.
    • Performance improvement in snapshot copy-on-write.
    • On 64 bit big endian architectures with 64 bit (void*) type and 32 bit (unsigned int) type, an old incorrect ccListVector::append() method stores the uint in memory such that a subsequent vec[i] access method retrieves 0000 instead of the stored value. A first attempted fix did not work due to a bug in the optimizing compiler (did work at level -O0), so a final fix was made to ts/pc/fc/cclistvect.h on 2011/05/06 using an explicit union type instead of "funky" casting.
    • Cleanup mmnfsmonitor monitor process after remove CNFS node.
    • Found TSM command exit codes were being discarded by execscript due to missing escape slash before $rc.
    • When exec script returns nonzero and mmapplypolicy returns EBADF(9) just print message and allow cleanup code to discern whether failure is fatal or not.
    • Change variable name to badPathCnt as itrepresents path names mangled by TSM, not files that were "skipped" by TSM because they were busy. Eliminate use of lstat() in determining this and simply see if backupDir is a common root of the failed path. If badPathCntis nonzero, will have to fail the backup.
    • Upgraded the Windows build environment to WiX 3.5, which includes an updated DifxApp.
    • Fixed regression caused by token management performance improvements.
    • Re-bury the --choice-algorithm=fast, so that program behaviour defaults to the old "exact" with sort choice algorithm. Also, fix the choice-algorith option propagation bug.
    • This update addresses the following APARs: IZ98687 IZ98699 IZ98702 IZ98771 IZ98981 IZ99353 IZ99355 IZ99356.

    Problems fixed in GPFS 3.4.0.5 [April 07, 2011]

    • Avoid showing 'unknown' version in mmfslog.
    • Fix assignment causing unaligned access warning in Linux for IA64.
    • Fix logging recovery when data replication is enabled and metadata replication is not, or following a node failure.
    • Stop internal mount for lsquota command.
    • Change "mmwindisk initialize" to create the GPFS data partition with a 16MB alignment.
    • Allow a node which cannot externally mount a file system, due to page size restrictions, to still serve as file system manager.
    • Fix immutability flags being lost after a file is restored from TSM restore program.
    • Fix Signal 8 (divide by zero) error when processing a deleted inode for prefetch.
    • Write new filesystem device name to all disks to avoid the unwanted warning messages.
    • Added a post-install script in GPFS for Windows that runs "mmautoload postinstall".
    • Create gpfs init lock file on system startup. This ensures GPFS shutdown will be called during system shutdown on RHEL distributions.
    • Fix fileset metadata files with inode numbers outside the reserved range being handled as orphans.
    • Fix file system stopping with an assert on structure error related to an invalid number of disk pointers stored in the inode.
    • Fix quota file block expansion code resulting in an 'oldDiskAddrP == NULL' assert.
    • Allow user space attributes to be set by user commands.
    • Fix an inodeScan interface clean up error that could cause long waiters during unmount.
    • Fix panic in cxiStartIO when disk device drivers are configured for 1024 scatter gather lists.
    • Fix mmbackup failing to backup file with "NONE" in pathname.
    • Fix a fcntl retry message arriving and being processed before the original gpfs_v_lockctl operation completes. This could have resulted in the lock operation referencing the sleepElement after the message handler had freed it.
    • Rework the handling of options for mmapplypolicy to accept blank characters arguments.
    • Fix mmdeldisk allowing deleting a disk without moving existing data off the disk first. This would only have occured on file systems with metadata replication enabled (-m 2), strict allocation enforced (-K always; the default is "whenpossible"), when running mmdeldisk shortly after creating a new snapshot, and if the only disks remaining are in a single failure group.
    • Performance improvements to mmdelsnapshot.
    • Disallow the colon character in a filename during create/open from Windows. The Unix nodes can still create filename that have a colon. Such files can be accessed on Windows using their 8.3 names.
    • Fix excessive acquires on TcpConnTab mutex. Every five seconds, every receiver threads checks for broken connection timeout, when only one is needed.
    • Fix an bug in the log recovery code that was forcing a filesystem panic while offline fsck is in progress.
    • dump functions now dump regionsPerPass for each storage pool.
    • Fix fsck code so that it is able to recover data from a filesystem with a fatal root inode which is a candidate for delete.
    • Fix problem where mmdf would not allow specifying more than one of the -d, -m and -F options.
    • Fix when using EXTERNAL POOL EXEC 'script' rules with an external storage manager that supports "premigration", such as TSM/HSM, mmapplypolicy mistakenly invoked the rules' EXEC 'script' with the subcommand MIGRATE instead of PREMIGRATE for files that should be pre-migrated.
    • Fix for a bug where small synchronous writes to a block pre-allocated usinggpfs_prealloc may be lost.
    • Fix isStoragePoolIndexValid(poolIndex) asserts after a storage pool has been deleted.
    • This update addresses the following APARs: IZ94724 IZ95818 IZ95819 IZ95821 IZ95855 IZ96323 IZ96325.

    Problems fixed in GPFS 3.4.0.4 [February 17, 2011]

    • Update mmtracectl to have --format and --noformat flags which allow one greater control over whether to format traces.
    • Fix rare race between flushBuffer and mergeInode updating lastDataBlock.
    • Fix a problem when RDMA connecting between two clusters where the IB networks are not connected.
    • Fix an assertion at daemon startup on 32 bit linux nodes when prefetchThreads and worker1Threads configuration variables are set too high.
    • Fix a bug in mmchdisk caused by premature loop exiting if a metadta disk and a data disk are found.
    • Add a make parameter "LINUX_DISTRIBUTION" for non-standard Linux distributions. e.g. make LINUX_DISTRIBUTION=REDHAT_AS_LINUX Autoconfig.
    • Fix race between nfsWatchdog and delsnapshot.
    • Fix the allocation code which caused an assert on filesystem manager node after encounter I/O error.
    • Fix code in displaying of cmd "mmlsfileset dev -L -p". Let this command show all the fileset status.
    • Fix a bug in mmwindisk utility (called from mmdevdiscover, for instance) that could cause the program to fail when the Windows node has certain uncommon storage devices attached.
    • Fix cxiIsNFSLock erroneously returning FALSE for NFSv4 lockctl calls.
    • Detect and skip the continuously repeated lines (from pgalloc), enable the MaxLoopCheck_dumpBufQueue and set it's default value to be 2, so it will not impact performance greatly.
    • Fix allocation code causing delete disk to fail when deleting last disk of a failure group.
    • Fix copyInodeBlock to not try to copy data block if it is EOF.
    • Fix Windows code to handle, or enable access on, all Unix filesystem object such as a block device, fifo, socket, symlinks. All such objects will be listed as 0 byte files on Windows. No other operation such as read/write/delete/modify will be allowed from Windows.
    • Fix code which caused long waiters and daemon shutdown on leaseloss.
    • Fix sublock to disk sector conversion routines handling invalid disk addresses. This is done by returning E_INVAL back to the caller during fsck scan.
    • Fix a race condition which can cause daemon shutdown when adding/deleting disk from the file system and when the inode prefetch thread is hitting a tiny time window.
    • Improve performance of file system metadata scan phases of mmrestripefs.
    • Add gpfs_set_times() and gpfs_set_times_path() API.
    • Avoid assert when a snapshot is created immediately after deleting a large file.
    • Fix subblock size not keeping consistency among the cluster.
    • Fix mmrestripefs or mmdeldisk command, after being killed, background activity continueing for a significant time.
    • Fix code which could cause daemon assert and segmentation fault during filesystem takeover.
    • Include mount event disposition in dm_get_disp() call.
    • Fix invalid assert in fcntl lock token relinquish path.
    • Fix lost quota code when calling tslsquota -j with fully qualified device name specified.
    • Prevent GPFS from starting while certain admin commands are running.
    • Fix assertion in mmcheckquota when it encounters quota entries with invalid fileset ids.
    • GPFS for Windows now disables SMB2 on the node during installation.
    • Fix unnecessary work during mmrestripefs and mmdeldisk commands.
    • Fix quorum formation when the /var/mmfs/gen/BallotFile file is too small.
    • Fix a longwaiter problem caused by and infinite loop in mmdefragfs.
    • Fix kernel assert in gotVinfoLock when doing read/write mmap.
    • Fix a workload that continuously invokes operations that require exclusive inode locks, such as chmod or chown, could prevent mmrestripefs or mmdeldisk commands from making progress.
    • Fix rsh/rcp failing in mmcrcluster when the second interface is being used to create a node and the hostname interface does not have permission.
    • Fix connecting to server, when running mmsdrcli, in cluster with daemon and admin on two different interfaces.
    • Do not return windows attributes blindly for gpfs_fgetattrs() API.
    • Fix file creates and deletes being blocked for a long time after a node failure, especially in file systems with a large number of inodes.
    • Fix allocation code to prevent an assert that could occur while trying to delete/replace disk.
    • Fix the fsck block compare method to support variable data / metadata block size routines failing and resulting in asserts during the fsck compare operation.
    • Fix file system policy information obtained by mmsnmpagentd.
    • Fix mmsdrfs generation changed event in events exporter.
    • Ensure the scope of remaining configuration parameters is not changed as a result of a delete operation on a configuration parameter.
    • Always refresh session list that registered for mount. This avoids a mount failure scenario where the session could be deleted or added while another node is processing the mount event.
    • Improve GPFS admin commands initiated on a daemon down node, forcing the execution to run on an active node, in a large cluster.
    • Fix IcQueryDirectory implementation for FILE_ID_BOTH_DIR_INFORMATION and FILE_ID_FULL_DIR_INFORMATION to correctly assign the file ID field.
    • Fix performance when sequentially overwriting an existing file in contiguous sector-aligned pieces that are less than the block size, do not read or prefetch any data.
    • Add GPFS mount options nfsHashName and nonfsHashName. If nfsHashName is in effect, the NFS FH will include the hash value of the file name. This option will improve performance but is off by default. Turning it on might cause some ENOENT error with NFS.
    • Fix ACL garbage collector delete auto-generated Window SID mappings.
    • Fix race condition starting too many mmkprocs when using many mmapped files.
    • Change the allocation code to prevent looping when migrating blocks after a disk's failure group assignment, data type or storage pool has changed.
    • Use seqDiscardThreshhold for determining if a buffer stays in pagepool after it has been consumed by a reader, or after it has been flushed by a writebehind thread. This allows setting a large seqDiscardThreshhold to cache most files below that size. Setting a low writebehindThreshold will start flushing sequentially written files so that a large pagepool does not fill up with dirty buffers.
    • Improve performance of snapshot, copy-on-write in particular, immediately after the snapshot is created and a large number of files are updated from multiple nodes in the cluster.
    • Fix gpfsInodeCache slab (and cpu) usage high due to NFS anon dentry allocations.
    • Improve sync performance when many dirty buffers.
    • Fix race condistion when performing extensive AIO on a node.
    • Add padding variable to SGPoolDataStored structure to prevent data misalignment.
    • Update gpfs_fcntl.h copyright statement.
    • Fix code in initializing a dummy super block "shutdownSuperP", which may cause kernel crash in some of the Linux kernel versions.
    • This update addresses the following APARs: IZ90535 IZ90866 IZ92310 IZ92316 IZ92318 IZ92320 IZ92325 IZ92330 IZ92413 IZ92427 IZ92467.

    Problems fixed in GPFS 3.4.0.3 [January 06, 2011]

    • Fix assert happening in setCachedRecAddr when the cached disk address is NULL while the disk address read from disk is a real disk address. Modify the assert to allow this kind of change. And, update the cached address locally.
    • Fix a potential metadata allocation problem where wrong disk may be selected-occurs in selectRandom code branch when invoked in getRandomTargetDisk.
    • Fix repeated RDMA connection attempts due to IBV_EVENT_PORT_ERR.
    • Fix AIX crash caused by kxFreeAllSharedMemory.
    • Fix code, when value of the option "-p" for cmd mmcrfileset is invalid, returning misleading error message, "InvalidOption".
    • Fix mmbackup of fs with no changes sending files to TSM.
    • Fix mmbackup migration from straight TSM, older backup format (3.2), or recovery when shadow file not present would cause full backup.
    • Avoid very rare assert (BaseFilesetMetadataRecord) during restripe operations such as mmdeldisk.
    • Fix a deadlock between the tsdeldisk and the inode expansion.
    • Fix: An additional characteristic of pathnames with special characters present is they can cause TSM to exit with rc=4. Sometimes this was being mis-handled in mmbackup because the highest and "net status" error code from all the runs of TSM was not recorded/provided.
    • Permit TSM install to be in "bin64" for AIX and find needed config file (dsm.opt) there. Provide enhanced debugging in tsbackup33 using DEBUGmmbackup bits.
    • Add new functions to carefully split file list lines and notice if the split char is showing up in the file name as well.
    • Fix the allocation code which can cause a filesystem to panic with "Too manydisks are unavailable" when running out of disk space.
    • Fix a rare race condition where a file system manager failure during a disk status change could cause temporary loss of file system access.
    • Fix kernel assert when dmapi event generator is accessing null sgP pointer.
    • Fix mmcheckquota showing confusion error message when scanning large fs. This was due to insufficient memory.
    • Provide inode space expansion during directory entry creation if new inode number is out of range.
    • Fix an erroneous assert check in fsck cleanup path.
    • Fix rare deadlock that occurs if NSD server fails during mmcrfs cmd.
    • Fix mmapplypolicy having an internal error. Or, as a workaround, avoid using -i and -g and -N together.
    • Provide support for recognizing DSM_CONFIG env variable in mmbackup.
    • Fix mmimgbackup failing when rebooting another node. Keep tsapolicy exit code at 0 when recoverable or expected errors occur. Important for scripted usage of mmapplypolicy depending on mmapplypolicy exit code to be 0-even during multi-node operation, or when one or more "helper" nodes fail.
    • Fix FSErrValidate error in ACL GC while inode expansion is also running.
    • Provide a --block-size option for the mmlssnapshot and mmlsfileset commands.
    • Fix race condition between two remove threads removing same file.
    • Fix deleting a snapshot containing sparse files that could, in some rare cases, cause temporary loss of file system access.
    • Fix race condition between deferred deletions and policy file creation.
    • Fix assert "aceLength > 0" in tsgetacl for default ACL on a dir in a remote fs.
    • Fix a rare problem in mmrestorefs error code path- CHECK_CONTINUE_ON_ERROR was starting the next inode without advancing inode block buffer to next inode buffer pointer.
    • Fix asserts in fsck while trying to fix corrupt directories.
    • Fix deadlock preMount callback invokes mm commands.
    • memcmp now defined in rtnetlink.
    • Fix permitting FSET snap handles inappropriately modifying the fssnap->magic number prior to a tsfattr(GPFS_SYNC_FS).
    • Fix for quote error in the m4 command invocation when running mmapplypolicy.
    • Relocate mmbackup related temporary files from root to /.mmbackupCfg/.
    • Fix sort->$sort.
    • Permit recovery after non-fatal TSM error codes.
    • Improve GPFS mmstartup time & other GPFS commands in adminMode=allToAll cluster.
    • mmexpel --wait command aborts the wait in case of quorum loss.
    • Avoid structure error assert after mmdelsnapshot when cached files in other snapshots are accessed.
    • Fix buffer length calculation for dmapi user event returned by dm_get_events call.
    • Fix dm_handle_to_path so that it can look up the directory name by its own handle.
    • Fix rare assert in fsync code path.
    • Fix problem of disk going off line with error 733.
    • Fix mmrestoreconfig failing. This occurs when quota command returns E_NO_QMGR while file system is being closed but, has not completely closed yet.
    • Fix problem of disk going off line with error 735.
    • Fix return code of appendOnly file checking routine.
    • Improve mmdeldisk progress time.
    • Add additional exmaple files, tspgrep and tsprm, to samples/ilm.
    • Fix a remote cluster not always picking a local NSD server when readReplicaPolicy=local is set.
    • Fix a startup synchronization issue that prevented GPFS autoload from working on some Windows systems.
    • Fix to fputattrs to restore/clear xattrs and to fputattrs_withpathname to not overwrite pathname once copied into the kernel. Users of IBM Tivoli Storage Manager Backup Archive Client and GPFS storage pools with policy RESTORE rules for data placement should apply this fix before restoring data to the GPFS file system.
    • Improve performance of small writes (<32k) over NFS to a file opened with with O_DIRECT on the NFS client.
    • Fix Linux crashing in cxigetinodenum on lockd call.
    • Fixed mmchfs code path that is leaving filesystem mounted internally.
    • Add support for RHEL 4.X x86_64.
    • Fix mmlssnapshot with -d option not showing data and metadata usage.
    • mm commands now display conflicting messages when the command is waiting to run, due to a conflicting program running, instead of after it has finished.
    • mmapplypolicy now suppresses progress messages if a non-interactive device is being used.
    • Fix the fsck block compare operation that results in a SEGV due to buffer overrun.
    • Reduce message traffic when writing a file with NFS.
    • Fix synchronization of dmapi destroy event thread and dmapi event response thread.
    • Fix assert in dmapi event timeout handlers.
    • Fix forced unlink of a fileset (mmunlinkfileset -f) on Linux causing temporary loss of file system access if there were deleted files still open in the fileset at the time it was unlinked.
    • Use TRCBUFSIZE environment variable for trace buffer size and ensure it is not overwritten by config parameter.
    • Fix some 64 bit counters in GPFS SNMP.
    • Properly restore windows attributes.
    • Improve performance of mixed random read/write workloads on large files over NFS.
    • Fix an invalid conditional assert in the fsck orphan management code.
    • Fix mmrestoreconfig failing, with tsdefquotaon failed rc=245, if quota command returns E_NO_QMGR while file system is being closed, but has not completely closed yet.
    • Avoid asserts and deadlock by having mmlsfileset and mmlssnapshot commands wait while mmcrsnapshot command runs.
    • Fix logAssertFailed: rmr1 != rmr2 when using GPFS RDMA.
    • Fix enabling fs with diskea and overflow block feature.
    • Fix a startup synchronization issue that prevented GPFS autoload from working on some Windows systems.
    • Fix a SEGV problem in mmlsquota when printing user quotas for a fileset.
    • Fix a case in mmchfs which leaves the FS internally mounted for a small time period after mmchfs is over.
    • Initialize filesetNameP.
    • Fix deadlock issue for flushing mmapped files.
    • FILESET_CMD_PERM for mmchfileset on Windows has been temporarily disabled.
    • Fix a startup synchronization issue that prevented GPFS autoload from working on some Windows systems.
    • Fix Linux mmdelacl returning E_OPNOTSUPP for files in a "-k nfs4" fs.
    • Add useDIOXW configuration variable to avoid Direct IO token thrashing when using some IO requests that match the GPFS blocksize.
    • On SLES11, a privileged user is now not allowed to create a file in a remote filesystem even though root squashing is enabled because the DAC_OVERRIDE capability was specified in the credential.
    • Fix mmbackup recording failed files incorrectly.
    • Update unlinked fileset handling code to properly cull paths from a new (3.4)-style shadow file and sort by inode number into the updated list file. Exempts the unlinked fileset contents from being expired from TSM.
    • Fix duplicate session id returned by dm_create_session due to clock out of sync problem.
    • Speed up snapshot creation and unmount on systems with a large amount of dirty data in the cache.
    • Suppress implicit file time updates after an explicit set time operation was perform on the same handle-these semantics only apply to Windows systems.
    • Allow dmapi clients to acquire access rights to a file that is being destroyed.
    • Add two new API's which can be used to improve performance for applications that make many API calls on Linux.
    • Tweak TSM Query code to recover files more accurately when doing shadow file reconstruction, including file names with break chars in the name.
    • Improve performance of random updates to a large file from a single node, if that file was previously accessed by another node.
    • Fix an assert during multiple instances of restripe running in parallel.
    • Fix a surplus indirect block not being processed during restripe.
    • Fix that could cause some files to become unreadable when running mmrestripefs on a system with small page pool or a workload that causes high demand for page pool buffers.
    • Speed up the reclaim of unused GPFS inodes on Linux.
    • Do not allow GPFS internal extended attributes to be set using the gpfs_fputattrs API. Require root authority to set DMAPI external attributes or external namespace attributes other than "user."
    • Avoid problems preventing upgrading v3.3 filesystems with snapshots.
    • Fix problem of restoring EA of pre 3.4 file system without fastea enabled.
    • This update addresses the following APARs: IZ86550 IZ87150 IZ87153 IZ88685 IZ88701 IZ88704 IZ88745 IZ88747 IZ88748 IZ88749 IZ88752 IZ88828 IZ88900 IZ88904.

    Problems fixed in GPFS 3.4.0.2 [October 07, 2010]

    • FSCK checks log file inodes even if they have log group number set to -1.
    • gpfsInodeCache slab (and cpu) usage high due to NFS anon dentry allocations.
    • Fix rare occurrence of file fragment expansion happening during file sync that can cause the assert failure related to GETSUBBLOCKSPERFILEBLOC.
    • If node cannot do cNFS recovery for a failed node then commit suicide so another node can do the takeover for both nodes.
    • Prevent FGDL kernel memory fault caused by very narrow race condition during directory lookup.
    • Fix assert related to RCTX.REPLIED, TSCOMM.C that occurs on the FS manager node if the FS manager is running GPFS release 3.2, and a release 3.3 client tries to mount the filesystem.
    • Linux IO: check mm_struct before pinning pages.
    • Improve performance of stat operations on Linux under certain multi-node access patterns.
    • Fixed an incompatibility between GPFS for Windows and the Interop Systems software bundles. This incompatibility caused Interop Systems bundle installation failure.
    • Fix hang between node join thread and events exporter request handler thread.
    • This update addresses the following APARs: IZ81230 IZ83798 IZ84008 IZ84016 IZ84039 IZ84041 IZ84045 IZ84145 IZ84161.

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSFKCN","label":"General Parallel File System"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
25 June 2021

UID

isg400000778