IBM Support

IBM Spectrum Scale (GPFS) 4.2.3 and 5.0 levels: concurrent mmap and read from the same files may result in undetected data corruption

Flashes (Alerts)


Abstract

IBM has identified an issue in IBM Spectrum Scale (GPFS) V4.2.3.15 through 4.2.3.16, and V5.0.3.0 through 5.0.3.2 levels, in which concurrent read and mmap operations on the same file by multiple processes or threads may result in undetected data corruption, in the form of a read operation returning incorrect data.

Content

IBM has identified an issue in IBM Spectrum Scale (GPFS) V4.2.3.15 through 4.2.3.16, and V5.0.3.0 through 5.0.3.2 levels, in which concurrent read and mmap operations on the same file by multiple processes or threads may result in the read operation returning incorrect data. Whether the problem will occur depends on the system workload and the timing of the mmap and read operations across the multiple processes or threads. In the cases where the problem does occur, the content of the user buffer once the read operation returns will not be fully updated at the time of the read. As a result, at least part of the buffer will remain the same as before the read operation was issued -- instead of the buffer containing only data read from the file.

The problem may occur only if the read operation takes place on the last file system (data) block of the file, and the mmap range can be anywhere in the file.

It is only possible to encounter the problem when a process/thread runs an mmap operation on a file while another thread/process is in the middle of a read operation on last block of the same file.
Users Affected: 

This issue affects customers running IBM Spectrum Scale (GPFS) V4.2.3.15 through 4.2.3.16, and V5.0.3.0 through 5.0.3.2 levels, when the following conditions are all met:

1) The same file is accessed both via read and mmap operations, by different processes/threads.

2) The read occurs in the last file system (data) block of the file. The read operation must start somewhere within the last block of the file and end at the end of the file

3) The mmap range can be anywhere in the file.


Run command
  • /usr/lpp/mmfs/bin/mmlsfs <FS name> -B

to find out the file system data block size for the given file system where the file resides.
 
Problem Determination: 

There are no external indicators that this problem may be occurring -- no error messages are produced to mmfs.log or syslog.  The only indication of the problem is a read operation returning unexpected data from a file stored in a GPFS file system. In particular, some or all the data returned in the read will corresponds to the content of the user buffer before the read operation was issued.
 
Recommendations:
Contact IBM service to obtain and apply an efix for your code level(s):

IBM Spectrum Scale V5.0.3.0 through 5.0.3.2, reference APAR IJ17825

IBM Spectrum Scale V4.2.3.15 through 4.2.3.16, reference APAR IJ17823.

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"STXKQY","label":"IBM Spectrum Scale"},"Component":"","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
26 September 2022

UID

ibm10960396