IBM has identified an issue relating to independent filesets supported in SONAS running with R1.3.0 or higher.The issue is related to the Access Control Lists (ACLs), causing the loss of access to files/directories and requiring restoration from a backup source. The loss of access occurs as ACL space is reclaimed through a ACL garbage collection background process that is initiated as the number of ACLs grows. This also occurs if the ACL is created outside of the independent filesets. If no backup is available, the issue may potentially cause data loss.
Problem Environment / Exposure:
SONAS 18.104.22.168 or higher system may be exposed to this issue in each of the following instances:
1) If the filesystem contains independent filesets.
2) If the filesystem has had independent filesets which have been deleted.
3) If you have used the GUI to create filesets, and you have not changed the default to choose dependent filesets instead.
4) You have not created filesets yet and wish to create independent filesets now.
PLEASE CALL IBM SUPPORT IMMEDIATELY IF YOU MEET ANY OF THE ABOVE CRITERIA.
1. This procedure requires root privileges to execute.
2. This data/output should be collected and provided to IBM support, if IBM support is required.
3. Steps (a) & (b) should be repeated for each filesystem configured on the system.
(a) To determine if your system may be affected, issue the following command on the management node.
echo desc | /usr/lpp/mmfs/bin/tsdbfs <filesystem-name> | grep inodeSpaceMask
System at Risk:
If the values reported for the inodeSpaceMask are non-zero the filesystem is at risk for this problem.
If so, proceed to next step (b) below.
Example (System at Risk):
# echo desc | /usr/lpp/mmfs/bin/tsdbfs pk1 | grep inodeSpaceMask
Note: In the above example, filesystem pk1 is at risk (independent filesets were created).
System NOT at Risk:
If the values reported for the inodeSpaceMask are all zero, then you are currently not at risk. The problem will be corrected in a future SONAS.
|SONAS PTF release and IBM strongly advises to not to create independent filesets until this future version is applied on the system. If the use of independent filesets is required prior to availability of the future SONAS PTF release, please contact IBM Support.|
Example (System NOT at Risk):
# echo desc | /usr/lpp/mmfs/bin/tsdbfs pk2 | grep inodeSpaceMask
Note: In the above example, filesystem pk2 is safe (no independent filesets were created).
(b) To determine if the ACL reclaim background job may have run, issue the following command on the management node.
echo inode 4| /usr/lpp/mmfs/bin/tsdbfs <filesystem-name>|grep -e fileSize -e currentMetadataRepl
If the value of nFullBlocks equals currentMetadataReplicas, then the ACL garbage collection background job will not run until at least another whole block of ACLs are created. This usually provides a significant margin of safety. You should still contact IBM support to further evaluate the risk factors and seek guidance as this could fill up depending on the usage pattern.
# echo inode 4| /usr/lpp/mmfs/bin/tsdbfs pk1|grep -e fileSize -e currentMetadataRepl
Note: In the above example, the ACL garbage collection background job has not been run yet (nFullBlocks equals currentMetadataReplicas). The files in filesystem pk1 with ACLs are at risk in future, though there is some margin of safety at this moment.
However, if the value of nFullBlocks is greater than currentMetadataReplicas, you may or may not have already hit the issue, losing access to some files. You should contact IBM support immediately to analyze further.
Problem Resolution / Prevention:
To avoid potential access / data loss, it is critical to prevent the ACL garbage collection process from getting triggered.
Customers are advised to contact IBM Support immediately, with the diagnostic data explained above, for guidance on determining and obtaining the appropriate fix.
Rate this page:
Copyright and trademark information
IBM, the IBM logo and ibm.com are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.