ACL loss issues when using Independent Filesets on Storwize V7000 Unified System

Content

Problem Environment / Exposure:

Storwize V7000 Unified systems, running V1.3.0.x, V1.3.1.0 and V1.3.1.1, may be exposed to this issue in each of the following instances:

1) If the filesystem contains independent filesets.
2) If the filesystem has had independent filesets which have been deleted.
3) If you have used the GUI to create filesets, and you have not changed the default to choose dependent filesets instead.
4) You have not created filesets yet and wish to create independent filesets now.

PLEASE CALL IBM SUPPORT IMMEDIATELY IF YOU MEET ANY OF THE ABOVE CRITERIA.


Problem Diagnosis:

Notes:
1. This procedure requires root privileges to execute.
2. This data/output should be collected and provided to IBM support, if IBM support is required.
3. Steps (a) & (b) should be repeated for each filesystem configured on the system.

(a) To determine if your system may be affected, issue the following command on a file module
echo desc | /usr/lpp/mmfs/bin/tsdbfs <filesystem-name> | grep inodeSpaceMask

System at Risk:
If the values reported for the inodeSpaceMask are non-zero the filesystem is at risk for this problem. If so, proceed to next step (b) below.
Example (System at Risk):
---------------------------------------------------------------
 # echo desc | /usr/lpp/mmfs/bin/tsdbfs pk1 | grep inodeSpaceMask

     inodeSpaceMask  0000000000000100
0000000000000000000000000000000000000000000000000000000100000000
---------------------------------------------------------------
Note: In above example, filesystem pk1 is at risk (independent filesets were created).

System NOT at Risk:
If the values reported for the inodeSpaceMask are all zero, then you are currently not at risk.
A preventive fix is available in V1.3.2. Customers are strongly advised to upgrade to V1.3.2 immediately and not create independent filesets until the upgrade.

Example (System NOT at Risk):
---------------------------------------------------------------
 # echo desc | /usr/lpp/mmfs/bin/tsdbfs pk2 | grep inodeSpaceMask

     inodeSpaceMask  0000000000000000
0000000000000000000000000000000000000000000000000000000000000000
---------------------------------------------------------------
Note: In this example, filesystem pk2 is safe (no independent filesets were created).

(b) To determine if the ACL garbage collector background job may have run, issue the following command on a file module.
echo inode 4| /usr/lpp/mmfs/bin/tsdbfs <filesystem-name>|grep -e fileSize -e currentMetadataRepl

If the value of nFullBlocks equals currentMetadataReplicas, then the ACL garbage collection background job will not run until at least another whole block of ACLs are created. This usually provides a significant margin of safety. You should still contact IBM support to further evaluate the risk factors and seek guidance as this could fill up depending on the usage pattern.
Example:
---------------------------------------------------------------
 # echo inode 4| /usr/lpp/mmfs/bin/tsdbfs pk1|grep -e fileSize -e currentMetadataRepl

fileSize=262144 nFullBlocks=2
currentMetadataReplicas=2 maxMetadataReplicas=2
---------------------------------------------------------------
Note: In above example, the ACL garbage collection background job has not been run yet (nFullBlocks equals currentMetadataReplicas). The files in filesystem pk1 with ACLs are at risk in future, though there is some margin of safety at this moment.

However, if the value of nFullBlocks is greater than currentMetadataReplicas, you may or may not have already hit the issue, losing access to some files. You should contact IBM support immediately to analyze further.


Problem Resolution / Prevention:

To avoid potential access / data loss, it is critical to prevent the ACL garbage collection process from being triggered. This fix is available in V1.3.2.0. Customers are strongly advised to upgrade at the earliest opportunity.

Customers are advised to contact IBM Support IMMEDIATELY, with the diagnostic data explained above, for guidance on determining whether the system has been already exposed to this issue.


Fix

This issue has been fixed in V1.3.2.0 and later versions of IBM Storwize V7000 Unified.