IJ44691: SPECTRUM SCALE ERASURE CODE EDITION BACK-END STORAGE MANAGEMENT DEGRADATION CAN CAUSE NODE EXPULSION

APAR status

• Closed as program error.

Error description

• Spectrum Scale Erasure code edition interacts with third party
  software/hardware APIs for internal disk enclosure management.
  If the management interface becomes degraded and starts to hang
  commands in the kernel, the hang may also block communication
  handling threads. This causes a node to fail to renew its lease,
  causing it to be fenced off from the rest of the cluster.
  This may lead to additional outages
  

Local fix

• The node with hardware problems will show waiters 'Until
  NSPDServer discovery completes.'
  It is recommended to reboot nodes with those GPFS waiters
  exceeding 2 minutes if this node is also being expelled.
  

Problem summary

• Spectrum Scale Erasure code edition interacts with third party
   software/hardware APIs for internal disk enclosure management.
  If the management interface becomes degraded and starts to hang
  commands in the kernel, the hang may also block communication
  handling threads. This causes a node to fail to renew its lease,
  causing it to be fenced off from the rest of the cluster.
  This may lead to additional outages.
  

Problem conclusion

• This problem is fixed in 5.1.2.9
  To see all Spectrum Scale APARs and their respective
  Fix solutions refer to page:
  https://public.dhe.ibm.com/storage/spectrumscale/spectrum_scale_
  apars.html
  
  Benefits of the solution:
  Break the dependency between back-end storage management
  and communication handling threads.
  Degradation of back-end storage management no longer causes
  node expels.
  
  Work Around:
  The node with hardware problems will show waiters 'Until
  NSPDServer discovery completes.'
  It is recommended to reboot nodes with those GPFS waiters
  exceeding 2 minutes if this node is also being expelled.
  
  Problem trigger:
  Degradation in back-end storage management that causes
  commands to hang in the kernel.
  
  Symptom:
  Hang/Deadlock/Unresponsiveness/Long Waiters
  
  Platforms affected:
  Linux Only
  
  Functional Area affected:
  ESS/GNR
  
  Customer Impact:
  High Importance
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SPEC SCALE STD

• Fixed component ID

  5737F33AP

Applicable component levels