Battery Maintenance Cycle May Cause Loss of Access to Data

Content

Every 90 days the batteries in the Storwize V7000 Control Enclosure perform a maintenance discharge cycle. This occurs automatically without any expected impact to the system.

In some cases, however, batteries that were shipped from March 2013 onwards, which are identified by FRU identity 11S85Y6127, may temporarily misreport their amount of available charge shortly after completing a maintenance cycle if operating in an environment in which the battery temperature is below 20^oC.

svcinfo lsenclosurebattery -battery 1 <enclosure_id>

svcinfo lsenclosurebattery -battery 2 <enclosure_id>

The FRU identity can be determined by examining the value of the FRU_identity field in the resulting output.

In the following scenario, this misreporting of available charge can lead to a temporary loss of access to data:

1. Battery A starts and completes a maintenance cycle
2. Battery B starts a maintenance cycle as soon as battery A's cycle has completed
3. While battery B is unavailable due to its maintenance cycle, battery A temporarily misreports its amount of available charge, causing the system to conclude that neither battery has sufficient charge to accommodate a graceful shutdown should an input power loss occur. In order to protect against the risk of cache data loss if input power is lost, the system will therefore gracefully shutdown until at least one battery is deemed to have sufficient charge.
   
   N.B. The same fault could also occur on battery B after it has completed its maintenance cycle, however in this scenario battery A will be available to keep the control enclosure online and prevent a loss of access to data.

Fix

This issue was resolved by APAR HU00013 in the V6.4.1.6 and V7.1.0.5 PTF releases.

Once an affected system is running these levels or higher, the utility will no longer be required.

Workaround

A utility package has been created that will analyse the current state of the maintenance discharge cycle and introduce a delay between the maintenance cycles of battery A and battery B. This will prevent the loss of access to data from occurring in the case that either of the batteries experiences the temporary misreporting fault.

It is strongly recommended that all customers with batteries with FRU identity 11S85Y6127 in their system that cannot immediately upgrade to a level containing the fix run the utility.

Notes:

• This utility can be installed and run concurrently with no impact to I/O operations.
• If the system shuts down on battery multiple times due to input power loss, the maintenance discharge cycles of battery A and battery B may become re-synchronised and the utility will need to be re-run.
• Once run, the utility does not need to be re-run every 90 days.
• It is also perfectly safe to run the utility on systems containing other battery FRU identities, with no adverse side effects.

Utility Package Download

The utility package can be download from IBM Fix Central using the following URL:

Utility Package Installation

The utility package can only be installed and run on a Storwize V7000 running V6.4.1 or V7.1. This correlates to V7000 Unified customers running V1.4.0 or V1.4.1

Installation of this utility package will not cause any nodes to restart.

Note: V7000 Unified customers must log onto the V7000 Block Storage device GUI to complete these steps

1. Start the Launch Upgrade Wizard from the GUI.
   The wizard can be found under Settings > General >Upgrade Software
2. The GUI will prompt you to upload the upgrade test utility. At this prompt you can upload the battery utility instead of the upgrade test utility
3. You may be prompted with a message informing you that the uploaded file is not a valid upgrade test utility. If so, press OK to install the utility anyway
4. When the wizard asks what version you are about to upgrade to, press the cancel button to abort the process.

Alternatively the utility package can be uploaded to the /upgrade directory of the system using scp and installed using the applysoftware CLI command

Utility Package Operation

Note: V7000 Unified customers must log onto the V7000 Block Storage device Command Line to complete these steps

1. Use SSH or PuTTY to connect to the Command Line interface of the system. Details about how to connect to the command line interface can be found in the publications.
2. From the SSH or Putty session, issue the following command:
   
   battery_md_oos
   
   
3. The command will return a summary of all control enclosures and the actions taken for that control enclosure. Please review the output carefully, as the utility may ask you to run the utility again in a week.
   
   Example output:
   

   IBM_2076:V7000system:admin>battery_md_oos
    ---------- + -----------------------------------------------------------------
    Control    | Battery                                                          
    Enclosure  | Status                                                           
    ---------- + -----------------------------------------------------------------
    1          | It has been less than one day since the batteries last performed 
               | a maintenance cycle. Please re-run this utility in 
               | approximately 1 week 
    ---------- + -----------------------------------------------------------------
    5          | Battery maintenance cycles are already out of sync.  No action
               | needed
    ---------- + -----------------------------------------------------------------
   IBM_2076:V7000system:admin> 

   
   
4. If required the utility will commence a maintenance cycle on up to one of the batteries per control enclosure. This process can take up to 12 hours to complete.
5. The utility package can be run again once the maintenance cycle has completed to check that the batteries are out of sync. There is an optional -fake flag that can be specified to prevent the utility requesting additional maintenance cycles.
   Note: Until the maintenance discharge cycle has completed, the utility will still report a maintenance cycle is required.