IBM Support

Upgrades of FlashSystem 840/900 behind 8.1.x Virtualized products can result in outage

Troubleshooting


Problem

The 8.x code introduced Write Same UNMAP support. A bug was found that if a Write Same command is issued while a FlashSystem is being upgraded, the Flash I/O controller could interrupt the flow and cause an outage. This issue is fixed in 1.5.1.0 but systems with older firmware are exposed if they are behind a Spectrum Virtualize system on 8.x code level. (SVC, V7000, V9000)

Symptom

A symptom of this issue being encountered is to see a “Data Storage exception”.

Cause

This issue is due to an incompatibility in the Unmap/Write Same command handling and this request being encountered while an interface is doing its reconfiguration during the CCL process. The most likely feature in the control nodes to encounter this issue is Easy Tier. However, any host function that issues SCSI unmap commands could see the issue during an upgrade as well.

Environment

Spectrum Virtualized controllers with Easy Tier and FlashSystem 840/900 enclosures behind them are affected. This document provides a "proactive action plan to assist with upgrades in these environments. Other host-based environments may also encounter this issue if they are issuing the Unmap/Write Same command during the upgrade of the FlashSystem and steps should be taken to prevent these commands from be issued until the upgrade is completed.

Diagnosing The Problem

The latest upgrade test utility release 24.10 has been updated to detect if Easy Tier is active while the customer is on an 8.1.x.x release. When Easy Tier is active a warning will be issued and a link provided to the proactive action plan included below.

Resolving The Problem

To avoid this issue use the following matrix to determine what which actions should be followed for your specific scenario:

For SVC/V840 running older releases before 8.1.0.2:

  • Upgrade the flashsystem to 1.5.1.0 or higher *before* upgrading the SVC/AC nodes to avoid the issue entirely.
  • Upgrade the SVC/AC nodes to 8.1.1.1 or higher after the above flashsystem upgrade is complete.


For SVC, FlashSystem V840, and FlashSystem V9000 with AE3 enclosures running 8.1.0.2 or 8.1.1.0 or later.
  • If user desires to upgrade their system to current release of 1.5.1.0 and 8.1.1.1 they will need to follow the "Proactive Action Plan" below:

For the V9000 with AE2, the enclosure’s code will be incorporated into the new V9000 release 8.1.2.0 and above; as well as a future 8.1.1.2.PTF release

For V9000 with AE2, upgrading *to* 8.1.0.2, 8.1.1.0, 8.1.1.1 the unmap changes were not incorporated as yet, and the customers are able to upgrade using a standard CCL procedure.
  • If in the future they decide to upgrade above to 8.1.2.0 or the future 8.1.1.2 PTF higher releases, they will then need to follow "Proactive Action Plan" below:

For V9000 with AE2 *upgrading from* 8.1.0.2, 8.1.1.0, 8.1.1.1 to the new 8.1.2.0 or higher releases, follow the "Proactive Action Plan" below:


Proactive Action Plan
To be performed on Virtualization Cluster (ie; SVC, V840/V9000 AC cluster config node): 
  1. 1. Set EasyTier to monitor mode, wait 30 minute for any long running migrations to complete

    lsmdiskgrp will show the status of easytier
    chmdiskgrp -easytier measure mdisk_group_name

2. run lsmigrate and ensure there are no running migrates, if there are, do no continue until it completes

3. Avoid the use of the following actions while the upgrade is ran:

    rmmdisk
    rmvdisk
    rmvdiskcopy
    stopfcmap
    writing zeroes to compressed volumes
    mdisk mirro auto delete

4. Preform the upgrade as follows for your system environment:

    For SVC, V840, V9000 with AE3
        FlashSystem must be upgraded first to 1.5.1.0
        Control nodes must be upgraded to 8.1.1.1
    For V9000 with AE2 upgrade to 8.1.2.0 or when available, the future 8.1.1.2 PTF

5. Re-enable EasyTier to previous setting prior to start of this actionplan.

chmdiskgrp -easytier [auto/on] mdisk_group_name

[{"Product":{"code":"STKMQV","label":"IBM FlashSystem V9000"},"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Component":"Firmware","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"Version Independent","Edition":"N\/A","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"STKMQB","label":"IBM FlashSystem 900"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"STKMQB","label":"IBM FlashSystem 900"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"ST2HTZ","label":"IBM FlashSystem Software"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"STPVGU","label":"SAN Volume Controller"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
17 February 2023

UID

ssg1S1012163

Page Feedback