Dynamic Tracking of Fibre Channel devices

AIX® supports dynamic tracking of Fibre Channel (FC) devices.

AIX earlier versions required a user to unconfigure FC storage device and adapter device instances before you modify the system area network (SAN) settings that might result in an N_Port ID (SCSI ID) change of any remote storage ports.

If dynamic tracking of FC devices is enabled, the FC adapter driver detects when the Fibre Channel N_Port ID of a device changes. The FC adapter driver then reroutes traffic that is destined for that device to the new address while the devices are still online. Events that can cause an N_Port ID to change include one of the following scenarios:
  • Moving a cable between a switch and storage device from one switch port to another.
  • Connecting two separate switches by using an inter-switch link (ISL).
  • Rebooting a switch.
Dynamic tracking of FC devices is controlled by a new fscsi device attribute, dyntrk. The default setting for this attribute is no. To enable dynamic tracking of FC devices, set this attribute to dyntrk=yes, as shown in the example.
chdev -l fscsi0 -a dyntrk=yes  
In this example, the fscsi device instance is fscsi0. Dynamic tracking logic is called when the adapter driver receives an indication from the switch that there is a link event with a remote storage device port.
Dynamic tracking support requires the following configuration:
  • A switched environment. It is not supported in arbitrated loop environments, including public loop.
  • FC 6227 adapter firmware, level 3.22A 1 or higher.
  • FC 6228 adapter firmware, level 3.82A 1 or higher.
  • FC 6239 adapter firmware, all levels.
  • All subsequent FC adapter releases support Fast I/O Failure.
  • The worldwide Name (Port Name) and Node Names devices must remain constant, and the worldwide Name device must be unique. Changing the worldwide Name or Node Name of an available or online device can result in I/O failures. In addition, each FC storage device instance must have world_wide_name and node_name attributes. Updated filesets that contain the sn_location attribute (see the following bullet) must also be updated to contain both of these attributes.
  • The storage device must provide a reliable method to extract a unique serial number for each LUN. The AIX FC device drivers do not automatically detect the location of the serial number. The method for serial number extraction must be provided by the storage vendor to support dynamic tracking for the specific devices. This information is conveyed to the drivers by using the sn_location ODM attribute for each storage device. If the disk or tape driver detects that the sn_location ODM attribute is missing, an error log of type INFO is generated and dynamic tracking is not enabled.
    Note: When the lsattr command is run on a hdisk, the sn_location attribute might not be displayed. That is, the attribute name is not shown even though it is present in ODM.
  • The FC device drivers can track devices on a SAN fabric, if the N_Port IDs on the fabric stabilize within 15 seconds. The SAN fabric is a fabric as seen from a single host bus adapter. If cables are not reseated or N_Port IDs continue to change after the initial 15 seconds, I/O failures occur.
  • Devices are not tracked across host bus adapters. Devices are tracked if they remain visible from the same HBA that they are originally connected to.

    For example, if device A is moved from one location to another on fabric A that is attached to host bus adapter A (in other words, its N_Port on fabric A changes), the device is tracked without any user intervention, and I/O to this device can continue.

    However, if a device A is visible from HBA A but not from HBA B, and device A is moved from the fabric that is attached to HBA A to the fabric attached to HBA B, device A is not accessible on fabric A nor on fabric B. User intervention would be required to make it available on fabric B by running the cfgmgr command. The AIX device instance on fabric A is not usable, and a device instance on fabric B must be created. This device must be added manually to volume groups, multipath device instances, and so on. This procedure is similar to removing a device from fabric A and adding a device to fabric B.

  • No dynamic tracking can be performed for FC dump devices while an AIX system memory dump is in progress. In addition, dynamic tracking is not supported during system restart or by running the cfgmgr command. SAN changes cannot be made while any of these operations are in progress.
  • After devices are tracked, ODM might contain stale information. This is because Small Computer System Interface (SCSI) IDs in ODM no longer reflect actual SCSI IDs on the SAN. ODM remains in this state until cfgmgr command is run manually or the system is rebooted (provided all drivers, including any third-party FC SCSI target drivers, supports dynamic tracking). If cfgmgr command is run manually, it must be run on all affected fscsi devices. This can be accomplished by running cfgmgr without any options, or by running cfgmgr on each fscsi device individually.
    Note: Running thecfgmgr at run time to recalibrate the SCSI IDs might not update the SCSI ID in ODM for a storage device. This is true when the storage device is opened that is when the volume groups are varied on. The cfgmgr command must be run on devices that are not opened or the system must be rebooted to recalibrate the SCSI IDs. Stale SCSI IDs in ODM have no adverse effect on the FC drivers, and recalibration of SCSI IDs in ODM is not necessary for the FC drivers to function properly. Any applications that communicate with the adapter driver directly by using the ioctl calls and that use the SCSI ID values from ODM, however, must be updated (see the next bullet) to avoid the use of stale SCSI IDs.
  • All applications and kernel extensions that communicate with the FC adapter driver, either through ioctl calls or directly to the FC driver's entry points, must support the version 1 ioctl and scsi_buf APIs of the FC adapter driver to work properly with FC dynamic tracking. Noncompliant applications or kernel extensions might not function properly or might even fail after a dynamic tracking event. If the FC adapter driver detects an application or kernel extension that is not adhering to the new version 1 ioctl and scsi_buf API, an error log of type INFO is generated and dynamic tracking is not enabled for the device that this application or kernel extension is trying to communicate.

    For ISVs developing kernel extensions or applications that communicate with the AIX Fibre Channel Driver stack, refer to the Required FCP, iSCSI, and Virtual SCSI Client Adapter Device Driver ioctl Commands and Understanding the scsi_buf Structure for changes necessary to support dynamic tracking.

  • Even with dynamic tracking enabled, users must do the SAN changes, such as moving or swapping cables and establishing ISL links, during maintenance windows. Making SAN changes during full production runs is discouraged because the interval of time to run any SAN changes is too short. Cables that are not reseated correctly, for example, results in I/O failure. Running these operations during periods of little or no traffic minimizes the impact of I/O failures.
  • The dynamic tracking on AIX partitions enables software that has the express purpose of recovering configuration events from the SAN. The configuration events are expected during LPAR mobility operations. The AIX policy is used to avoid a service outage when preparing for a mobility event or during a mobility event. Therefore, dynamic tracking is always enabled for virtual FC client adapters and cannot be disabled.

The base AIX FC SCSI Disk and FC SCSI Tape and FastT device drivers support dynamic tracking. The IBM® ESS, EMC Symmetrix, and HDS storage devices support dynamic tracking, if the vendor provides the ODM filesets with the necessary sn_location and node_name attributes. Contact the storage vendor to know if the current level of ODM fileset supports dynamic tracking.

If vendor-specific ODM entries are not being used for the storage device, but the ESS, Symmetrix, or HDS storage subsystem is configured with the MPIO Other FC SCSI Disk message, dynamic tracking is supported for the devices with this configuration. This method replaces the need for the sn_location attribute. All current AIX Path Control Modules (PCM) shipped with the AIX base support dynamic tracking.

The STK tape device using the standard AIX device driver also supports dynamic tracking provided the STK fileset contains the necessary sn_location and node_name attributes.
Note: SAN changes involving tape devices must be made with no active I/O. Because of the serial nature of tape devices, a single I/O failure can cause an application to fail, including tape backups.

Devices that are configured with the Other FC SCSI Disk or Other FC SCSI Tape messages do not support dynamic tracking.

Some FC devices support enablement and disablement of dynamic tracking while the device is in the Available state. To verify whether the device supports the dynamic tracking, use the lsattr command. Dynamic tracking can be changed for the supporting devices without unconfiguration and reconfiguration of the device or cycling the link. The changes must be requested when the storage area network (SAN) fabric is stable. A request fails if the error recovery is active in the SAN during the time of the request. Change requests can fail if the associated devices such as disks and tape devices are not able to withstand the changes.