PCIe3 LP CAPI Fibre Channel Flash Accelerator Adapter (FC EJ16)

Learn about the specifications and operating system requirements for the feature code (FC) EJ16 adapter.

Overview

FC EJ16, EJ17, and EJ18 are electronically identical, but have different tailstock brackets.

The PCIe3 LP CAPI Accelerator Adapter (FC EJ16) is a short, low-profile, PCI express (PCIe) generation 3 (Gen3) x8 adapter. This adapter acts as a co-processor for the POWER8® processor, and is designed to offload CPU access to external Fibre Channel flash storage.
Important: Graphics processing units (GPUs) and Coherent Accelerator Processor Interface (CAPI) adapters cannot be replaced with the power turned on.
The adapter provides the following features:
  • The adapter requires a direct-attach, point-to-point 8Gb Fibre Channel link to external storage.
  • The adapter includes a programmable CAPI Fibre Channel offload Accelerator Functional Unit (AFU).
  • The adapter requires one or more 4096-byte sector logical unit numbers (LUNs) to be provisioned. The adapter and its accompanying software libraries will not function with 512-byte sector LUNs.
  • The adapter requires both 3.3 V and 12 V power to be available in the PCIe slot.
  • The adapter must be plugged into a direct host-bus PCIe slot to function.
  • A maximum of one adapter per processor is supported in the system.
Figure 1. PCIe3 LP CAPI Fibre Channel Flash Accelerator Adapter
PCIe3 LP CAPI Fibre Channel Flash Accelerator Adapter

Software requirements

The software requirements for the FC EJ16 adapter follow:
  • The adapter requires that additional software packages for the IBM® Data Engine for NoSQL are installed for CAPI Flash exploitation.
  • Software packages and adapter microcode must be installed as a matched pair.
  • Software packages are available for download on the Fix Central website under EJ16 feature code.
  • Functionality of the adapter is dependent on operating system and software library versions.
  • Installation of mismatched adapter microcode and operating system or software libraries might result in warning messages in the Linux syslog during boot, such as Back Level AFU.
  • Linux kernel 3.x supports virtual LUN use cases. Linux kernel 4.2, or later is required to fully exploit CAPI Flash functionality such as fine-grained multipathing, multiple LUN support, and persistent storage.

Specifications

Item
Description
Adapter FRU number
00NK025 (Designed to comply with RoHS requirement)
I/O bus architecture
PCIe3 x16
Slot requirement
One PCIe x16 low-profile slot
Voltage
3.3 V and 12 V
Form factor
Short, low-profile
Maximum number

For details about slot priorities and placement rules, see PCIe adapter placement rules and slot priorities and select the system you are working on.

Operating system or partition requirements

The adapter is supported on the following versions of the operating systems, depending on the system you are using:
  • Linux
    • Linux Ubuntu 14.10, RHEL 7.2, or later.
      Note: To run the following commands, you must have Linux Kernel 4.2.x, or later.
    • For support details, see the Linux Alert website.

Create a host in IBM FlashSystem storage

Use this command to create a host in IBM FlashSystem storage by using Flash World Wide Port Names (WWPN).
Note: To run this command, you must have Linux Kernel 4.2.x, or later. For support details, see the Linux Alert website.
  1. To create a host in IBM FlashSystem® storage, complete the following steps:
    1. Log into the Linux system console as the root user.
    2. To list all adapters to determine the ACPI Flash World Wide Port Names (WWPN) for each present accelerator's ports, enter the following command:

      lspci |grep 'IBM Device 04cf'

      An output similar to the following example is displayed:

      • 0000:01:00.0 Processing accelerators: IBM Device 04cf [rev 01]
      • 0002:01:00.0 Processing accelerators: IBM Device 04cf [rev 01]
    3. From the output list, select an adapter using the identifiers in the left column. To read the World Wide Port Names (WWPN) for the first adapter listed, enter the following command:

      lspci -s 0000.01:00.0 -vv |grep -e V6

      An output similar to the following example is displayed:

      • lspci -s 0000:01:00.0 -vv |grep -e V5 -e V6
      • [V5] Vendor specific: 5005076069800230
      • [V6] Vendor specific: 5005076069800231
    4. Find the WWPNs in the output.
      Note: In the example above, the WWPNs of the adapter are: 5005076069800230 and 5005076069800231.
    5. Use the WWPNs to create a new host or hosts in the IBM FlashSystem GUI or command-line interface.

Control access to the Accelerator

Use this command to enable an account to control access to the accelerator.
Note: To run the commands in this section, you must have Linux Kernel 4.2.x, or later. For support details, see the Linux Alert website.
  1. To control access to the accelerator, complete the following step:
    Note: By default, installation of the IBM Data Engine for NoSQL software libraries creates a cxl user and adds Udev rules to restrict read and write access to the accelerator's volumes to members of the cxl system.
    1. To enable an account to read and write to the accelerator, add the account to the cxl by entering the following command:

      sudo usermod -a -G cxl userid

View the status of the Accelerator

Use this command to view the status of each accelerator adapter's logical unit numbers (LUN). The libraries of the IBM DataEngine for NoSQL software include scripts that display the status of the volumes mapped to the accelerator.
Note: To run the commands in this section, you must have Linux Kernel 4.2.x, or later. For support details, see the Linux Alert website.
  1. To view the status of each accelerator adapter's LUN, enter the following command:
    sudo /opt/ibm/capikv/bin/cxlfstatus
    Notes:
    • Volumes may be in either legacy or superpipe mode. The default mode is legacy.
    • Volumes must be in superpipe mode for exploitation by the CAPI Flash block or arkdb APIs.
    Table 1. Coherent Accelerator Interface (CXL) Flash device
    Device SCSI Block Mode LUN WWID
    sg9: 33:0:0:0 sdc legacy 60050768218b0818200000000400006e
    sg10: 33:1:0:0 sdd superpipe 60050768218b0818200000000600006f
    sg11: 34:0:0:0 sde superpipe 60050768218b08182000000007000070
    sg12: 34:1:0:0 sdf superpipe 60050768218b0818200000000300006d

    Table 1 shows two accelerators, each with two ports, and a single volume mapped to each port's WWPN. One volume is in legacy mode, and three volumes are in superpipe mode. The WWID for each volume in the IBM FlashSystem are displayed for convenience of administration. This matches the WWID shown in the IBM FlashSystem GUI or CLI (ibm@power8:~$ sudo /opt/ibm/capikv/bin/cxlfstatus)

Set the mode for a volume

Use this command to set the mode for a volume.
Note: To run the commands in this section, you must have Linux Kernel 4.2.x, or later. For support details, see the Linux Alert website.
  1. To set the mode for a volume, enter the following command:

    sudo /opt/ibm/capikv/bin/cxlfsetlunmode <LUN> <Mode>

    • Notes:

      An output similar to the following example is displayed:

        • ibm@power8:~$ /opt/ibm/capikv/bin/cxlfsetlunmode
        • 60050768218b0818200000000400006e 1
        • INFO: Adding LUN 60050768218b0818200000000400006e to Super IO table. SUCCESS
      • LUN modes might be 0 for legacy mode, or 1 for superpipe mode.
      • After a LUN is set to superpipe mode, all paths to that volume will also be set to superpipe mode.
      • As shown in Table 1, each volume might be in either legacy or superpipe mode.
Parent topic: PCIe adapters
Related information:
IBM Prerequisite website
Installing, removing, or replacing PCIe adapters
PCIe adapter placement rules and slot priorities
PCIe adapter information by feature type


Last updated: Thu, June 27, 2019