Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter enables high-performance, on demand computing

IBM United States Hardware Announcement 106-478
June 27, 2006


 ENUS106-478.PDF (46KB)

Table of contents   Document options  
TOC link At a glance TOC link Reference information
TOC link Overview TOC link Prices
TOC link Description TOC link IBM Global Financing
Printable version Printable version

At a glance

Cisco Systems 4X InfiniBand Switch Module features:

  • Up to eight external 4X (10 Gbps1) InfiniBand ports
  • 16 internal 4X (2.5 Gbps) InfiniBand ports
  • Up to 240 Gbps of aggregate bandwidth available per switch
  • Dual switch configurations to provide additional bandwidth, and redundancy

Cisco Systems 4X InfiniBand HCA Expansion Card (HCA) features:

  • Dual 10 Gbps ports
  • Server blade connection to one or two installed InfiniBand Switch Modules
  • Ethernet and Fibre Channel connectivity for each InfiniBand connected server blade, through external gateways

Other highlights:

  • Provides an ideal clustering solution for High Performance Computing (HPC), scaled-out distributed databases, and on demand computing
  • Leverages the low-latency, high-bandwidth characteristics and the Remote Direct Memory Access (RDMA) capabilities of the InfiniBand standard
  • Allows you to scale out on demand by interconnecting many blades and many BladeCenters with InfiniBand into a high-performance data center
  • Provides the ability to consolidate, virtualize, and share I/O and storage across an entire BladeCenter or a collection of BladeCenters for cost savings and high availability

Back topBack to top

The Cisco Systems 4X InfiniBand Switch Module is a new high-speed switch module for IBM BladeCenter® H. This new switch for IBM BladeCenter H delivers low-latency, high-bandwidth connectivity (up to 240 Gbps full duplex) between InfiniBand connected internal BladeCenter server blades, additional BladeCenter H chassis, stand-alone servers, and external gateways for connectivity to Ethernet LANs and Fibre Channel SANs.

For High Performance Computing (HPC) customers, InfiniBand is recognized as an industry-standard, high-performance cluster fabric. The Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter and the Cisco Systems 4X InfiniBand HCA Expansion Card for IBM BladeCenter together provide a complete low-latency, high-bandwidth interconnect solution for your scientific, technical, or financial BladeCenter cluster applications. Scaling from one to five BladeCenters can be done by just interconnecting the InfiniBand Switch Modules. For scaling to even larger clusters, while minimizing switch hops and maximizing bisection bandwidth, external Cisco InfiniBand switches can be used to create larger tree, mesh, or even full bisection bandwidth Clos network topologies.

Enterprise data centers need to address the growing demands for processing power from bandwidth-hungry and latency-sensitive applications. This need is often addressed by "scaling out" with clusters of smaller, less-expensive servers or blades as an alternative to the classic "scale-up" model with bigger servers. The scale out model offers economic and flexibility benefits, but historically has been hindered by two major issues: performance of legacy server interconnects, and "I/O sprawl" of adapters, cables, and switch ports required to connect every server directly with its LANs and SANs.

The BladeCenter along with the Cisco Systems 4X InfiniBand Switch Module and the Cisco Systems 4X InfiniBand HCA Expansion Card allows enterprise administrators to get the benefits of scale out while reducing the classic hindrances. IBM, using standards-based InfiniBand interconnect technology, delivers a solution that offers low latency and high bandwidth performance. Clustered blades can perform as well as a single monolithic server at a fraction of the cost.

You can centrally manage I/O and storage connectivity for a single BladeCenter, or an entire data center, without touching a cable. The Cisco solution allows one or more external I/O chassis to provide scalable LAN and SAN connectivity for an entire data center. Data center administrators can reduce the number of adapters, cables, and switch ports required, manage the addition, removal, or allocation of I/O or storage bandwidth centrally, and adjust I/O connectivity on demand usually without down time.

With the major obstacles to scale out alleviated, data center administrators gain an elegant and cost-effective solution for HPC, and distributed databases for an on demand computing environment.

For a copy of the applicable product warranties, contact your reseller or IBM2. IBM makes no representation or warranty regarding third-party products or services including those designated as ServerProven® or ClusterProven®.
Back topBack to top


Key prerequisites

eServer® BladeCenter chassis and processor blade

Planned availability date

August 11, 2006

High Performance Computing (HPC)

HPC applications are compute-intensive, and require a high-performance interconnect to make it possible to do fast and efficient computation across multiple discreet server blades.

Benefits include:

  • Node-to-node latency of under six microseconds is readily achievable using open standard protocols such as the popular Message Passing Interface (MPI). MPI is a key component of many HPC applications for message communication and data transfers between compute servers within a cluster.

    Low latency is critical to these applications. Less time is spent on communication overhead. More time is available for valuable data processing.

  • InfiniBand scalability reaches easily into the thousands of nodes.
  • High bandwidth communication is another of InfiniBand's strengths. The Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter delivers up to 80 Gbps of uplink bandwidth to build nonblocking BladeCenter clusters.

    Each switch module contains two 10 Gbps 4X ports and two 30 Gbps 12X ports.

    Each 12X port can also be used as three independent 10 Gbps 4X ports.

Distributed databases

Distributed databases such as IBM DB2® Parallel Edition and Oracle Real Application Clusters (RAC) can deliver scalability and high availability using inexpensive server platforms. This potential can go unrealized because of interconnect limitations such as high latency, low bandwidth, and high CPU overhead. BladeCenter configured with InfiniBand unlocks this potential.

Key attributes:

  • Faster interprocess communication: When a database is distributed across many servers, subprocesses frequently exchange messages to synchronize the database and share information. Often, a process must wait for a reply from another server before responding to a database query or update. Low latency reduces the time spent waiting for data.

    With BladeCenter configured with InfiniBand, the database application enjoys a significant throughput boost through a combination of low-latency messaging and high-bandwidth data transfer rates.

  • Increased database performance: Host CPU offload increases database performance by freeing each CPU from processing communication overhead. RDMA is a key ingredient in the CPU offload benefit. It allows a process on one server to directly access the memory on another server, once the transfer has been authenticated. The remote server's CPU does not intervene and suffers little or no performance penalty in handling the data transfer.
  • Storage aggregation: Databases are traditionally storage-hungry applications. An InfiniBand-equipped BladeCenter using external Fibre Channel gateway modules allows a seamless interface to Fibre Channel switches and storage targets.

    Administrators manage gateways centrally and all blades may share storage access through each gateway. Peak storage loads are addressed without deploying excessive hardware throughout the data center.

    Also, administrators save the cost of redundant HBAs and Fibre Channel switches while taking advantage of easy serviceability.

On demand computing

On demand computing addresses inefficiency and waste in the data center. You can avoid dedicating servers to part-time functions and leverage a common pool of shared, stateless, server resources that can be matched to I/O, storage, and applications on demand.

On demand computing allows data center managers to:

  • Share CPU and I/O resources
  • Improve high availability
  • Reduce costs and waste
  • Increase application performance

The Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter provides a programmable fabric for on demand computing.

The BladeCenter with InfiniBand solution also makes it possible to manage I/O and storage connectivity for an entire BladeCenter cluster in a central optimized fashion. Rather than requiring Ethernet and Fibre Channel off of each blade, or even off of each BladeCenter chassis, the BladeCenter with InfiniBand allows a single external I/O chassis to provide LAN and SAN connectivity to an entire data center. This allows the enterprise administrator to reduce the amount of adapters, cables, and switch ports required, manage the addition or removal of I/O or storage bandwidth centrally, and offer the ability to adjust on demand connectivity without any downtime required.

The BladeCenter on demand solution set is built around the following major building blocks:

  1. High performance server-to-server connectivity: The InfiniBand switch module provides the fabric for server virtualization. Central to the server switch is InfiniBand, which offers the extremely high bandwidth and low latency ideal for interconnecting servers.
  2. Virtualization and boot services: These capabilities allow blade servers to be deployed in a stateless fashion and enable you to provision any combination of application, operating system, storage, and I/O resources to a server on demand over the InfiniBand fabric.
  3. Seamless I/O and storage connectivity: BladeCenter is compatible with Cisco's external InfiniBand solutions set including Ethernet and Fibre Channel gateways and unique topology transparency software that enables the high-performance server fabric to seamlessly connect with existing LANs and SANs with little or no management or infrastructure changes.
  4. Virtualization software suite: BladeCenter is compatible with Cisco's VFrame software which embeds policy and provisioning intelligence into the fabric. This intelligence enables the fabric automatically to create virtual servers by interconnecting the right combination of resources at the right time, based on policies a data center manager defines.

    As conditions change, the switch can be programmed to add and remove resources from virtual servers as necessary.

With this flexibility, the Cisco solution for BladeCenter allows data center managers to easily deploy new applications, repurpose servers from one function to another on demand, and respond rapidly to server or network failures. The result helps to dramatically reduce downtime.

The external Cisco gateway hardware and virtualization software referenced above is available directly from Cisco Communications.

All offers are subject to availability. IBM reserves the right to alter product offerings and specifications at any time without notice. IBM is not responsible for photographic or typographic errors.

IBM makes no representation or warranty regarding third-party products or services.
Back topBack to top

Reference information
Gbps, GBps, and MBps denote the rated physical speed of the device, not the number of data bits or bytes transferred. Many factors affect actual data throughput.
For information on the IBM Statement of Limited Warranty, call 800-IBM-SERV (426-7378), or contact your IBM representative or reseller, or write to:

Warranty Information
P.O. Box 12195
RTP, NC 27709
Attn: Dept. JDJA/B203.

Copies are available upon request.


BladeCenter, ClusterProven, ServerProven, eServer, and DB2 are registered trademarks of International Business Machines Corporation in the United States or other countries or both.
Other company, product, and service names may be trademarks or service marks of others.

Cisco Systems 4X InfiniBand Switch Module for BladeCenter (32R1756)

Cisco Systems 4X InfiniBand HCA Expansion Card for BladeCenter (32R1760)

Technical information

Performance specifications: An enterprise InfiniBand switch for fabric integration management into BladeCenter architecture and associated options

Physical specifications

  • Switch
    • Approximate height: 20 mm (0.8 in)
    • Approximate width: 285 mm (11.2 in)
    • Approximate depth: 265 mm (10.5 in)
  • Daughter card
    • Approximate height: 2 mm (0.08 in)
    • Approximate width: 2 mm (0.08 in)

Operating environment

  • Temperature: 10° to 35°C (50° to 95°F)
  • Relative humidity: 8% to 80% noncondensing

Agency approvals

  • FCC part 15B Class A
  • CAN/CSA-C22.2 No. 60950-1-03
  • Canada ICES-003
  • VCCI Class A ITE (Japan)
  • C-Tick
  • CE Mark
  • CISPR 22 Class A
  • UL
  • cUL
  • TUV
  • A4EN55022 level A (EC)
  • Voltage fluctuations: EN61000-3-3
  • Harmonics: EN61000-3-2
  • Immunity: EN55024:1998

ENERGY STAR-compliant: No

Hardware requirements: The Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter is supported on the following IBM platforms.

System name                     type                   Model 
BladeCenter H                   8852                   All

The Cisco Systems 4X InfiniBand HCA Expansion Card for IBM BladeCenter is supported on the following BladeCenter blade servers.

System name                     type                   Model 
BladeCenter HS21                8853                   All

Software requirements: The Cisco Systems 4X InfiniBand HCA Expansion Card for IBM BladeCenter is supported on the following operating systems:

  • Linux™
    • Red Hat 4 Advanced Server
    • SUSE Linux Enterprise Server 9
  • Microsoft™
    • Windows™ Server 2000 (Standard, Enterprise, Web)
    • Windows Server 2003 (Standard, Enterprise, Web)

Compatibility: The product is designed to maintain compatibility with many IBM products. As a part of the ServerProven® program, xSeries® compatibility listings are updated on the Web. Detailed information relative to IBM and non-IBM vendor devices, adapters, software, and network operating systems supported with xSeries products can be accessed via the Web

Limitations: None

Planning information

Cable orders required: Yes

Cable                                                part
description                                          number
IB 3 meter 4x Cable for IBM BladeCenter              26R0813
IB 3 meter 12x to (3) 4x Cable for                   26R0816
 IBM BladeCenter
IB 8 meter 4x Cable for IBM BladeCenter              26R0847
IB 8 meter 12x to (3) 4x Cable for                   26R0849
 IBM BladeCenter

Installation time: Approximately 20 minutes

Packaging: One box

Shipment group — Switch Module

  • Cisco 4X InfiniBand Switch Module
  • Installation publication
  • License agreement
  • Safety flyer

Packaging dimensions/weight

  • Single box (HxDxW):
    • 431.80 mm (17 in)
    • 406.40 mm (16 in)
    • 127.00 mm (5 in)
  • Weight: 5 lb

Shipment group — Host Channel Adapter

  • Cisco 4x InfiniBand Host Channel Adapter Expansion Card
  • Installation publication
  • License agreement
  • Safety flyer

Packaging dimensions/weight

  • Single box (HxDxW):
    • 279.40 mm (11 in)
    • 381.00 mm (15 in)
    • 50.80 mm (2 in)
  • Weight: 3 lb

Security, auditability, and control

This product uses the security and auditability features of host hardware, host software, and application software.

Terms and conditions

Field installable: Yes

Customer setup: Yes

Warranty period: One year3

Service type (CCE/CCR/CRU): CRU4

Maintenance agreement: No

Optional upgrade: No

Licensed programs: Yes

Optional features initially installed in an IBM machine carry the same warranty classification as the machine. If installed after the initial machine installation, they carry the balance of the machine or the optional feature warranty, whichever is greater.
This optional feature is designed as Customer Replaceable Unit (CRU).

In the United States, call 800-IBM-SERV (426-7378), or write to:

Warranty Information
P.O. Box 12195
RTP, NC 27709
Attn: Dept. UF2A/B203

Copies of the IBM Statement of Limited Warranty can be obtained by contacting your reseller or accessing the online services listed in the Help and Service Information section of this announcement, or by calling IBM in the United States, at 800-426-7378. In Canada, call IBM at 800-565-3344.

Note: Programs included with this product are licensed under the terms and conditions of the License Agreements that are shipped with the feature or option.

All other terms and conditions are the same as those applicable to the IBM machine type in which the feature is installed.

International Warranty Service (IWS): IWS is available during the warranty period to customers who travel or relocate to countries where their computer is sold and serviced by IBM or IBM resellers authorized to perform warranty service. Eligible IBM computers are identified by their four-digit machine type.

You can obtain IWS through the method of service, such as CRU, depot, carry-in or on-site, provided in the servicing country. Service methods and procedures vary by country, and some service or parts may not be available in all countries. Service centers in certain countries may not able to service all models of a particular machine type. In addition, some countries may have fees and restrictions that apply at the time of service.

To determine the eligibility of your computer and to view a list of countries where service is available, visit

Note: Due to the earth's magnetic field, cathode ray tube (CRT) monitors are manufactured to work in northern, southern, and equatorial regions of the earth and may not produce a satisfactory image when moved between them. Any required adjustment (if possible) is not covered under IWS and may be subject to a chargeable action. The magnetic field does not affect flat panel LCD monitors.

Help and service information — Features and options

During the warranty period, support for a warranty-related issue can be obtained by accessing IBM's online services (requires internet access account, not included) or calling the IBM Support Center at 800-426-7378, 24 hours a day, 7 days a week, to answer any questions regarding your new IBM option. Response time will vary depending on the number and nature of calls received. If you need warranty service, return or exchange is available. In addition, if your IBM option is installed in an IBM computer, you may be entitled to service at your location. For general questions related to the use of your options, fee-based support is available. Your technical support representative can help you determine the best alternative.

Accessing IBM online services

IBM Web site at

Back topBack to top

The prices provided in this announcement are suggested retail prices for the U.S. only and are provided for your information only. Dealer prices may vary, and prices may also vary by country. Prices are subject to change without notice. For additional information and current prices, contact your local IBM representative.

                                    Part              list
Description                         number            price(5)
Cisco 4xIB Switch Module            32R1756           $11,499
 for IBM BladeCenter
Cisco 4xIB Host Channel             32R1760             1,399
 Adapter Expansion Card
 for IBM BladeCenter
List price does not include tax or shipping and is subject to change without notice. Reseller prices may vary.

Back topBack to top
IBM Global Financing

IBM Global Financing offers competitive financing to credit-qualified customers to assist them in acquiring IT solutions. Offerings include financing for IT acquisition, including hardware, software, and services, from both IBM and other manufacturers or vendors. Offerings (for all customer segments: small, medium, and large enterprise), rates, terms, and availability can vary by country. Contact your local IBM Global Financing organization or visit

IBM Global Financing offerings are provided through IBM Credit LLC in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government customers. Rates are based on a customer's credit rating, financing terms, offering type, equipment type, and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice.


BladeCenter, ServerProven, xSeries, and HelpWare are registered trademarks of International Business Machines Corporation in the United States or other countries or both.
Microsoft and Windows are trademarks of Microsoft Corporation.
Linux is a trademark of Linus Torvalds in the United States, other countries or both.
Other company, product, and service names may be trademarks or service marks of others.

Back to topBack to top

Printable version Printable version  

Share this page

Digg Linked In

Contact IBM

Considering a purchase?