Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter enables high-performance, on demand computingIBM United States Hardware Announcement 106-478
June 27, 2006
|Table of contents||Document options|
|At a glance|
Cisco Systems 4X InfiniBand Switch Module features:
- Up to eight external 4X (10 Gbps1) InfiniBand ports
- 16 internal 4X (2.5 Gbps) InfiniBand ports
- Up to 240 Gbps of aggregate bandwidth available per switch
- Dual switch configurations to provide additional bandwidth, and redundancy
Cisco Systems 4X InfiniBand HCA Expansion Card (HCA) features:
- Dual 10 Gbps ports
- Server blade connection to one or two installed InfiniBand Switch Modules
- Ethernet and Fibre Channel connectivity for each InfiniBand connected server blade, through external gateways
- Provides an ideal clustering solution for High Performance Computing (HPC), scaled-out distributed databases, and on demand computing
- Leverages the low-latency, high-bandwidth characteristics and the Remote Direct Memory Access (RDMA) capabilities of the InfiniBand standard
- Allows you to scale out on demand by interconnecting many blades and many BladeCenters with InfiniBand into a high-performance data center
- Provides the ability to consolidate, virtualize, and share I/O and storage across an entire BladeCenter or a collection of BladeCenters for cost savings and high availability
Back to top
The Cisco Systems 4X InfiniBand Switch Module is a new high-speed switch module for IBM BladeCenter® H. This new switch for IBM BladeCenter H delivers low-latency, high-bandwidth connectivity (up to 240 Gbps full duplex) between InfiniBand connected internal BladeCenter server blades, additional BladeCenter H chassis, stand-alone servers, and external gateways for connectivity to Ethernet LANs and Fibre Channel SANs.
For High Performance Computing (HPC) customers, InfiniBand is recognized as an industry-standard, high-performance cluster fabric. The Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter and the Cisco Systems 4X InfiniBand HCA Expansion Card for IBM BladeCenter together provide a complete low-latency, high-bandwidth interconnect solution for your scientific, technical, or financial BladeCenter cluster applications. Scaling from one to five BladeCenters can be done by just interconnecting the InfiniBand Switch Modules. For scaling to even larger clusters, while minimizing switch hops and maximizing bisection bandwidth, external Cisco InfiniBand switches can be used to create larger tree, mesh, or even full bisection bandwidth Clos network topologies.
Enterprise data centers need to address the growing demands for processing power from bandwidth-hungry and latency-sensitive applications. This need is often addressed by "scaling out" with clusters of smaller, less-expensive servers or blades as an alternative to the classic "scale-up" model with bigger servers. The scale out model offers economic and flexibility benefits, but historically has been hindered by two major issues: performance of legacy server interconnects, and "I/O sprawl" of adapters, cables, and switch ports required to connect every server directly with its LANs and SANs.
The BladeCenter along with the Cisco Systems 4X InfiniBand Switch Module and the Cisco Systems 4X InfiniBand HCA Expansion Card allows enterprise administrators to get the benefits of scale out while reducing the classic hindrances. IBM, using standards-based InfiniBand interconnect technology, delivers a solution that offers low latency and high bandwidth performance. Clustered blades can perform as well as a single monolithic server at a fraction of the cost.
You can centrally manage I/O and storage connectivity for a single BladeCenter, or an entire data center, without touching a cable. The Cisco solution allows one or more external I/O chassis to provide scalable LAN and SAN connectivity for an entire data center. Data center administrators can reduce the number of adapters, cables, and switch ports required, manage the addition, removal, or allocation of I/O or storage bandwidth centrally, and adjust I/O connectivity on demand usually without down time.
With the major obstacles to scale out alleviated, data center administrators gain an elegant and cost-effective solution for HPC, and distributed databases for an on demand computing environment.
For a copy of the applicable product warranties, contact your reseller or IBM2. IBM makes no
representation or warranty regarding third-party products or services including those designated as
ServerProven® or ClusterProven®.
Back to top
eServer® BladeCenter chassis and processor blade
Planned availability date
August 11, 2006
High Performance Computing (HPC)
HPC applications are compute-intensive, and require a high-performance interconnect to make it possible to do fast and efficient computation across multiple discreet server blades.
Node-to-node latency of under six microseconds is readily achievable using open standard protocols
such as the popular Message Passing Interface (MPI). MPI is a key component of many HPC
applications for message communication and data transfers between compute servers within a cluster.
Low latency is critical to these applications. Less time is spent on communication overhead. More time is available for valuable data processing.
- InfiniBand scalability reaches easily into the thousands of nodes.
High bandwidth communication is another of InfiniBand's strengths. The Cisco Systems 4X InfiniBand
Switch Module for IBM BladeCenter delivers up to 80 Gbps of uplink bandwidth to build nonblocking
Each switch module contains two 10 Gbps 4X ports and two 30 Gbps 12X ports.
Each 12X port can also be used as three independent 10 Gbps 4X ports.
Distributed databases such as IBM DB2® Parallel Edition and Oracle Real Application Clusters (RAC) can deliver scalability and high availability using inexpensive server platforms. This potential can go unrealized because of interconnect limitations such as high latency, low bandwidth, and high CPU overhead. BladeCenter configured with InfiniBand unlocks this potential.
Faster interprocess communication: When a database is distributed across many servers, subprocesses
frequently exchange messages to synchronize the database and share information. Often, a process
must wait for a reply from another server before responding to a database query or update. Low
latency reduces the time spent waiting for data.
With BladeCenter configured with InfiniBand, the database application enjoys a significant throughput boost through a combination of low-latency messaging and high-bandwidth data transfer rates.
- Increased database performance: Host CPU offload increases database performance by freeing each CPU from processing communication overhead. RDMA is a key ingredient in the CPU offload benefit. It allows a process on one server to directly access the memory on another server, once the transfer has been authenticated. The remote server's CPU does not intervene and suffers little or no performance penalty in handling the data transfer.
Storage aggregation: Databases are traditionally storage-hungry applications. An
InfiniBand-equipped BladeCenter using external Fibre Channel gateway modules allows a seamless
interface to Fibre Channel switches and storage targets.
Administrators manage gateways centrally and all blades may share storage access through each gateway. Peak storage loads are addressed without deploying excessive hardware throughout the data center.
Also, administrators save the cost of redundant HBAs and Fibre Channel switches while taking advantage of easy serviceability.
On demand computing
On demand computing addresses inefficiency and waste in the data center. You can avoid dedicating servers to part-time functions and leverage a common pool of shared, stateless, server resources that can be matched to I/O, storage, and applications on demand.
On demand computing allows data center managers to:
- Share CPU and I/O resources
- Improve high availability
- Reduce costs and waste
- Increase application performance
The Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter provides a programmable fabric for on demand computing.
The BladeCenter with InfiniBand solution also makes it possible to manage I/O and storage connectivity for an entire BladeCenter cluster in a central optimized fashion. Rather than requiring Ethernet and Fibre Channel off of each blade, or even off of each BladeCenter chassis, the BladeCenter with InfiniBand allows a single external I/O chassis to provide LAN and SAN connectivity to an entire data center. This allows the enterprise administrator to reduce the amount of adapters, cables, and switch ports required, manage the addition or removal of I/O or storage bandwidth centrally, and offer the ability to adjust on demand connectivity without any downtime required.
The BladeCenter on demand solution set is built around the following major building blocks:
- High performance server-to-server connectivity: The InfiniBand switch module provides the fabric for server virtualization. Central to the server switch is InfiniBand, which offers the extremely high bandwidth and low latency ideal for interconnecting servers.
- Virtualization and boot services: These capabilities allow blade servers to be deployed in a stateless fashion and enable you to provision any combination of application, operating system, storage, and I/O resources to a server on demand over the InfiniBand fabric.
- Seamless I/O and storage connectivity: BladeCenter is compatible with Cisco's external InfiniBand solutions set including Ethernet and Fibre Channel gateways and unique topology transparency software that enables the high-performance server fabric to seamlessly connect with existing LANs and SANs with little or no management or infrastructure changes.
Virtualization software suite: BladeCenter is compatible with Cisco's VFrame software which embeds
policy and provisioning intelligence into the fabric. This intelligence enables the fabric
automatically to create virtual servers by interconnecting the right combination of resources at the
right time, based on policies a data center manager defines.
As conditions change, the switch can be programmed to add and remove resources from virtual servers as necessary.
With this flexibility, the Cisco solution for BladeCenter allows data center managers to easily deploy new applications, repurpose servers from one function to another on demand, and respond rapidly to server or network failures. The result helps to dramatically reduce downtime.
The external Cisco gateway hardware and virtualization software referenced above is available directly from Cisco Communications.
All offers are subject to availability. IBM reserves the right to alter product offerings and specifications at any time without notice. IBM is not responsible for photographic or typographic errors.
IBM makes no representation or warranty regarding third-party products or services.
Back to top
- Gbps, GBps, and MBps denote the rated physical speed of the device, not the number of data bits or bytes transferred. Many factors affect actual data throughput.
For information on the IBM Statement of Limited Warranty, call 800-IBM-SERV (426-7378), or contact
your IBM representative or reseller, or write to:
P.O. Box 12195
RTP, NC 27709
Attn: Dept. JDJA/B203.
Copies are available upon request.
Cisco Systems 4X InfiniBand Switch Module for BladeCenter (32R1756)
Cisco Systems 4X InfiniBand HCA Expansion Card for BladeCenter (32R1760)
Performance specifications: An enterprise InfiniBand switch for fabric integration management into BladeCenter architecture and associated options
- Approximate height: 20 mm (0.8 in)
- Approximate width: 285 mm (11.2 in)
- Approximate depth: 265 mm (10.5 in)
- Approximate height: 2 mm (0.08 in)
- Approximate width: 2 mm (0.08 in)
- Temperature: 10° to 35°C (50° to 95°F)
- Relative humidity: 8% to 80% noncondensing
- FCC part 15B Class A
- CAN/CSA-C22.2 No. 60950-1-03
- Canada ICES-003
- VCCI Class A ITE (Japan)
- CE Mark
- CISPR 22 Class A
- A4EN55022 level A (EC)
- Voltage fluctuations: EN61000-3-3
- Harmonics: EN61000-3-2
- Immunity: EN55024:1998
ENERGY STAR-compliant: No
Hardware requirements: The Cisco Systems 4X InfiniBand Switch Module for IBM BladeCenter is supported on the following IBM platforms.
Machine System name type Model BladeCenter H 8852 All
The Cisco Systems 4X InfiniBand HCA Expansion Card for IBM BladeCenter is supported on the following BladeCenter blade servers.
Machine System name type Model BladeCenter HS21 8853 All
Software requirements: The Cisco Systems 4X InfiniBand HCA Expansion Card for IBM BladeCenter is supported on the following operating systems:
- Red Hat 4 Advanced Server
- SUSE Linux Enterprise Server 9
- Windows Server 2000 (Standard, Enterprise, Web)
- Windows Server 2003 (Standard, Enterprise, Web)
Compatibility: The product is designed to maintain compatibility with many IBM products. As a part of the ServerProven® program, xSeries® compatibility listings are updated on the Web. Detailed information relative to IBM and non-IBM vendor devices, adapters, software, and network operating systems supported with xSeries products can be accessed via the Web
Cable orders required: Yes
Cable Cable part description number IB 3 meter 4x Cable for IBM BladeCenter 26R0813 IB 3 meter 12x to (3) 4x Cable for 26R0816 IBM BladeCenter IB 8 meter 4x Cable for IBM BladeCenter 26R0847 IB 8 meter 12x to (3) 4x Cable for 26R0849 IBM BladeCenter
Installation time: Approximately 20 minutes
Packaging: One box
Shipment group Switch Module
- Cisco 4X InfiniBand Switch Module
- Installation publication
- License agreement
- Safety flyer
Single box (HxDxW):
- 431.80 mm (17 in)
- 406.40 mm (16 in)
- 127.00 mm (5 in)
- Weight: 5 lb
Shipment group Host Channel Adapter
- Cisco 4x InfiniBand Host Channel Adapter Expansion Card
- Installation publication
- License agreement
- Safety flyer
Single box (HxDxW):
- 279.40 mm (11 in)
- 381.00 mm (15 in)
- 50.80 mm (2 in)
- Weight: 3 lb
Security, auditability, and control
This product uses the security and auditability features of host hardware, host software, and application software.
Terms and conditions
Field installable: Yes
Customer setup: Yes
Warranty period: One year3
Service type (CCE/CCR/CRU): CRU4
Maintenance agreement: No
Optional upgrade: No
In the United States, call 800-IBM-SERV (426-7378), or write to:
P.O. Box 12195
RTP, NC 27709
Attn: Dept. UF2A/B203
Copies of the IBM Statement of Limited Warranty can be obtained by contacting your reseller or accessing the online services listed in the Help and Service Information section of this announcement, or by calling IBM in the United States, at 800-426-7378. In Canada, call IBM at 800-565-3344.
Note: Programs included with this product are licensed under the terms and conditions of the License Agreements that are shipped with the feature or option.
All other terms and conditions are the same as those applicable to the IBM machine type in which the feature is installed.
International Warranty Service (IWS): IWS is available during the warranty period to customers who travel or relocate to countries where their computer is sold and serviced by IBM or IBM resellers authorized to perform warranty service. Eligible IBM computers are identified by their four-digit machine type.
You can obtain IWS through the method of service, such as CRU, depot, carry-in or on-site, provided in the servicing country. Service methods and procedures vary by country, and some service or parts may not be available in all countries. Service centers in certain countries may not able to service all models of a particular machine type. In addition, some countries may have fees and restrictions that apply at the time of service.
To determine the eligibility of your computer and to view a list of countries where service is available, visit
Note: Due to the earth's magnetic field, cathode ray tube (CRT) monitors are manufactured to work in northern, southern, and equatorial regions of the earth and may not produce a satisfactory image when moved between them. Any required adjustment (if possible) is not covered under IWS and may be subject to a chargeable action. The magnetic field does not affect flat panel LCD monitors.
Help and service information Features and options
During the warranty period, support for a warranty-related issue can be obtained by accessing IBM's online services (requires internet access account, not included) or calling the IBM Support Center at 800-426-7378, 24 hours a day, 7 days a week, to answer any questions regarding your new IBM option. Response time will vary depending on the number and nature of calls received. If you need warranty service, return or exchange is available. In addition, if your IBM option is installed in an IBM computer, you may be entitled to service at your location. For general questions related to the use of your options, fee-based support is available. Your technical support representative can help you determine the best alternative.
Accessing IBM online services
IBM Web site at
Back to top
The prices provided in this announcement are suggested retail prices for the U.S. only and are provided for your information only. Dealer prices may vary, and prices may also vary by country. Prices are subject to change without notice. For additional information and current prices, contact your local IBM representative.
IBM Part list Description number price(5) Cisco 4xIB Switch Module 32R1756 $11,499 for IBM BladeCenter Cisco 4xIB Host Channel 32R1760 1,399 Adapter Expansion Card for IBM BladeCenter
- List price does not include tax or shipping and is subject to change without notice. Reseller prices may vary.
Back to top
|IBM Global Financing|
IBM Global Financing offers competitive financing to credit-qualified customers to assist them in acquiring IT solutions. Offerings include financing for IT acquisition, including hardware, software, and services, from both IBM and other manufacturers or vendors. Offerings (for all customer segments: small, medium, and large enterprise), rates, terms, and availability can vary by country. Contact your local IBM Global Financing organization or visit
IBM Global Financing offerings are provided through IBM Credit LLC in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government customers. Rates are based on a customer's credit rating, financing terms, offering type, equipment type, and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice.
Back to top