Georgia Tech implements a cool solution for green HPC with IBM

Founded in 1885, the Georgia Institute of Technology (Georgia Tech) is one of America’s leading research universities. Its over 18,000 undergraduate and graduate students are taught by more than 900 full-time staff on a 400-acre campus in the heart of Atlanta, GA.

Georgia Tech’s Center for the Study of Systems Biology requires supercomputing capabilities for protein structure simulations and other techniques supporting research into new drugs. By creating detailed computer models of new chemical compounds, researchers can potentially reduce by a factor of ten the number of physical compounds that need to be tested, which can help them to dramatically accelerate their research supporting the creation of new life-saving drugs.

Using advanced IBM BladeCenter and cooling technology, Georgia Tech built a compact supercomputer with low power consumption and heat output. Says Bartosz Ilkowski, Georgia Tech senior research technologist, “Our own innovative data center design, allied with the IBM technologies, enabled us to pack a huge amount of computing power in a compact space. The new supercomputer is far more cost effective and environmentally friendly than a traditional design, while offering the same compute power.”


Revolutionary design
Georgia Tech upgrades its supercomputing assets on a three-year cycle, to ensure that it continues to attract world-class students, researchers and academics. For its most recent upgrade, Georgia Tech specified a high-density 4,000-core computational solution.

“We needed a compact solution to reduce data center floor space—and therefore hosting costs. IBM BladeCenter offered 25 to 40 percent more servers per rack for no additional power or cooling,” says Ilkowski. “The environmental factor was also important: blades would consume up to three times less power than the alternative offerings based on rack-mounted servers.”

Increasing the density of the solution would make better use of available floor space, but would also create “hotspots” in the data center requiring significant cooling. The university wanted the new supercomputer to act as a showcase for its ground-breaking research facilities, so it was vital to keep air conditioning noise and air displacement to a minimum.

The “traditional” solution design called for 85 tons of air conditioning (a total of just over 1 million BTU per hour) and very high airflow—which would have been expensive, noisy and impractical. Instead, Georgia Tech created a revolutionary cooling solution employing air conditioning with physically separate hot/cold aisles, and water cooling via an IBM Rear Door Heat eXchanger unit on each rack. A unique aspect of the cooling solution is Georgia Tech’s creation of “open floor” tiles on the cold aisle, with special directional baffles that project cool air to the top of the cabinets and minimize hotspots.


Fast, compact, cool
The supercomputer itself is an IBM System Cluster 1350 solution with more than 1,000 IBM BladeCenter nodes, each with four AMD Opteron cores running Linux®. With a maximal LINPACK performance of 8.5 TFlops (trillion floating-point operations per second), the solution was the 51^st fastest supercomputer in the world as of June 2006.

Each of the 12 racks used in the solution is equipped with an IBM Rear Door Heat eXchanger unit and contains six BladeCenter chassis, each with 14 BladeCenter LS20 servers.

The use of IBM Rear Door Heat eXchanger technology reduced the air conditioning requirements by approximately 55 percent, and cut the operational cost of cooling the blades by between 10 and 15 percent. Georgia Tech estimates annual cost savings for air conditioning of approximately $160,000. In addition, the compactness of the solution design reduced the actual amount of floor space that Georgia Tech needed to lease by around one third.

Airflow requirements are also considerably lower. Says Ilkowski, “Using just air conditioning, the airflow required to cool the data center would have been approximately 52,500 cubic feet per minute (CFM). By achieving around 50 percent of the total cooling effect through water cooling combined with our innovative data center design, we reduced the amount of air conditioning needed and thereby also reduced the required airflow to around 30,000 CFM. Lower airflow means less disruption and less noise for visitors to this showcase facility.”

He concludes, “The new supercomputer is the result of a highly successful collaboration by Georgia Tech, IBM, Minick Engineering and our hosting partner, BellSouth (now AT&T) in Atlanta. We are pleased that we now have a very compact, energy-efficient solution that is easy to install, manage and scale.”