Los Alamos National Laboratory

Expanding scientific research with IBM System x and ScaleMP

Published on 11-Nov-2011

Validated on 03 Feb 2014

"The IBM solution seemed like the best option because it was Intel based and we didn’t have to run any sort of proprietary operating system. We haven’t had cluster nodes with this much memory capacity to date." - Gary Grider, deputy division leader, high performance computing division, Los Alamos National Laboratory

Customer:
Los Alamos National Laboratory

Industry:
Aerospace & Defense

Deployment country:
United States

Solution:
IT/infrastructure, Linux, Optimizing IT, Virtualization - Applications, Virtualization - Network, Virtualization - Server, Virtualization - Storage

Overview

Los Alamos National Laboratory (LANL) is the nation’s premier national security research institution. From its beginning in 1943 to its leadership role today in a variety of fundamental science research vital to national interests, LANL has always applied the best scientific and engineering know-how to some of the nation’s most crucial challenges.

Business need:
Los Alamos National Laboratory needed a high-performance computing solution with a highly distributed memory and processor architecture to address a small but growing area of study called informatics.

Solution:
LANL deployed a test solution comprised of four IBM® System x® 3850 X5 enterprise-class servers, each with 32 Intel® Xeon® X7550 processors running at 2 gigahertz and 256 gigabytes of memory and used ScaleMP vSMP Foundation to create a virtual machine with 128 cores and a total of 1 terabyte of addressable memory.

Benefits:
The IBM solution enables scientific research in highly connected graph science problems, delivers a large addressable memory footprint and provides a cost-effective alternative to costly, specialized solutions.

Case Study

Los Alamos National Laboratory (LANL) is the nation’s premier national security research institution. From its beginning in 1943 to its leadership role today in a variety of fundamental science research vital to national interests, LANL has always applied the best scientific and engineering know-how to some of the nation’s most crucial challenges.

Historically, these challenges have been met with brute computer processing power, and the lab remains home to some of the world’s most powerful supercomputers, including Roadrunner. The Roadrunner system was created with IBM using commercially available hybrid technology, and when it launched in 2008, it was the first supercomputer to break the petaflop barrier—the equivalent of one quadrillion floating point operations per second.

Seeing the need for high performance informatics

Subsequently, much of the research conducted by the approximately 10,000 scientists at LANL, located about 100 miles north of Albuquerque, New Mexico, has seen a massive benefit from Roadrunner and other similar high performance computing resources in areas such as gene sequencing, virus research and plasma physics. Yet a small but growing area of study called informatics requires a vastly different computing architecture, explains Gary Grider, deputy division leader of the high performance computing division at LANL.

“The bulk of what we do in computing here at the lab is where we’re simulating some physical thing. These sorts of problems are best solved with highly distributed memory and processor architectures,” says Grider. “However, we are starting to see more informatics kinds of problems, where instead of a material simulation, we’re trying to connect the dots to find out how things are correlated. We call these graphing problems.”

The problem with traditional supercomputers

Grider explains that solving these graphing problems on a traditional supercomputer is not ideal because of the way those systems are built to tackle specific scientific applications through rapid calculations. Most supercomputers are very distributed, with many racks of processor nodes connected over a vast network.

For complex graphing applications, such supercomputer designs lack the large amount of shared memory space that is required to do large, highly-connected and data-intensive graph problems. “You need everything in shared memory because you have this really fine-grained, highly-correlated data access,” says Grider.

Grider adds that there are a few specialized graphing computers on the market, but because they are so highly specialized, they cost millions of dollars. This was not an option for LANL because the scope of the lab’s graphing problem research remains relatively small, says Grider.

“We just wanted to do a little bit of work in this area and we didn’t really want to go buy something expensive,” he explains. “We were looking for a test bed so we could evaluate very large memory and highly interconnected architectures, and IBM provided a potential solution.”

IBM, Intel and ScaleMP: A potent combination

The solution chosen by LANL consists of four IBM® System x® 3850 X5 enterprise-class servers. Each x3850 X5 contains 32 Intel® Xeon® X7550 processors running at 2 gigahertz and 256 gigabytes of memory. The systems are connected using multiple InfiniBand quad data rate (QDR) links and virtualized to a single resource using vSMP Foundation from ScaleMP. The virtualized resource runs Linux and gives the researchers 128 cores and 1 terabyte of memory to run complex graph problems.

“The IBM solution provided the best option because it was Intel based and we didn’t have to run any sort of proprietary operating system—we can just run Linux,” says Grider. “It’s an incredibly rich memory machine, and the fact that we could put multiple machines together using vSMP Foundation from ScaleMP over the InfiniBand is a big advantage for us. We haven’t had cluster nodes with this much memory capacity to date.”

Promising results for advanced research

The IBM solution was purchased by LANL as a pilot program to determine if it can address the needs of lab scientists in search of a fast graphing problem computing solution. Grider says the pilot program is being conducted a part of a LANL college undergraduate program, where teams of students have set up and configured the IBM solution to deliver working results on graph problems.

And the results, so far, “look very promising,” says Grider. The results may even be enough to land them a spot in the Graph 500 benchmark ranking for the very fastest computers designed to solve data-intensive applications.

One of the key areas where the IBM solution may provide a benefit to LANL is with biothreat reduction studies. This includes better understanding how pathogens naturally found in the environment, especially in cities, differ from biothreat pathogens which have been intentionally released into the environment. In the event of a bioterrorist attack such as the anthrax cases in 2001, the research is expected to help decision makers and law enforcement know quickly what kind of threat they are facing..

“We have plenty of demand in the analysis of graphs here at the lab, and informatics is growing in the scientific community as well,” says Grider. “There’s all of this data in the world today, and informatics is all about trying to make heads or tails of it.”

And IBM is helping to meet the challenges in this new world of informatics computing. Says Grider: “The overall solution combining IBM, Intel and ScaleMP is kind of unique, and what you can do with it is pretty interesting. I think maybe it’s a little bit ahead of its time.”

For more information

Contact your IBM representative or IBM Business Partner, or visit us at: ibm.com/systems/x

For more information about Los Alamos National Laboratory, visit: www.lanl.gov

For more information about ScaleMP, visit: www.scalemp.com

Products and services used

IBM products and services that were used in this case study.

Hardware:
System x: System x3850 X5

Operating system:
Linux

Legal Information

© Copyright IBM Corporation 2011 IBM Systems and Technology Group Route 100 Somers, New York 10589 U.S.A. Produced in the United States of America November 2011 All Rights Reserved IBM, the IBM logo, ibm.com and System x are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml Intel, Intel logo and Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product and service names may be trademarks or service marks of others. References in this publication to IBM products, programs or services do not imply that IBM intends to make these available in all countries in which IBM operates. Offerings are subject to change, extension or withdrawal without notice. All client examples cited represent how some clients have used IBM products and the results they may have achieved. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS-IS” WITHOUT ANY WARRANTY, EITHER EXPRESSED OR IMPLIED.