Published on 13-Nov-2012
"With Platform HPC, the master node is aware of resource demands of the jobs queued and what each node is doing. It will power down those nodes not being used, aside from keeping about 5-10 percent available at all times. When usage increases, the master node powers on what’s needed within a couple minutes." - Chris Collins, Head of Research and Specialist Computing Support, UEA
University of East Anglia
Technical Computing, Energy Efficiency
The University of East Anglia (UEA) is widely recognized as one of the world’s premier research universities. The university’s Climatic Research Unit (CRU) has been at the forefront of environmental research on climate change since being founded in 1972, and the university community prides itself on its ongoing effort to be an example of campus-wide good environmental practice and sustainability.
Create a "green" HPC resource Expand HPC usage beyond scientific disciplines to researchers in the arts and humanities Increase compute power in an efficient manner Support for research across multiple operating systems, including both Windows and Linux
Platform HPC provided complete solution including advanced cluster and workload management while lowering the administration burden.
Compute power increase from 9 to 21.5 teraflops Grew cluster size from 900 to more than 2000 cores System hardware now running 20 - 40 percent faster Cut power consumption rates and costs
Established in 1963 in Norwich, England, the University of East Anglia (UEA) is widely recognized as one of the world’s premier research universities. The university’s Climatic Research Unit (CRU) has been at the forefront of environmental research on climate change since being founded in 1972, and the university community prides itself on its ongoing effort to be an example of campus-wide good environmental practice and sustainability. In 2007, more than 20 scientists from UEA shared the Nobel Peace Prize with Al Gore for their environmental research. In addition, nearly 90 percent of the university’s research was rated of “international excellence” in the latest Research Assessment Exercise, with over 50 percent of that research classified as “world leading.”
With such a heavy focus on research, high performance computing (HPC) has been an integral part of the university for the past ten years, with the campus running its own dedicated general purpose HPC facility. The HPC resource has traditionally supported research within the scientific disciplines, such as the university’s climate and environmental research and chemistry, computer science and mathematics departments. But as the university’s research demands grew, so did their need for compute power. The system they had in place was no longer able to keep up with the science the researchers were doing.
The university also wanted to extend usage of the HPC compute system beyond the scientific disciplines to support research needs within the rest of the university with the goal of encouraging use within disciplines not traditionally associated with HPC applications, such as the social sciences, arts and humanities. They also required a solution that was faster, easier to manage and that provided more power.
Due to a campus-wide focus on sustainability, having a green HPC resource was a high priority not only for the climate researchers who regularly use the center to process their research but for the entire community. As such, UEA set out to find an HPC solution that could offer them the high performance they needed, but that could also meet their strict requirements for reducing the university’s carbon footprint and power consumption at lower cost.
To create their Green HPC solution, UEA turned to Viglen and IBM® Platform™ HPC.
Extending HPC beyond the Sciences
With the dual needs of environmentally friendly, power-efficient computing and the desire to embrace new users across the university in mind, UEA embarked on a project to upgrade their HPC resources. Working together with systems integrator Viglen, UEA more than doubled its HPC core capacity from 900 to more than 2000 cores to meet their needs. According to Chris Collins, Head of Research and Specialist Computing Support at UEA, the university needed a system that provided more intelligence and features than the previous solutions they already had in place. Key to the decision process to implement Platform HPC management software was the ability to provide a complete solution that offered advanced cluster management and scheduling while also lowering the administration burden on the IT department so as to allow administrators more face time with users to assist them with problems when necessary.
Bordan Tkachuk, CEO of Viglen said “Viglen takes very seriously the greening of computing, especially in the area of HPC where traditionally huge amounts of both computing power and power consumption go hand in hand. This installation at UEA is an exciting step in proving that using leading edge HPC technology can use less power without compromising computing power.”
He added “The increase in compute ability allows UEA to perform research across larger data sets and at a finer level of granularity than before. They really needed a faster, easier to manage system with more capability. This new system provides the research platform to address the requirements of the world leading research being conducted at UEA.”
“We’re migrating new users on to the system all the time,” noted UEA’s Collins. “We also use Platform MPI (included in Platform HPC) for migrations since transferring and linking applications can be a tedious process. Before, we had to differentiate what nodes applications could run on. Now we can run on both Ethernet and Infiniband, make a request for an application and easily link it to Platform MPI.”
The easy-to-use web interface in Platform HPC was another factor in UEA’s decision. “That was a huge benefit compared to not having a web interface before. With the challenge of getting non-HPC users to use the system, we could allow them to use the web front-end instead of using the command line interface,” Collins noted. “We don’t have to do as much work when releasing a new application or do as much new coding. Platform’s software was a clear leader from the beginning of the process.”
Powering up to power down
UEA’s new HPC resource has provided the university a significant increase in computational power. The solution including Platform HPC has more than doubled UEA’s computer power, going from a capacity of 9 teraflops to 21.5 teraflops in just a few months. The new system has 2032 cores under management and 168 compute nodes. UEA also added a GPU node to the cluster for added processing power. Platform HPC has provided UEA with the following additional benefits:
· Enables UEA to improve utilization of their resources by intelligently placing workloads based on the hardware required by specific tasks
· Allows UEA to use software licenses more efficiently, maximize throughput and reduce administration by utilizing Platform HPC when considering availability of software licenses in scheduling decisions
· Improves the performance of UEA’s build environment, and helps simplify the management of parallel jobs with IBM® Platform™ MPI, a component of Platform HPC
Despite the increase in power, UEA has been able to remain true to the university’s mission of environmental sustainability. Although the Data Center is the university’s largest consumer of electricity, Platform HPC is helping to reduce their carbon footprint and save money. This is in large part due to the ability to automatically power systems on and off dependent on need and usage. Prior to implementing Platform HPC, UEA was not properly utilizing its power structure because the system was still consuming energy even if jobs were not being run. UEA can now shut down power to nodes when they are not in use, significantly reducing wasted energy and costs.
Also key, according to Viglen HPC Product Manager David Power, was the ability of IBM Platform Computing to support Windows environments and applications so use of the HPC resource could be extended beyond the traditional HPC scientific disciplines. Because users across all of the university’s academic disciplines run hundreds of different applications specific to their own research, UEA also needed a solution that could be easily integrated with new software packages. To allow researchers to use both Windows and Linux operating systems simultaneously, UEA implemented Platform HPC, which allows multiple operating systems to run on the same cluster at the same time thus reaching users across the entire university.
“With Platform HPC, the master node is aware of resource demands of the jobs queued and what each node is doing. It will power down those nodes not being used, aside from keeping about 5-10 percent available at all times. When usage increases, the master node powers on what’s needed within a couple minutes” said Collins.
Such administration benefits have been extremely helpful to Collins and his team, as well as the users throughout the university. “Platform HPC has rich functionality and the documentation is extremely helpful, and we’ve received ongoing positive feedback from users.”
As the administration team and users continue to learn more about the solution, Collins is extremely positive about the options IBM Platform Computing can provide UEA. UEA currently plans to add more IBM Platform Computing licenses to the deployment and are interested in adding additional GPUs for even more processing power and to double the size of the system once again within 12 months. With the entire university community set to engage in high performance computing, the University of East Anglia will be poised to maintain its world leading research status for years to come.
To learn more about IBM Platform Computing please contact your IBM marketing representative or IBM Business Partner, or visit the following website: ibm.com/platformcomputing
Products and services used
IBM products and services that were used in this case study.
Platform HPC for System x
© Copyright IBM Corporation 2012 IBM Corporation Systems and Technology Group Route 100 Somers, NY 10589 Produced in the United States of America May 2012 IBM, the IBM logo, ibm.com, Platform Computing, Platform HPC and Platform MPI are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml Microsoft, Windows and Windows NT are trademarks of Microsoft Corporation in the United States, other countries, or both. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates. The performance data and client examples cited are presented for illustrative purposes only. Actual performance results may vary depending on specific configurations and operating conditions. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided. Actual available storage capacity may be reported for both uncompressed and compressed data and will vary and may be less than stated.