Published on 22-Apr-2012
"There are lots of good reasons to use Revolution Analytics in an analytics appliance super computer like Netezza. It’s faster and easier to program. It will speed up our computation. And, we have more data sets available to us because of how flexible and quick Revolution Analytics makes it to add and delete variables in our model." - Dr. Murali Ramanathan, University at Buffalo, State University of New York
State University of New York (SUNY) at Buffalo
PureData System for Analytics (powered by Netezza technology), PureSystems, Big Data, Data Warehouse, Smarter Planet
IBM Business Partner:
IBM® Netezza® and Revolution Analytics power the study of exponentially large gene interaction associated with multiple sclerosis
Researchers required the ability to quickly build models using a range of variable types and run them on a high-performing environment on huge data sets.
The solution helped the SUNY Buffalo researchers consolidate all reporting and analysis in one location to improve the efficiency, sophistication and impact of their research.
The SUNY researchers were able to reduce the time required to conduct analysis from 27.2 hours without the IBM Netezza data warehouse appliance to 11.7 minutes with it.
The State University of New York (SUNY) at Buffalo is home to one of the leading multiple sclerosis (MS) research centers in the world. MS is a devastating, chronic neurological disease that affects nearly one million people worldwide. The disease causes physical and cognitive disabilities in individuals and is characterized by inflammation and neuro-degeneration of the brain and spinal cord. From the beginning, the genetics of MS were known to be complex and it was apparent that no single gene was likely causative for the disease.
Since 2007, the SUNY team has been looking at data obtained from scanned genomes of MS patients to identify genes whose variations could contribute to the risk of developing MS. New technologies now enable hundreds of thousands of genetic variations, called single nucleotide polymorphisms (SNPs), to be obtained from single samples. According to research lead, Dr. Murali Ramanathan, a critical fact in the study of MS is that “gene products work by interacting with both other gene products and environmental factors.”
“Because of this, researchers have postulated that multiple SNPs—combined with environmental variables—would better explain the risk of developing MS. Finding these combinations is analogous to the proverbial search for a needle in a haystack,” says Dr. Ramanathan. Identifying a candidate’s environmental factors that could be used to prevent the disease from progressing in patients was of great importance. Examples of this include sun exposure and vitamin D levels, Epstein-Barr virus infection and smoking.
The researchers needed to create algorithms that would efficiently identify interactions and attempt ‘parsimony’—an extreme efficiency. The team was looking for the fewest number of SNPs, environmental and phenotypic variable combinations that would help explain the presence of MS.
The researchers at the State University of New York developed an approach they called AMBIENCE that is distinctive in its use of new information theoretic methods along with its versatility and scalability. The theoretic underpinnings of AMBIENCE enable the detection of both linear and non-linear dependencies in the data. The AMBIENCE algorithm is capable of conducting an efficient search of the large combinatorial space because of the unique nature of the information theoretic metrics, which allows for greedy search identification of the most promising combinations.
The data sets used in this type of multi-variable research are very large and the analysis is computationally very demanding because the researchers are looking for significant interactions between thousands of genetic and environmental factors. There are two issues to overcome: crunching through the immense data set and building analytic models that allow the team to look at more than simply first order interactions. The researchers want to see not only which variable is significant, but also which pairs of variables or which three variables are significant. This requires the ability to quickly build models using a range of variable types and run them on a high-performing environment on huge data sets. It also requires the ability to include an almost limitless variety of dependent variable types.
The computational challenge in gene-environmental interaction analyses is due to a phenomenon called ‘combinatorial explosion.’ Considering that there are thousands of SNPs, the number of combinations of SNPs that have to be assessed for uncovering potential interaction becomes incredibly large.
Before they could run the analysis using all of the variables presented in the data set, the researchers needed an analytic framework that would allow them to add and remove variables from the model quickly and easily, without having to write hundreds of lines of code. The sheer number of SNPs combined with environmental variables and phenotype values mean that the amount of computations necessary for data mining could number in the quintillions (18 zeros).
SUNY Buffalo is using Revolution R Enterprise for IBM Netezza® in conjunction with the IBM Netezza Analytics Appliance to dramatically simplify and speed up very complex analysis on large data sets.
The organization’s researchers knew they needed the level of processing only available through a high-performance computer (HPC). They also needed capabilities found in relational databases—not often included in HPC platforms. HPCs typically run their processes only in parallel, breaking up calculations to run simultaneously across multiple processors. In addition, they also often used field-programmable gate array (FPGA) architectures for additional speed.
The IBM Netezza data warehouse appliance offered the features and performance the researchers needed and much more. From an analytics perspective, the SUNY Buffalo researchers were able to write a version of their software tools in Revolution R Enterprise for IBM Netezza, so that all their reporting and analysis were consolidated in one location. This prevented the need to move very large amounts of data in and out of the IBM Netezza data warehouse appliance, which would cause further delays. They were also able to use a wider variety of data sets.
Due to the nature of SUNY Buffalo's research work, there is immense value in now being able to use a wide variety of data sets to study the interactions among a greater range of variables. The solution improved the efficiency, sophistication and impact of the research. SUNY’s team adopted R—the core of Revolution R Enterprise for IBM Netezza—because it offered the flexibility to include diverse types of variables such as categorical, discrete Poisson-dependent, or continuous normally-distributed variables by simply adding a few lines of code. In the past, the SUNY team would have to rewrite entire algorithms, requiring a great deal of staff time. Now, scientists can change the algorithm themselves by adding a new entropy function, and focus less on writing algorithms and more on the science.
Once SUNY Buffalo deployed the IBM Netezza data warehouse appliance as its research analytics infrastructure along with Revolution R Enterprise for IBM Netezza, and the genetic data were assembled, the environmental and phenotype data were combined, and the algorithms were customized, the researchers were empowered to look for potential factors contributing to the risk of developing MS.
The SUNY researchers were able to:
- Use the new algorithms and add multiple variables that before were nearly impossible to achieve.
- Reduce the time required to conduct analysis from 27.2 hours without the IBM Netezza data warehouse appliance to 11.7 minutes with it.
- Carry out their research with little to no database administration. (Unlike other HPC platforms or databases available, the IBM Netezza data warehouse appliance was designed to require a minimum amount of maintenance.)
- Publish multiple articles in scientific journals, with more in process.
- Proceed with studies based on ‘vector phenotypes’—a more complex variable that will further push the IBM Netezza data warehouse appliance platform.
- IBM Netezza® 1000-24
- IBM Netezza Analytics Appliance
- Revolution R Enterprise for IBM Netezza
For more information
To learn more about the IBM Netezza data warehouse appliances, please contact your IBM representative or IBM Business Partner, or visit the following website: ibm.com/software/data/netezza
For more information about Revolution Analytics, visit www.revolutionanalytics.com
For more information on SUNY Buffalo, visit: www.buffalo.edu/ , or for more information on SUNY’s research, consult the following sample publications:
AMBIENCE: a novel approach and efficient algorithm for identifying informative genetic and environmental associations with complex phenotypes. Chanda P, Sucheston L, Zhang A, Brazeau D, Freudenheim JL, Ambrosone C, Ramanathan M. Genetics. 2008 Oct; 180(2): 1191-210. Epub 2008 Sep 9. PMID: 18780753 [PubMed - indexed for MEDLINE]
Information-theoretic gene-gene and gene-environment interaction analysis of quantitative traits. Chanda P, Sucheston L, Liu S, Zhang A, Ramanathan M. BMC Genomics. 2009 Nov 4; 10:509. PMID: 19889230.
Comparison of information-theoretic to statistical methods for gene-gene interactions in the presence of genetic heterogeneity. Sucheston L, Chanda P, Zhang A, Tritchler D, Ramanathan M. BMC Genomics. 2010 Sep 3; 11:487. PMID: 20815886.
The interaction index, a novel information-theoretic metric for prioritizing interacting genetic variations and environmental factors. Chanda P, Sucheston L, Zhang A, Ramanathan M. Eur J Hum Genet. 2009 Oct;17(10): 1274-86. Epub 2009 Mar 18. PMID: 19293841.
Products and services used
IBM products and services that were used in this case study.
IBM Netezza Analytics, IBM Netezza 1000
© Copyright IBM Corporation 2012 IBM Software Group Route 100 Somers, NY 10589 Produced in the United States of America April 2012 IBM, the IBM logo, and ibm.com are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml Netezza is a registered trademark of IBM International Group B.V., an IBM Company. This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates. The performance data and client examples cited are presented for illustrative purposes only. Actual performance results may vary depending on specific configurations and operating conditions. It is the user’s responsibility to evaluate and verify the operation of any other products or programs with IBM products and programs. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided. Revolution R Enterprise for IBM Netezza is not an IBM product or offering. Revolution R Enterprise for IBM Netezza is sold or licensed, as the case may be, to users under Revolution Analytics’ terms and conditions, which are provided with the product or offering. Availability, and any and all warranties, services and support for Revolution R Enterprise for IBM Netezza is the direct responsibility of, and is provided directly to users by, Revolution Analytics.