Published on 25-Mar-2013
"Detecting manipulation on a trading floor, anticipating market changes, and identifying deficiencies in our system—these are all things that help us shape our business and our future." - Emile Werr, Head of Enterprise Data Architecture, Vice President of Global Data Services, NYSE Euronext
PureData System for Analytics (powered by Netezza technology), PureSystems, Big Data, Data Warehouse, Smarter Planet
At NYSE Euronext, a global trading and technology provider, data experts found that their ability to uncover patterns buried deep within billions of trades required a simple approach to a complex problem. Several years ago, NYSE Euronext had used traditional database technology to drive analytics.
With steadily growing data volumes, data analysts found it increasingly difficult to process the data—especially when searching for questionable trading activity.
By replacing third-party database software with the IBM® Netezza® 1000 data warehouse appliance, the exchange sped up and simplified analytic processes for capacity planning, operational assessment, and compliance and regulation. Moving its analytic solutions from a traditional database environment to a powerful, integrated data-ready platform also helped NYSE Euronext reduce the time and cost IT staff spent delivering analytic services to the business.
• Reduced the time required to run market surveillance algorithms by more than 99 percent • Decreased the number of IT resources required to support the solution by more than 35 percent • Helped compliance personnel to detect suspicious patterns of trading activity early, thus minimizing potential damage
Uncovering questionable trading patterns by using powerful detection algorithms.
Amid a soaring number of transactions, securities markets not only need processing power, but also the flexibility to incorporate new algorithms that can spot risks—such as questionable trading activity—before they can do damage. At NYSE Euronext, a powerful new analytics engine has sped up and simplified analytic processes so that even as the volume of securities trading continues to grow, the exchange has the capability to help maintain a level playing field for all investors.
|Smarter Trading:||Rapidly uncovering patterns across billions of trades|
|Instrumented||Extracts trading data from a wide variety of systems within NYSE Euronext.|
|Interconnected||Automatically adapts to the wide array of data structures in which trading data is stored, making it unnecessary for IT staff to optimize data before analysis.|
|Intelligent||Gives compliance staff the flexibility to modify surveillance models based on patterns uncovered and better adapt to emerging threats.|
At NYSE Euronext, a global trading and technology provider, data experts found that their ability to uncover patterns buried deep within billions of trades required a simple approach to a complex problem.
Based in New York City, NYSE Euronext was formed in 2007 when the New York Stock Exchange (NYSE) acquired Euronext, creating the first trans-Atlantic stock exchange and the world’s largest stock market.
In addition to the NYSE, the organization operates stock exchanges in Paris, Brussels, Amsterdam and Lisbon. NYSE Euronext also offers financial products and services in cash equities, futures, options, exchange-traded products, bonds, market data, and commercial technology solutions.
“Everything we do is about analyzing information and looking for a ‘needle in a haystack’,” says Emile Werr, head of Enterprise Data Architecture and vice president of Global Data Services for NYSE Euronext. “We currently process approximately two terabytes of data daily, and, by 2015, we expect to exceed 10 petabytes a day. So we must select the appropriate technologies to analyze these huge volumes in near real time.”
Old approaches obstructed insight
Several years ago, NYSE Euronext had used traditional database technology to drive analytics. But with steadily growing data volumes, data analysts found it increasingly difficult to process the data—especially when searching for questionable trading activity.
“The biggest problem we had was in market surveillance,” says Werr. “Finding manipulation is complex because you need to sift through and correlate large amounts of information. Most relational database technologies don’t do that well so you have to break your data into pieces and apply various programming models, which requires a lot of manual work.”
Better business outcomes—faster
Teaming with IBM, NYSE Euronext replaced its third-party database software with the IBM® Netezza® 1000 data warehouse appliance. With the IBM Netezza data warehouse appliance, the exchange sped up and simplified analytic processes for capacity planning, operational assessment, and compliance and regulation. For example, NYSE Euronext reduced the time to run market surveillance algorithms by more than 99 percent. As a result, surveillance experts could design and test many more algorithms to spot emerging trends in insider trading and market manipulation.
“Detecting manipulation on a trading floor, anticipating market changes, and identifying deficiencies in our system—these are all things that help us shape our business and our future,” says Werr. “Prior to our deployment of the Netezza appliance, these activities took hours, and sometimes exceeded a 24-hour window, which meant we couldn’t meet our SLAs. Once we implemented Netezza, we streamlined data ingestion and analytic processes, and it opened up a lot of doors for us.”
The journey to Smarter Computing
Moving its analytic solutions from a traditional database environment to a powerful, integrated data-ready platform helped NYSE Euronext reduce the time and cost IT staff spent delivering analytic services to the business. These savings include a 35 percent decrease in the number of IT resources required to support the solution.
“At a minimum, we are 200 percent more productive and our productivity levels continue to grow,” says Werr. “Before, we spent a lot of money and development time trying to optimize our existing database. Within two weeks of deploying the Netezza data warehouse appliance, we had a multi-terabyte data warehouse up and running, and we only needed two resources to accomplish this.”
IBM PureSystems™ offering to deliver near-real-time insight and reduce costs
According to Werr, the IBM PureData™ System for Analytics platform, powered by Netezza technology, will help staff even more quickly and efficiently analyze data—even as the amount of data expands from terabytes to petabytes. The PureData System for Analytics platform is also expected to help the organization create a self-service environment that dramatically speeds time-to-market and lowers costs for new business intelligence activities.
“The PureData System for Analytics delivers the flexibility and agility our businesses need to operate on their own,” says Werr. “IBM has optimized the various pieces so we have less moving parts. It reduces costs because we have fewer technologies to support and we need fewer people to support the systems.”
- Reduced the time required to run market surveillance algorithms by more than 99 percent
- Decreased the number of IT resources required to support the solution by more than 35 percent
- Helped compliance personnel to detect suspicious patterns of trading activity early, thus minimizing damage to the investing public
The inside story: Getting there
The work currently underway in the Global Data Services division will likely have far-reaching implications across the financial sector.
Werr explains: “NYSE Euronext represents multiple companies and we also are in the business of selling technology. So the expertise that we build is used on a much broader scale in the marketplace.”
As Werr’s team works to create “big data” solutions that help NYSE Euronext and its clients gain actionable insight from their data, it is focused on creating flexible, agile and easy-to-manage solutions.
“The whole aspect of data management is a serious problem within the big data space,” says Werr. “As we build solutions for our clients, we want to provide data domain knowledge that addresses data integration, data access, user provisioning and self-service out-of-the-box.”
For more information
To learn more about how IBM can help you transform your business, please contact your IBM sales representative or IBM Business Partner.
For more information about NYSE Euronext, visit: www.nyx.com
Products and services used
IBM products and services that were used in this case study.
© Copyright IBM Corporation 2013 IBM Corporation Software Group Route 100 Somers, NY 10589 Produced in the United States of America March 2013 IBM, the IBM logo, ibm.com, Netezza, PureData, and PureSystems are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml Netezza is a registered trademark of IBM International Group B.V., an IBM Company. This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates. The performance data and client examples cited are presented for illustrative purposes only. Actual performance results may vary depending on specific configurations and operating conditions. It is the user’s responsibility to evaluate and verify the operation of any other products or programs with IBM products and programs. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided. Actual available storage capacity may be reported for both uncompressed and compressed data and will vary and may be less than stated.