Using emerging technology to enable smarter, better, faster data analytics
Recently, IBM released a study in which 2,598 CIOs were asked about what was keeping them up at night. It turns out it's data--and the challenge in understanding what that data is telling them. 83% of them identified business intelligence and analytics as their top answer when asked "what kind of visionary plans do you have for enhanced competitiveness?" This, of course, fits within a broader initiative at IBM, something we're calling New Intelligence--and it's part of IBM's greater Smarter Planet initiative. But what does that mean? And why will it have a significant impact on businesses as we move from the information economy to the insight economy?
How Big is the Data Challenge?
In 2010, IDC estimates that 988 exabytes of information will be generated--in that single year. An exabyte is a one followed by eighteen zeros. That's over 18 million times the amount of information contained in every book ever written over the course of human history (Source: IDC, "An Updated Forecast of Worldwide Information Growth Through 2011", March 2008). That's a lot of information--it's called "Big Data" for a reason. What's even more challenging is figuring out how to handle the volumes of data that's being created--and then taking the next step and actually analyzing it for insights.
OK, that's a lot of info. Now what?
Imagine the amount of data which will be generated from customer interactions, supply chain vendors, partners, and logistic systems, and you get a sense of why companies are increasingly becoming aware of the need to start investigating how they can get their arms around this challenge. So, the tidal wave of information is headed our way. Now what do we do about it? First, there will be a need to figure out how not to drown in the information overload. Companies who figure out not only how to deal with this data, but how to leverage the information contained in it will provide themselves a significant competitive advantage: afterall there is meaning in all of that data. There are insights to be revealed. And for businesses, most importantly, there are opportunities.
Addressing the issue:
emerging technologies
What we've been focused on at IBM's Emerging Internet Technologies is how build the tools which will allow businesses not to just survive the flood of information, but rise above it--to carry the analogy further--to trawl the water for valuable catches that will yield them tangible benefits. We're creating and are leveraging a suite of emerging technologies: M42 and Hadoop to help companies gather the information, distill it, and process it, languageWare to help with analyzing the data, and blue spruce to help companies take advantage of the information created to collaborate in new and innovative ways. Because it's not just about handling the data--it's what you do with it.
How Big is the Data Challenge?
In 2010, IDC estimates that 988 exabytes of information will be generated--in that single year. An exabyte is a one followed by eighteen zeros. That's over 18 million times the amount of information contained in every book ever written over the course of human history (Source: IDC, "An Updated Forecast of Worldwide Information Growth Through 2011", March 2008). That's a lot of information--it's called "Big Data" for a reason. What's even more challenging is figuring out how to handle the volumes of data that's being created--and then taking the next step and actually analyzing it for insights.
- "In 2010...information will be generated...that's over 18 million times the amount of information contained in every book ever written over the course of human history."
OK, that's a lot of info. Now what?
Imagine the amount of data which will be generated from customer interactions, supply chain vendors, partners, and logistic systems, and you get a sense of why companies are increasingly becoming aware of the need to start investigating how they can get their arms around this challenge. So, the tidal wave of information is headed our way. Now what do we do about it? First, there will be a need to figure out how not to drown in the information overload. Companies who figure out not only how to deal with this data, but how to leverage the information contained in it will provide themselves a significant competitive advantage: afterall there is meaning in all of that data. There are insights to be revealed. And for businesses, most importantly, there are opportunities.
Addressing the issue:
emerging technologies
What we've been focused on at IBM's Emerging Internet Technologies is how build the tools which will allow businesses not to just survive the flood of information, but rise above it--to carry the analogy further--to trawl the water for valuable catches that will yield them tangible benefits. We're creating and are leveraging a suite of emerging technologies: M42 and Hadoop to help companies gather the information, distill it, and process it, languageWare to help with analyzing the data, and blue spruce to help companies take advantage of the information created to collaborate in new and innovative ways. Because it's not just about handling the data--it's what you do with it.
 | |
 | Press/Media Pack Rod's personal biography and a photograph suitable for publication. |
 | |
Moving Forward.
But there's another disruptive factor emerging in the IT landscape, a new wave of content-centric web applications. Enterprises are beginning to leverage these applications (or "apps") for broader business intelligence coverage, while at the same time taking advantage of the more sophisticated nature of these apps to create richer visualizations, smarter analytics, and even tying together apps to form platforms. These platforms--formed of widgets, larger apps, and back-end systems, are being tied together with deep analytic engines, and in turn are generating even more data--data which can be correlated and compiled to allow for even deeper insights. And these capabilities are being made available via web-based applications and their platforms. It's a vision that's rapidly coming into focus, and we've been involved in some technologies--some which we've invented, such as M42, and some of which we are leveraging from the Open Source community, like Hadoop--to create the first iterations of these novel, yet significantly powerful data analytic systems.
 | |
| One of the fastest growing fields for college graduates is statistics. And they're being paid well--up to $125,000 right out of college. We believe that there is a very good reason for this.
Source: NY Times, August 5, 2009 | |
|
Business Value Being Realized, Today.
Incredible pressures have been placed into the marketplace due to the recent economic challenges, demanding that companies not only execute smarter, but also in increasingly shorter time horizons--and with offerings that are one step ahead of their competitors. In some ways, it's no different than the challenges businesses have traditionally faced--with one caveat: businesses are being flooded with information--but that information can be leveraged as an opportunity. Earlier, we noted that 83% of CIOs think business intelligence and analytics is important to their companies. We agree. And it's why we're exploring technologies to aid business perform business intelligence and analytics smarter, faster, and better. In the process we're continuing with to explore emerging technologies--and in some cases inventing it--to see how those technologies can be applied in real, tangible ways, to address real and tangible challenges businesses face today...and will face tomorrow.
 | |
 | |
Previous Presentation (Hadoop World 2009): | |
 | Download the Presentation (4.9MB) |
 | |
 | |
| The Latest from Rod | |
|
| |
| |
| For More Information Interested in getting started with emerging technologies? Contact Rod and the jStart team. | |
| |