Published on 23-Oct-2013
"This is why pundits extol the virtues of DevOps—smart coordination across the teams' three dimensions of core technology, client solutions and managed services can have amazing impact." - Carl Kraenzel, director of the IBM Watson managed services group, IBM Distinguished Engineer, IBM
Agile Software Development, Business Continuity, DevOps
A global computer products, services and solutions company based in the United States, IBM operates in more than 170 countries and has nearly 427,000 employees.
To develop leading-edge client solutions based on the IBM® Watson™ cognitive computing platform, IBM needed to establish software development and deployment practices to support constant, rapid change.
The IBM Watson teams used a DevOps approach, including IBM Rational® software, to support continuous delivery of complex new releases.
The teams are moving from 9-week to 3-week sprints and pushing every sprint release to production, resulting in three times greater delivery velocity, reduced costs and fewer disruptions for clients.
A global computer products, services and solutions company based in the United States, IBM operates in more than 170 countries and has nearly 427,000 employees. With thousands of business partners and computer products suppliers worldwide, IBM generates more than USD99 billion in annual revenue. The company was founded in 1911 in Endicott, New York.
Applying cognitive computing to real-world problems
In 2011, the IBM Watson computer gained widespread attention by beating two of the all-time champion human contestants, Ken Jennings and Brad Rutter, on the TV show Jeopardy! Descended from a long line of IBM computers including Deep Blue, the supercomputer that won a chess match against chess grandmaster Garry Kasparov in 1997, IBM designed the IBM Watson computer for three primary purposes: natural language processing, hypothesis generation and evidence-based learning. Armed with 200 million pages of structured and unstructured content, including the full text of Wikipedia, the IBM Watson platform defeated the Jeopardy! champions in three days of play.
After the victory on Jeopardy!, IBM began to look for ways to apply the groundbreaking capabilities of the IBM Watson platform, which is based on IBM Power® 750 servers, to real-world problems. IBM is developing offerings based on the IBM Watson platform that will enable users to interact with the IBM Watson computer on any device at any time in plain English to get personalized responses to questions and receive actionable insight with supporting evidence. In the same way that the IBM Watson computer used its vast stores of knowledge to respond to Jeopardy! clues, now healthcare organizations, financial institutions and research organizations can use the platform to get questions answered and support better decision making.
In May 2013, IBM announced the consumer-facing IBM Watson Engagement Advisor solution, an offering that enables companies to use the IBM Watson platform to answer contact center questions. The IBM Watson offering is unlike anything IBM has done in the past, which is both an opportunity and a challenge. “We have the strength and the weakness of having no precedent to follow or modify,” says Carl Kraenzel, the director of the IBM Watson managed services group and an IBM Distinguished Engineer. The IBM Watson teams, like many development organizations, are dealing with limited resources. “We’re small teams relative to what we need to do,” says Kraenzel. So it was essential for the group to approach development strategically.
To facilitate fast scaling of the initial consumer offering and subsequent development of additional solutions based on the IBM Watson platform, the IBM Watson teams needed to make sure they were equipped to handle a constant force shaping the IBM Watson platform and, in reality, every software development organization: the force of change.
Managing change to support continuous delivery
Change plays a large role in the ongoing development of the IBM Watson solution as a service. In fact, for the IBM Watson platform to meet companies’ needs, it must be learning all the time. “It’s constantly evolving,” says Kraenzel. “Compared to any other technology we have in the IBM portfolio and potentially in the marketplace, this is a completely new class of technology. There are very few examples of technology whose value proposition is so closely linked to constant evolution. Watson is meaningless if it doesn’t constantly change.” That’s why the IBM Watson teams are focused on properly managing change through a DevOps approach, which uses lean principles and agile development methods. “If you don’t approach it the right way, change is a cost driver. It will add cost instead of adding value,” says Kraenzel. “DevOps helps us achieve continuous delivery, and DevOps helps us deliver continuous value.”
Implementing a triple active approach
As part of the DevOps approach, the IBM Watson teams embraced a form of continuous availability that they call triple active. “It’s a form of using disaster recovery principles in a way that gives you disaster recovery but also gives you other benefits. Smooth deployment without interruption, otherwise known as continuous delivery and continuous availability,” says Kraenzel. The triple active architecture is based on triple redundancy, which enables the teams to update and roll back the system without disrupting service.
Before implementing the triple active architecture, the teams needed a maintenance window for system updates, which meant that the system was down for a period of time. “The triple active architecture allows us to take one of the three legs of that architecture down and upgrade it while the other two are still answering questions, and then proceed to upgrade the remaining systems,” says Bill Hume, director of IBM Watson core technology with the IBM Watson solutions team. Similarly, the teams can back out of an update if there are issues. “This is why pundits extol the virtues of DevOps—smart coordination across the teams' three dimensions of core technology, client solutions and managed services can have amazing impact,” says Kraenzel.
The triple active architecture will become even more important as IBM supports new IBM Watson platform-based solutions and adds clients. “We’re starting to build bigger solutions that need to scale,” says Jeff Eisen, chief architect of IBM Watson core technology and IBM Distinguished Engineer. “So it’s really important that the cost of operations, cost of delivery and speed of delivery per customer is low. DevOps is important for us to be able to roll out quickly for lots of customers fast.” Ultimately, the Watson team aims to have a continuous delivery solution that serves millions of clients. “We’re trying to move to a point where we have lots of customers. I keep telling the team to think ‘Turbo Tax,’” says Eisen. To meet these goals, the team must embrace methods that will continue to serve them in the long term. “We’re trying to establish principles now that will allow us to achieve our quality and service level objectives and value delivery objectives a year from now,” says Hume.
Becoming increasingly agile with IBM Rational software
The Watson teams are using IBM Rational Team Concert™ software to support agile development within their overall DevOps approach. For the IBM Watson groups, agile is about more than just software development. “It’s not just agile software development, but agile principles applied to managed service delivery, and our approach to licensing and selling our solutions,” says Kraenzel. “It’s across the board. We’ve had to look at things and ask, is that resilient to constant change in a way that the change doesn’t bring in cost, rather it amplifies value?”
"The IBM Watson solutions development team is currently using the planning and tracking functionality of IBM Rational Team Concert software to fully describe what a code release will contain, from the perspective of an entire product release down to each sprint release,” says Guy Slade, development manager, IBM Watson solutions, IBM. “It enables us to better control and understand what each team member is going to be doing during a development sprint and how loaded from a time perspective each team member is. As the sprint progresses, this functionality allows us to have a tight, real-time view on how well we are progressing and to load balance work on the fly, correcting trajectory so we increase the overall success of the development effort.”
The Rational Team Concert software also provides traceability through a self-documenting environment that enables staff to go back and view requirements, as well as the sprint in which the requirements were addressed. “That’s why Rational Team Concert is so critical,” says Hume. “Because user stories, which are a reflection of our requirements, are documented, tracked, clarified and understood within Rational Team Concert.” When the teams began using agile practices, they worked in 9-week sprints. Now they are moving to 3-week sprints and pushing every sprint release to production. This increased velocity enables the IBM Watson teams to rapidly deliver functionality that closely matches client expectations in compact, near-frictionless increments that minimize the costs and disruption of change for clients.
Achieving continuous innovation through a cultural shift
The DevOps approach requires more than software tools. Culture and process changes are also essential for a successful DevOps approach. IBM formed three teams to develop the IBM Watson platform as a cognitive computing, Software as a Service (SaaS) solution: core platform development, solutions development and managed services operations. Collaboration is the key to making sure the teams meet their frequent release schedules. The three IBM Watson teams are closely aligned, with few boundaries among them, working together in a value-chain factory. Investing in cross-team education helps foster awareness, so teams can work together to help improve IBM Watson solutions. “The DevOps approach promotes the idea that the business stakeholders, development teams responsible for automating deployments and the operations team are all interconnected and part of a continuous delivery cycle,” says Hume. “That’s how we achieve continuous innovation.”
A critical part of the cultural shift for the IBM Watson teams was the use of collaborative cost modeling. Traditionally, cost models are created and reviewed exclusively by the operations teams that host and run the systems. Developers and development managers rarely see cost models and are almost never asked to help author them. As a result, development teams can unintentionally create code or prioritize features in ways that drive unintended costs, many of which manifest as deployment difficulties and delays. To prevent this issue, the IBM Watson operations team engaged lead software developers and managers as coauthors and equal owners of the cost model. This collaborative approach to cost modeling increases transparency and helps foster a culture in which everyone understands and cares about how their choices affect operations.
About the IBM DevOps solution
The capabilities in this case study illustrate the collaborative development practices of the IBM DevOps solution. The DevOps approach is an enterprise capability for continuous software delivery that enables organizations to seize market opportunities, respond more rapidly to client feedback, and balance speed, cost, quality and risk. By applying lean and agile principles across the software delivery lifecycle, the DevOps approach helps organizations deliver a differentiated and engaging client experience, achieve quicker time to value and gain increased capacity to innovate. The IBM DevOps solution uses an open-standards-based platform and a continuous feedback loop across the delivery process, and it is designed to integrate into existing heterogeneous lifecycle environments to deliver value quickly.
For more information
To learn more about the IBM DevOps approach, please contact your IBM representative, or visit the following website:
Download the analyst report: Continuous Delivery is Reshaping the Future of ALM. https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=swg-rtl-sd-ar&S_PKG=ov17430
Products and services used
IBM products and services that were used in this case study.
Rational Team Concert
© Copyright IBM Corporation 2013 IBM Corporation Software Group Route 100 Somers, NY 10589 Produced in the United States of America October 2013 IBM, the IBM logo, ibm.com, IBM Watson, Power, Rational and Rational Team Concert are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml The content in this document (including currency OR pricing references which exclude applicable taxes) is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided.