A Large US City Delivering uninterrupted services to citizens

Smart is....

Creating a proactive service delivery model that allows IT staff to identify recurring problems and institute proactive measures to prevent service-affecting problems.

For one large US city, outages and performance issues with city-wide applications and infrastructure services were increasing costs and negatively affecting citizen interactions with city agencies. By moving to an Integrated Service Management model for its data center, the city is gaining the end-to-end visibility, control and automation needed to make sense of real-time information and deliver high-quality, uninterrupted services despite tight budgets and headcount constraints.

Smarter Cities: Predicting service problems

Instrumented: Real-time server, network and application information along with configuration and asset information are automatically collected and fed into service models.

Interconnected: Data from 400 servers, 100,000 network devices, 500 database instances and 60,000 telecom circuits is displayed on dashboards showing the health of services.

Intelligent: Ability to identify trends based on the number and severity of problems for each asset class helps IT staff predict and prevent service-impacting problems.

Economic slowdowns. Budget cuts. Security issues. Aging populations. As cities around the world grapple with these problems, they’re finding that the key to their success is in using information technology (IT) to transform how they work, collaborate and serve citizens. But as any organization in any industry knows, IT glitches can negatively affect customer satisfaction and dramatically increase the time and cost associated with service delivery.

For one large US city, a focus on Integrated Service Management that provides end-to-end visibility, control and automation of essential city services is becoming indispensable in its ability to address these challenges.

City commissioners and IT staff were specifically concerned with service levels for 29 critical city services that are used by millions of citizens. These include its:

• Citizen information hotline that provides citizens with fast and easy access to government information and non-emergency services.
• Intranet, which provides a community space and tools for the city’s nearly 300,000 employees to share information and work efficiently.
• Data-sharing service that enables agencies to share case information for criminal justice investigations, trial preparation and case follow up to improve public safety.
• Geographic information system that supports emergency response and planning operations.
• Online payment systems for parking tickets, property taxes, water bills and other city services.
• Online public screening tool that city residents can use to determine eligibility for health and human service benefit programs.

Underpinning these services is an incredibly heterogeneous infrastructure spanning hundreds of agencies and operational teams and comprising more than 400 servers, 100,000 network devices, 500 database instances and 60,000 telecom circuits.

Given the complexity of this infrastructure, it was difficult for support staff to correlate infrastructure issues, find the root cause of a problem, and identify what services were affected. This caused extended performance degradation and outages. It also increased the cost associated with answering calls from employees and citizens.

“We were monitoring the bits and pieces of our core services, but we didn’t have the end-to-end visibility for service modeling and tracking SLA performance,” says the city’s Network Monitoring Team manager. “It meant more calls, higher MTTR [mean time to repair], and extended outage durations.”

“Any large organization with distributed operations spanning multiple groups, agencies or locations have the same challenge in determining how hundreds of thousands of elements relate to the macro services offered to internal and external users,” adds Mohan Kompella, director of channels and technical sales for Softential, an IBM Business Partner that is working with the city to implement Integrated Service Management for its data center.

New insight into quality of service

Working with Softential, the city is creating service dashboards that provide real-time visibility into the health of critical services and their constituent components. The solution, which is based on IBM® Tivoli® Business Service Manager, provides IT and city executives with, for the first time, a view of how each service is operating. Staff can drill down on any problem to quickly pinpoint its root cause and proactively identify and resolve performance degradations before they result in outages.

Staff can also track key performance indicators (KPIs), such as mean time to repair and the number and severity of service impacting problems for each asset class. This will provide a new level of intelligence to enable IT to identify recurring problems and institute proactive measures so as to avoid service-affecting problems in the first place. In phase two of the project, dashboard views will be expanded to track SLA (service level agreement) violations and deliver agency-specific information.

To provide this level of visibility, Tivoli Business Service Manger aggregates and correlates service status from Tivoli software and third party performance monitors in real time. For example, IBM Tivoli Network Manager monitors network performance across more than 100,000 network devices. IBM Netcool®/Impact extracts configuration and asset information from the city’s change management database from BMC Remedy to populate the service models. The software also pulls data from service tickets along with details on the number of tickets for each component. IBM Netcool/OMNIbus collects and consolidates alarms from all devices and element managers. Tivoli Composite Application Manager monitors performance across the application stack.

**********
The inside story: Getting there
A foundation....The city’s journey to an Integrated Service Management model began in 2006 when Softential worked with IT staff to streamline systems and availability monitoring using IBM Tivoli software. Previously, each server, application and network team used point monitoring tools. “The work we conducted with the city using Tivoli software to consolidate systems, network and application alerts laid the foundation,” says Kompella. “Once that real-time data was being aggregated, we could then begin discussions about moving into a service model.

Building a case.... The city began discussions with several vendors in early 2009 regarding the creation of service dashboards based on IT Infrastructure Library® (ITIL®) best practices. “The executive managers in the city’s IT department were looking long term at the service management model but the sentiment wasn’t widespread,” says Kompella. “By building the foundation with IBM Netcool software and consolidated operations management, we started socializing the importance of end-to-end service modeling. This helped sell it to the different stakeholders across city agencies.

A flexible approach.... The city selected IBM over another IT solution provider for its ability to consolidate information across its heterogeneous environment. “The ability of Tivoli Business Service Manager to tap into back-end Oracle and SQL databases was quite useful,” says Kompella. “The other vendor was really focused on the data that its software was already collecting. But Tivoli Business Service Manager could pull from other data sources to give a complete picture and enabled us to customize KPIs with data in external repositories.
**********

Benefits of Integrated Service Management

What difference has this new approach made? Already, IT managers are seeing significant improvements in key performance indicators, such as mean time to repair (MTTR) and service availability. For example, before it would typically take 2 - 3 hours to investigate and manage incidents. Now it takes only an hour—a more than 50 percent reduction in MTTR. Availability for key services currently under management has improved 60 - 90 percent. In fact, previously, the city’s citizen hotline and Intranet services experienced major outages lasting up to four hours about once every three months. Since the implementation of Tivoli Business Service Manager, the city hasn’t had any significant outages to these highly visible services. (The citizen information hotline receives nearly 1.5 million calls a month and the Intranet is used by about 300,000 employees daily.) Additionally, by saving about four hours a month to repair problems associated with its citizen hotline service, the city estimates that it can service an additional 7,757 calls.

“Tivoli Business Service Manager helps us cut down on the low-level noise and make sense of real-time and historical alerts and events that are streaming in every minute of every day,” says the city’s Network Monitoring Team manager.

These improvements are helping the city deliver high quality services despite tight budgets and headcount constraints. “With Tivoli Business Service Manager, cities and other organizations with large, dynamic environments don’t have to constantly expand headcount as they expand their services,” says Kompella.

Business Benefits

• 50 percent reduction in mean time to repair (MTTR)
• 60 - 90 percent improvement in availability of critical services under management
• Prevented major outages lasting several hours on the city’s citizen information hotline and Intranet services
• Enabled the city’s citizen information hotline center to service an additional 7,757 calls per month

Solution Components:
Software

• IBM® Tivoli® Business Service Manager
• IBM Tivoli Composite Application Manager for Applications
• IBM Tivoli Monitoring
• IBM Tivoli Netcool®/OMNIbus
• IBM Tivoli Netcool/Impact

IBM Business Partner

• Softential

For more information

To learn more about how IBM can help you transform your business, please contact your IBM sales representative or IBM Business Partner.

Visit us at: ibm.com/tivoli

For more information about Softential, visit: www.softential.com