Empowering school districts to assess and improve performance

Published on 30-Jul-2010

"IBM SPSS Modeler is now providing school districts with information that they can use to intervene earlier in order to enhance student achievement or graduation outcomes... The analyses provided by Modeler help districts set their priorities so that programs and services are targeted to those who need it the most – which saves time and money." - Dr. Nicole Catapano, Coordinator of Data Analysis, WSWHE Board of Cooperative Educational Services

Customer:
Board of Cooperative Educational Services (WSWHE BOCES)

Industry:
Education

Deployment country:
United States

Solution:
BA - Business Analytics, BA - Business Intelligence, Information On Demand, Leveraging Information

Smarter Planet:
Smarter Education

Overview

The Washington-Saratoga-Warren-Hamilton-Essex BOCES (WSWHE BOCES) serves 31 school districts in a five-county region. One of many BOCES throughout New York State, the WSWHE BOCES partners with member districts to provide cost-effective shared programs and services that strengthen the quality of living and learning in the communities each district services.

Business need:
Like many K-12 organizations, WSWHE BOCES faces resource and budget challenges. Its Data Analysis Service seeks to help 31 school districts prioritize the allocation of resources and support students. With ever-increasing volumes of assessment and survey data to manage and analyze, the analysis team sought a faster, more cost-effective way to deliver its services.

Solution:
IBM® SPSS® solutions for statistical analysis, data mining and text analysis help WSWHE BOCES automate many routine analytical functions and quickly and accurately analyze open-ended text responses. As a result, the data analysis team has been able to significantly expand its service offerings, cut the cost of service delivery, and reduce the time required to complete client projects.

Benefits:
Enables student profiling that helps districts take action to improve academic performance and minimize dropouts. Sets priorities for special programs and services and identifies the students who are likely to benefit most from them – saving time and money for the districts. Provides faster analysis of open-ended survey questions, allowing districts to obtain timely information for strategic planning and qualify for critical funding. Enables new services that create new revenue streams: as a result, IBM SPSS Modeler delivered full return on investment within three months.

Case Study

Instrumented

The solution draws both structured and unstructured data from a multitude of IT systems across 31 school districts in New York state, and brings it into a central location for analysis.

Interconnected
Data are cleaned, normalized and loaded into a set of sophisticated statistical models for the Data Analysis Service to analyze. Text mining capabilities enable rapid analysis even of unstructured data.

Intelligent
Schools districts can profile their student population and identify students who are likely to have literacy problems, poor academic results, or drop out. Using this insight, the districts can intervene with special programs to get these students back on track.


The Washington-Saratoga-Warren-Hamilton-Essex BOCES (WSWHE BOCES) serves 31 school districts in a five-county region. One of many BOCES throughout New York State, the WSWHE BOCES partners with member districts to provide cost-effective shared programs and services that strengthen the quality of living and learning in the communities each district services.

The WSWHE BOCES Data Analysis Services department is a shared service that assists school districts with collecting and analyzing data to identify strengths and weaknesses at the district, school or student level. Much of the analysis and reporting is designed to enable districts to understand the connection between New York State Learning Standards and assessments, as well as comply with the requirements outlined in the federal No Child Left Behind Act.

The Data Analysis Services department relies largely on a pay-per-project basis. With increased efficiency, it can take on more projects without incurring additional staffing costs.

Software addresses the need for speed
Prior to implementing IBM SPSS Statistics, the three-person staff performed a significant amount of labor-intensive data preparation and hand-entered survey and assessment data into Microsoft® Excel®. These manual processes were becoming too time-consuming in light of the heavy volume of data the service handles each year, which averages about 4,000-5,000 individual surveys and workshop evaluations and 50,000 student exams.

IBM SPSS Statistics Base has streamlined the analysis and reporting process, because it enables analysts to load the data much more quickly and use time-saving features such as automated data preparation. In addition to improved efficiencies, this will eventually enable the Data Analysis Services team to customize project services to a level unobtainable in the past.

Large datasets of survey and assessment information also need to be merged from disparate sources. Among the 31+ districts WSWHE BOCES serves, anywhere from eight to ten student management systems may be used to house data needed for answering important research questions. The WSWHE BOCES team also receives large raw data files from a regional data center. These files frequently contain demographic and program-level data as well as assessment results.

“Without Statistics, we would need to reformat, recode and merge each file individually, a process that could take months. But now this work can be accomplished in several hours, tops,” noted Dr. Nicole Catapano, WSWHE BOCES coordinator for data analysis.

Text analysis reveals insight into open-ended responses
One way WSWHE BOCES discovered it could streamline the analysis process was to find a more efficient way to handle survey responses. IBM SPSS Text Analytics for Surveys processes the data that have been collected through several thousand surveys per year, including those administered to students, staff, parents and community residents. While student and staff surveys are primarily web based – enabling data to be downloaded directly into Microsoft Excel and pulled quickly into Statistics – many districts also rely on traditional paper questionnaires to collect feedback quickly following events such as faculty workshops. All paper-based responses, including free-form text items, are hand-entered into Statistics, but detecting nuances in text answers was often difficult or impossible due to the volume of responses.

Adding IBM SPSS Text Analytics for Surveys gave the WSWHE BOCES team the tools it needed to effectively assess open-ended survey items. For example, when analyzing a workshop evaluation administered to teachers from five districts, Catapano’s team was able to process more than 350 responses in five to seven minutes, both at the group level and as individual results, shaving weeks off the previous turnaround time.

In another survey project in which students were asked to describe the types of extra help they required, the team analyzed 512 text responses in about 10 minutes.

Modeling capabilities increase productivity, revenue
Several years ago, the Data Analysis Services department thought it had reached the limits of what it could do, both from a resource and technology standpoint. But then some of Catapano’s former colleagues in higher education approached her for insights on how to handle data analysis for a collaborative project between New York colleges and universities and several BOCES, aimed at preparing K-12 students for technical careers. In the past, these schools’ difficulty in aggregating and reporting on their data had cost them critical funding, and administrators wanted to be sure they succeeded with this project.

Catapano recommended IBM SPSS Modeler to pull in datasets from disparate sources and come up with one model and set of results. The consortium members were so impressed that they asked her team to run the data for the project – even though she didn’t have the software yet. This project served as justification for purchasing Modeler, which enabled the analysts to do what previously would have taken two to four weeks in two to four hours.

“We couldn’t have offered this as a solution before getting Modeler, which has provided us with a significant new revenue stream that enabled us to pay for the software in the first three months,” noted Catapano, adding that WSWHE BOCES is now able to take on more types of work for more districts. “We can build many more types of models, and have opened districts’ eyes to the types of questions they can ask of their data.”

Predictive insight
Catapano adds: “IBM SPSS Modeler is now providing school districts with information that they can use to intervene earlier in order to enhance student achievement or graduation outcomes. Instead of scanning pages of descriptive data and guessing about relationships between the data elements, the analyses provided by Modeler help districts set their priorities so that programs and services are targeted to those who need it the most – which saves time and money. We have started to apply the predictive models to current students in order for districts to see what the expected outcomes would be if no interventions were provided.”

IBM SPSS Modeler is also helping the Data Analysis Service identify predictors of early literacy skills, predictors of dropouts, and profiles of students who are in career and technical programs from high school to college.

“For one district, we built a longitudinal system to investigate patterns in student high school outcomes,” comments Catapano. “We subsequently used this model to generate student profiles that will be applied to other cohorts in order to identify students who might benefit from specific programs or courses of study. For another district, we are in the process of collecting ‘round two’ early literacy data, which we will feed through the model generated last year. This will help us predict student performance on state assessments, and the results can be used to identify students who are at risk of not meeting learning standards so that literacy interventions can begin as early as possible.”

About IBM Business Analytics
IBM Business Analytics software delivers complete, consistent and accurate information that decision-makers trust to improve business performance. A comprehensive portfolio of business intelligence, predictive analytics, financial performance and strategy management, and analytic applications provides clear, immediate and actionable insights into current performance and the ability to predict future outcomes.

Combined with rich industry solutions, proven practices and professional services, organisations of every size can drive the highest productivity, confidently automate decisions and deliver better results.

As part of this portfolio, IBM SPSS Predictive Analytics software helps organisations predict future events and proactively act upon that insight to drive better business outcomes. Commercial, government and academic customers worldwide rely on IBM SPSS technology as a competitive advantage in attracting, retaining and growing customers, while reducing fraud and mitigating risk. By incorporating IBM SPSS software into their daily operations, organisations become predictive enterprises – able to direct and automate decisions to meet business goals and achieve measurable competitive advantage.

Products and services used

IBM products and services that were used in this case study.

Software:
SPSS Modeler, SPSS Text Analytics for Surveys, SPSS Statistics Base

Legal Information

© Copyright IBM Corporation 2011. IBM Global Services, Route 100, Somers, NY 10589, U.S.A. US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Produced in the United States of America. May 2011. All Rights Reserved. IBM, the IBM logo, ibm.com, WebSphere, InfoSphere and Cognos are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at: ibm.com/legal/copytrade.shtml. SPSS is a businessmark of SPSS, Inc., an IBM Company, registered in many jurisdictions worldwide. Other product, company or service names may be trademarks or service marks of others. References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates.