Bavarian University of Applied Sciences

Optimising education quality with IBM predictive analytics software

Published on 27-Apr-2012

Validated on 02 Dec 2013

"The IBM solution is an easy-to-use and highly efficient instrument for the continuous evaluation of lecturers, education quality and basic university conditions. The programs run incredibly effectively and quickly." - Ulrich Hebauer, Member of Controlling Unit, Bavarian University of Applied Sciences

Customer:
Bavarian University of Applied Sciences for Public Administration and Judicature

Industry:
Education

Deployment country:
Germany

Solution:
BA - Business Analytics, Business Integration, Optimizing IT, BA - Predictive Analytics

Overview

The Bavarian University of Applied Sciences for Public Administration and Judicature (Fachhochschule für öffentliche Verwaltung und Rechtspflege in Bayern, or FHVR) is responsible for providing higher education to Bavarian civil servants in six main branches of study: General Administration, Archiving and Librarianship, Finance, Police, Judicature and Social Administration. The university employs approximately 185 administrative officers, as well as some 165 official and 180 extra-official lecturers, who teach around 3,500 students at eight places of study across Bavaria.

Business need:
In order to maintain and improve the high quality of the education it provides, the Bavarian University of Applied Sciences for Public Administration and Judicature is legally obliged to regularly survey its students, lecturers and administrative staff on syllabus content, teaching methods and general conditions. Using paper-based questionnaires would be inefficient and time-consuming, so the university decided to conduct its evaluation surveys online.

Solution:
For several years, the university has been using predictive analytics software from IBM to create a huge number of regular online surveys. The software allows the university to quickly and easily create and evaluate thousands of questionnaires every year.

Benefits:
Ensures that students are being taught relevant subject matter, and that they receive high levels of instruction under optimum conditions. Provides valuable insight into the condition of the educational institution, the quality of education and the performance of lecturers, thanks to regular evaluations by means of online surveys. Allows quick and targeted follow-up action to be taken in response to the evaluations. Saves up to the equivalent of two full-time employees by using online surveys instead of traditional paper-based questionnaires.

Case Study

To read a German version of this case study, click here.

The Bavarian University of Applied Sciences for Public Administration and Judicature (Fachhochschule für öffentliche Verwaltung und Rechtspflege in Bayern, or FHVR) is responsible for providing higher education to Bavarian civil servants in six main branches of study: General Administration, Archiving and Librarianship, Finance, Police, Judicature and Social Administration. The university employs approximately 185 administrative officers, as well as some 165 official and 180 extra-official lecturers, who teach around 3,500 students at eight places of study across Bavaria.

In order to ensure that the high quality of education at FHVR is maintained, the Free State of Bavaria has enacted a law for the safeguarding of education at the university, which requires FHVR to perform regular, anonymous evaluations. These evaluations cover lecture and syllabus content, teaching quality and general campus conditions. Every department and lecturer needs to be evaluated at least once per year.

Impractical, paper-based processes
The university quickly determined that it would be highly impractical and inefficient to use traditional surveying methods based on paper forms. FHVR calculated that it would require two full-time employees to record the results of the 12,000 questionnaires it is required to send out every year. Using manual processes would also incur additional effort and expenses around printing, distributing and collecting the questionnaires, as well as evaluating the feedback. Furthermore, the procedure would be highly prone to error and would result in a lengthy delay between survey completion and the publication of feedback – a delay that the university could not afford.

FHVR wanted to be able to perform surveys quickly and efficiently, without significant expense, and to ensure that the results would be recorded as accurately as possible. It considered several alternatives, and decided to conduct its evaluation surveys online, with the help of IBM SPSS software.

Continuous support for education with IBM SPSS
FHVR has relied on IBM SPSS software to fulfil its legal obligations for a number of years. Ulrich Hebauer, employee in the controlling staff unit, describes his experiences with the software as follows: “We create our questionnaires with IBM SPSS Data Collection Author Professional. The questionnaires differ slightly, depending on whether it’s a question of a lecturer’s evaluation, an assessment of the educational content of a course, or an evaluation of basic conditions. We start with a number of standard templates, and individual faculties can then pick and choose the exact questions they want to include on their questionnaires.

“We produce a large number of questionnaires on a regular basis: for example, this year alone we have created 7,000 questionnaires for more than 300 lecturer evaluations, and another 4,000 questionnaires for the evaluation of basic education conditions. Every single one of these must be edited individually. IBM SPSS helps us to do this quickly and accurately with its straightforward user interface and various automated features. It also enables us to keep expenses within reasonable limits.“

After being created, every questionnaire is posted online via the IBM SPSS Data Collection Web Interviews Server. Invitations to the online survey are usually sent out by e-mail to the required participants. An individual pin code ensures that no one can fill out the questionnaire more than once, and the process is completely anonymous. As most of the students at FHVR are studying to become public servants, they can generally be relied upon to return their questionnaires in a timely manner – leading to a high response rate of more than 70 percent.

Rapid evaluations reveal new insights
“We extensively evaluate the survey responses with IBM SPSS Statistics,” explains Ulrich Hebauer. “Now that we have been running the application for a number of years, we have built in approved job files for automatic evaluation runs, which require no manual interventions in the SPSS syntax. This helps to considerably speed up questionnaire analysis. Another advantage is that the records are produced directly in the original SPSS data format after the survey, which saves us a huge effort on converting the results.”

The results of these questionnaires can be very informative. By evaluating feedback on basic campus conditions by students and employees, the university can determine whether a catering company at one of its cafeterias should be replaced because of poor service or food quality. It can also learn if larger lecture rooms are needed for certain lectures, or if the library needs to purchase more copies of a particular textbook because of increased demand. Common statistical measures of responses can be used to accurately pinpoint trends.

Optimising lecture quality
The results of lecturer evaluation are especially informative. Students at FHVR generally provide very detailed assessments, which include information on the lecturer’s methods of knowledge transfer, professional and didactic competence, engagement and helpfulness. Students can also fill in further comments at the end of the survey, such as suggestions for future lecture topics.

Because of confidentiality requirements, the results of lecturer evaluations are shared with a small, well-chosen circle of the representatives and – in an anonymised form – to the professional faculty management. The results must be interpreted with the greatest care because they form the basis for the continuous improvement of educational quality at FHVR.

Interesting conclusions can be drawn by creating rankings of all the lecturers, or by comparing students’ responses to identical lectures given by different lecturers. Every lecturer receives her or his own evaluation, which also contains a comparison with all other lecturers. According to the evaluation order, all teachers are obliged to discuss their assessments afterwards with their students and to work on any weak spots that may have been identified.

In the event of significant deviations, the evaluation panel can choose to utilise special measures to help individual lecturers, such as additional seminars or personal coaching sessions. As a result, the lecturers’ qualifications and the overall quality of education can be systematically increased.

Impressive ease-of-use and performance
Ulrich Hebauer states, “The IBM solution is an easy-to-use and highly efficient instrument for the continuous evaluation of lecturers, education quality and basic university conditions. The programs run incredibly effectively and quickly.”

He is also pleased with the performance and reliability of the solution: “IBM SPSS Data Collection Web Interviews Server only needs very few resources, as it runs on a virtual server. Although an extensive evaluation run might take as much as two hours, the high degree of automation built into the process means that this is normally not a problem. For example, we can launch the program before lunch and the results are ready early in the afternoon. With this solution we will be able to maintain the high education quality at FHVR and work to continuously increase it in the future.”

Maintaining high standards
On account of the success of its surveying activities, the university’s small research office has been receiving more and more attention. “We have been getting a lot of queries from other government departments about whether we can conduct similar surveys for them as well,” reports Ulrich Hebauer.

“For example, we recently surveyed approximately 1,000 graduates from the police service about the relevance of our study program to their current work. The results are still being evaluated, but the initial responses seem to be very promising. In the future, surveys like this will certainly help us to adapt the educational activities we offer and make them even more effective.”

About IBM Business Analytics
IBM Business Analytics software delivers actionable insights decision-makers need to achieve better business performance. IBM offers a comprehensive, unified portfolio of business intelligence, predictive and advanced analytics, financial performance and strategy management, governance, risk and compliance and analytic applications.

With IBM software, companies can spot trends, patterns and anomalies, compare “what if” scenarios, predict potential threats and opportunities, identify and manage key business risks and plan, budget and forecast resources. With these deep analytic capabilities our customers around the world can better understand, anticipate and shape business outcomes.

Products and services used

Legal Information

© Copyright IBM Corporation 2012 IBM Deutschland IBM-Allee 1 D-71139 Ehningen Germany Produced in the Germany April 2012 IBM, the IBM logo, ibm.com, and SPSS are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. A current list of other IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml. Other company, product or service names may be trademarks, or service marks of others. References in this publication to IBM products, programs or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM product, program or service is not intended to imply that only IBM’s product, program or service may be used. Any functionally equivalent product, program or service may be used instead. All customer examples cited represent how some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. IBM hardware products are manufactured from new parts, or new and used parts. In some cases, the hardware product may not be new and may have been previously installed. Regardless, IBM warranty terms apply. This publication is for general guidance only. Photographs may show design models.