Published on 29-Apr-2010
Validated on 07 Oct 2013
"With help from the IBM Hursley team, we quickly found that Informix TimeSeries could deliver spectacular results… You don’t need to understand the technical details – the point is that suddenly, energy monitoring for three million homes or more became a practical proposition." - Clive Eisen, Chief Technical Officer, Hildebrand
Energy & Utilities
Database Management, Development & Technology Adoption, Energy Efficiency, IBM Research, Smarter Planet, Collaborative Innovation, Optimizing IT, Leveraging Information
Headquartered in London, UK, Hildebrand provides creative and technology consulting for service organisations. The company’s technology team specialises in combining innovative technology with real-world experience, turning complex problems into a source of competitive advantage. Hildebrand’s consultants became fascinated by the challenges posed by smart monitoring of home electricity usage, and began a project funded by the Seventh Framework Programme for EU Research (FP7)
As technology consultants on the Digital Environment Home Energy Management System (DEHEMS) project, the Hildebrand team was asked by the UK government to find a way to scale up its energy monitoring solution and enable it to monitor three million homes.
Hildebrand ran several proofs-of-concept at the IBM Hursley Innovation Centre, and created a solution based on IBM Informix technologies that has the potential to collect, store and analyse detailed energy usage information from homes in real-time. Sensors attached to a home’s electricity supply transmit readings over a wireless network to the central DEHEMS database via the Internet. Users can access an online dashboard that displays their electricity usage and performs analytics.
Enables real-time analysis of electricity usage for households, or even for individual appliances, helping people make better decisions about energy efficiency in the home. Can collect, store and analyse up to 50,000 data points per second – demonstrating scalability to three million homes and beyond. Delivers high performance on low-cost hardware by leveraging unique time-series data management technologies.
Headquartered in London, UK, Hildebrand provides creative and technology consulting for service organisations. The company’s technology team specialises in combining innovative technology with real-world experience, turning complex problems into a source of competitive advantage.
Hildebrand’s consultants became fascinated by the challenges posed by smart monitoring of home electricity usage, and applied for funding from the Seventh Framework Programme for EU Research (FP7), proposing a 30-month research project that would allow stakeholders from local authorities, private businesses and universities to study energy monitoring and its effect on human behaviour at ‘live labs’ in five cities across the European Union.
“FP7 proposals are reviewed by a central committee and awarded scores out of fifteen,” explains Clive Eisen, Chief Technology Officer at Hildebrand. “Our proposal received fourteen points, which is almost unheard-of. The intention is to develop and test a home energy management system for the home market, with the aim of reducing CO2 emissions and reducing electricity bills – a topic which is of considerable interest and importance to governments, citizens and the utilities sector across Europe.”
The DEHEMS project
The project, known as the Digital Environment Home Energy Management System (DEHEMS) involved installing small, low-cost energy monitoring devices at groups of homes in five European cities: Birmingham, Bristol and Manchester in the UK, and Plovdiv and Ivanovo in Bulgaria. Each of these groups forms a ‘living lab’, giving the researchers access to real-world energy usage data, and allowing them to study the behaviour of individuals and their attitudes towards energy management.
“The idea was to run the project in three cycles, and use a different type of analysis in each cycle, to see if we could draw any conclusions in terms of how to promote change in the way people view their energy consumption,” says Clive Eisen. “We also wanted to use the data we collected to create complex mathematical models that would allow us to predict energy usage more accurately, even for homes that don’t currently have any energy monitoring equipment.”
“They said it couldn’t be done”
The project attracted the attention of the UK government, which proposed a larger-scale implementation, to three million British homes.
“Scaling up to three million homes is an amazing opportunity for us – but also a tremendous challenge,” comments Eisen. “The problem is the sheer volume of data. Our monitoring system takes a reading from the electricity meter and transmits it to a local wireless hub, which sends it over the Internet to our central database. If three million homes are sending us a new reading once every minute, that means 50,000 new database entries every second! When we talked to other organisations in the smart metering business, most of them thought it couldn’t be done.”
Traditionally, databases struggle to deal with time-series data (i.e. ‘pulses’ of data arriving at regular intervals from one or more sources), because their structure makes it difficult to store and index this data efficiently. As a result, Hildebrand knew that it would be impossible to create a solution around a standard database engine without massive investment in high-end server hardware. The costs of this would be prohibitive, so a new kind of solution was required.
“We did some research into leading-edge database platforms, and came across IBM Informix TimeSeries DataBlade and Real-Time Loader, which are specifically designed to handle time-series data,” says Eisen. “We consulted the software lab at IBM Hursley and they helped us run a number of proof-of-concept projects using the facilities in the IBM Hursley Innovation Centre to see if it would be possible to handle the volume of data we needed in a cost-effective manner.”
The Informix TimeSeries technologies create a single database object for each data-source, and then simply update it with the latest readings whenever a new ‘pulse’ of data arrives. This provides a more manageable data structure, which makes it easier to store, extract and analyse data.
“With help from the IBM Hursley team, we quickly found that Informix TimeSeries could deliver spectacular results,” explains Eisen. “In the first proof-of-concept we simulated three million homes sending readings once a minute, and we were able to capture nearly 40,000 readings per second using only a quad-core, dual-processor Intel server. In the second, we moved to a slightly larger server and found we could deliver analytics response times of between one and three seconds for a load of 50,000 readings per second. You don’t need to understand the technical details – the point is that suddenly, energy monitoring for three million homes or more became a practical proposition.”
The possibilities for a truly scalable energy monitoring solution are vast.
“We’re currently working with a Dutch company, Plugwise, to introduce monitoring of individual appliances,” says Eisen. “This could, for example, help you work out exactly how much your fridge, washing machine and dishwasher cost to run. We tested a seven-year-old fridge and discovered it was costing £250 a year, while a comparable modern A-rated fridge would only cost £50 a year. So if you invest £200 in purchasing a new fridge, you could make the money back in a year, and the next few years could be pure profit.
“Consumers are going to want to know this kind of thing, and it’s a huge opportunity for electricity companies to offer a value-added service. At the same time, both central and local government will be keen to support an initiative that can monitor home electricity usage accurately, and potentially lead to behaviour changes that considerably reduce CO2 output.”
Building a smarter planet
As a more general point, the ability to handle large quantities of time-series data from many sources could have many other applications besides energy monitoring.
“The world is increasingly moving towards managing complex systems through instrumentation – whether it’s home energy monitoring, or tracking logistics with RFID tags, or any other similar application,” concludes Eisen. “We’re showing that collecting and analysing all the data needn’t necessarily be an impossible or impractical problem: with the right technology underneath, it’s possible to get great results without a huge investment in hardware.”
Products and services used
© Copyright IBM Corporation 2010 IBM United Kingdom Limited PO Box 41 North Harbour Portsmouth Hampshire PO6 3AU Produced inthe United Kingdom April 2010 All Rights Reserved IBM, the IBM logo, ibm.com and Informix are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. A current list of other IBM trademarks is available on the Web at “Copyright and trademark information” at: ibm.com/legal/copytrade.shtml. References in this publication to IBM products, programs or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM product, program or service is not intended to imply that only IBM’s product, program or service may be used. Any functionally equivalent product, program or service may be used instead. All customer examples cited represent how some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. IBM hardware products are manufactured from new parts, or new and used parts. In some cases, the hardware product may not be new and may have been previously installed. Regardless, IBM warranty terms apply. This publication is for general guidance only. Photographs may show design models.