The following was originally published on the Grid Insights by Intel energy blog.
Utilities are generating more data than ever before. Smart meters, SCADA equipment, transformer meters and other sensors on the distribution grid are generating hundreds of millions of metrics daily. And that’s not all: data pours in from many other sources, too, including customer information systems, weather and lightning systems, electric vehicle meters, solar panels, geographic information systems, asset management systems, and even animal movement tracking devices.
With all this data, utilities have more insight into their most mission-critical processes than ever before:
- Grid Reliability: Reduce truck rolls, respond to reliability issues, and avoid sustained outages by analyzing momentary outage data.
- Safety Analytics: Boost consumer and grid safety by analyzing meter temperature data in real-time in order to predict transformer and meter fires before they even happen.
- Voltage Optimization: Increase energy efficiency, reduce voltage complaints, and lower customer bills by optimizing voltage levels across the grid using 100 percent of actual premise-level voltage reads.
- Unfortunately, there are a number of obstacles that prevent utilities from taking advantage of all this data:
- It’s too much data for current systems. Traditional IT systems can’t handle the volume and diversity of data that utilities are generating at a price-point that makes sense.
- Data is locked away at source systems, and you’re restricted to a few hard-coded reports. Want to add a column to one of these reports? At best, that’ll require costly services and risky system upgrades that can take months to complete.
- Data is siloed across the organization. Distribution groups use distribution data; AMI groups use AMI data. But if you want to combine AMI data with SCADA data, and cross-reference it with lightning, GIS and asset data, you’re out of luck.
It turns out that every major industry is facing a similar trend: data volumes from many sources in multi-structured formats are exploding, and the business sees value in asking bigger questions of that data in real time. This is where Apache Hadoop comes in. Hadoop is an open-source software platform that stores and processes massive amounts of data – and it can do this for pennies on the dollar compared to traditional systems.
With Hadoop, you can store all your data from all your sources in in their original fidelity – smart meter data, SCADA exports, phase readings, geographic information, even digitized customer calls – all in a single place. Hadoop makes it easy to transform and analyze all these diverse types of data using familiar interfaces, such as SQL and easy scripting languages. You can even do your analysis with familiar tools such as Tableau and QlikView.
What questions about Hadoop and utilities do you have? How are you managing the growing data volumes? Contact us to learn more.