Time to get smarter with smart meter data?

In his comments to my article on the potential challenges with peer-to-peer electricity trading, James Muir asked me the difficult question about what the next steps might be to establish a pilot project enabling centralised access to meter data and how could we move this forward from a tech and regulatory perspective?

This isn't straight forward, given that the data involved has both regulatory and privacy implications. Certain data can link the consumption to a property, and so to the people residing at that property, and the Privacy Commissioner has written to retailers reminding them of their obligations in this area. That said, creating a set of annoymised historic data does seem to be possible though, as a set was created for the Lightning Lab Electric innovation challenge. So perhaps this is more a matter of willingness on the part of those creating and holding the data to create a platform for this sort of innovation to occur, and following a robust process to address privacy issues in the process.

To hell with history, how do we get to the future?

Historical data, with the identifying information removed, might enable innovators to do some initial investigation work, and perhaps some 'proof of concept' and initial AI training, but it is probably quite a step from there to creating a service using 'live' data. How might we make that next step and make live data available for a wider range of services? This is something the Electricity Authority has signalled it will be looking into in its consultation paper on multiple trading arrangements, which I understand is due out in the next week or two.

To get the conversation started ahead of the release of that paper, how about a world where the metering equipment providers (MEPs) establish a full 'menu' of data services, encompassing reconciliation-grade half-hour data, other consumption-based data such as real time data and peak demand, and network-related data such as 'last gasp' (i.e. loss of supply) and voltage? Consumers could then identify the services of importance to them, and enable the providers of those services to access the data streams they require. Beyond that, Callaghan Innovation could fund access to a limited set of annoymised historic data to facilitate service innovation, with a more complete pool available on a subscription basis for those wanting to develop new products and services.

Are we ready for real time meter data?

Another question might be whether we are ready to capture the value of all of this data? Has the necessary 'convergence' of systems, data and regulation occurred? For example, at the moment there is limited access to real time meter data, as the service that has typically been sought from MEPs is next day access to half hourly data. Moving to real time data will involve changes to communications and data storage. And what are we going to do with this data once we've got it? You could take it together with real time price information to start making some decisions on consumption, but will need to recognise that this price information is not completely robust (though this might ultimately be solved through the Authority's real time pricing project).

In an ideal world, you might take this real time usage and price information and automate the operation of some of your appliances and devices, but the relevant technologies are not yet readily available to your typical consumer (or, at least, they probably wouldn't see them as easy to use and offering useful value). The biggest appliance, your hot water cylinder, is often already being controlled by someone else. Some of the other markets that might provide opportunities for consumers, such as ancillary services and network support are not available yet either. There is probably an opportunity at a more 'bulk' level, for those with solar and batteries, but that is, for the time being at least, a relatively low volume part of the market (and one which has quite a few people already playing in it).

This isn't to say that pursuing greater data access and availability isn't important - the technology revolution we are currently experiencing is underpinned by data, connectivity and analytics - rather it is just to say the revolution is not on our doorstep so we shouldn't expect access to data to be immediately transformative. Let's look at ways of getting it out there though, as the innovation and transformation won't happen without it.