The Solar Off-Button - Part 2: Rooftop Solar Visibility

[Read Part 1]

In the last post discussing the "Solar Off-Button" in Australia, we explored some of the reasoning behind the 'emergency' powers assumed by the Australian Energy Market Operator to 'turn off' rooftop solar during periods of excess production, low demand and system risk. Right now, this activity is limited to South Australia, but other regions such as Queensland and West Australia are not far behind.

Our first article on the Solar Off-Button series stuck a chord with many in the industry, and several readers chimed in with potential solutions in the comments with enthusiasm. I genuinely appreciate everyone who has done so, and plan to include a post in this series where I share a few of those ideas - so keep them coming! I'm not at all surprised the industry is full of interesting ideas on how to avoiding using the Off-button, as I postulated previously - the Australia energy industry is "uniquely placed to demonstrate otherwise".

Since my last last post, we've learned a bit more about how pushing the Off-button will work - including a new list of 'approved' solar inverters which will follow the instructions of 'agents' - such as Solar Analytics - who hit the Off-button via an 'ability to remotely disconnect rooftop solar systems'. Details are still coming together on how this all will work.

"Where to from here?"

As I closed off Part 1 of this series - I asked the question "Where to from here?", promising:

In a series of upcoming posts, it is my intention to offer valuable information about the capabilities of state of the art technology for quantifying and predicting the contributions of rooftop solar power ... as well as dispelling some oft-repeated and poorly informed conclusions about the limitations of rooftop solar.

I'm going to break this 'valuable information' into three rooftop solar fundamentals: visibility, reliability and predictability. Each of these concepts are complicated, so let's get right to it. First up, visibility.

Stop calling rooftop solar 'invisible'

Any foray into the discussion around Australia's rooftop solar challenge will quickly introduce you to the concept of ‘solar visibility'. In particular - the assertion that rooftop solar is 'invisible' - which alleges that it's day to day energy generation profile is unknown. A poor man's runner up to this claim is 'little visibility', which, by way was once again repeated in the article I linked to above:

"The strict new standards ... are in response to the growing share of rooftop solar in the state’s generation mix, and fears from the Australian Energy Market Operator that is has little visibility or control"

Let's dispel this myth straight-away. Simply put, the concept that rooftop solar is 'invisible' is fundamentally incorrect. Moreover, it is an oversimplification of the problem and its potential solutions. And yet it is repeated over and over again across industry reports, web articles, social media and especially in reference to the discussion of the Off-button.

If you gain nothing from reading this article other than noticing how frequently the 'rooftop solar is invisible' argument is used, then my laboured keyboard strokes will be worth it!

Defining 'invisible'

Before we go much further, we first need to come to an agreement on what it means for rooftop solar to be 'invisible' or 'visible'. Perhaps it is best to start quite simply.

It could be considered fair to refer to rooftop solar as 'invisible' as it not directly measured at scale. In practical terms, this means that the energy market operator (e.g. AEMO), utilities (e.g. gentailers) and network operators (e.g. distribution network service providers) are not able to directly monitor their energy generation (real-time) in the same way they can for a power station, nor are they able to read this information directly from the electrical meter after the fact (historical performance).

And with 2.5 million rooftop solar systems installed across the country at 12GW+ of total capacity, this can sure look like a BIG problem. At first glance, it would appear we have 12GW+ of electricity generation assets, and not a clue how it may be performing on a given day.

Rooftop solar can be 'visible' as it needs to be

But there's a catch, of course, and it's a big one - this problem has already been solved. That is to say, the requisite technology for making solar PV 'visible' exists, is deployed at scale and is available to any utility, grid operator or network service provider who needs it. This technology can provide 'visibility' for rooftop solar, in that it can very accurately trace the power output profile of groups of rooftop solar systems across the day, in real-time.

Before I go onto detail the 'how' behind this claim, let me first establish:

• I spent my entire PhD digging into & solving the rooftop solar 'visibility' challenge (2011-2015) [read more here]
• This work was immediately followed by a $1M ARENA project to deploy and scale this solution, making rooftop solar visible for 12 Australian distribution networks (2016-2019)
• The team at Solcast operationalised this solution, with several major grid operator clients around the world using the technology operationally
• R&D work to further improve this capability is continuing in a follow-up project focused on South Australia, once again funded by ARENA

Making rooftop solar visible

We've accomplished the task of making rooftop solar visible through the advent of a new suite of algorithms for interpreting the imagery from modern weather satellites to classify and characterise the cloud cover that is present. With knowledge of the cloud cover 'opacity' or thickness to sunlight, we then model how much solar radiation passes through the cloud cover - with very little bias and high degree of certainty.

Rooftop solar system power output is driven by many factors - but the most important one is solar radiation, which drives approximately 90% of the process. Temperature is the next biggest impact (call it 3-5%), and we can account for that as well. So with accurate modelling of the solar radiation and temperature, we can already be 95% certain how much energy generation the typical rooftop solar PV system will generate on any given day (don't believe me? Try this tool). This is true in any case where we know the location and AC capacity of the rooftop solar system and are applying this technique to groups of rooftop PV sites.

SIDE NOTE: Oversimplification alert, I'm aware. If you have questions about system orientation, layout, shading, soiling and other 'details' - check out the "Addendum" at the bottom of this article. You can also 'nerd out' with questions in the comments.

Rooftop solar can be as 'visible' as it needs to be

This ability to accurately model solar radiation around the world, paired with knowledge of locations & capacity + PV power modelling tools, makes tracing the daily energy generation profile of groups of rooftop solar PV possible.

Not only is it possible, this method of creating rooftop solar 'visibility' supports the operations of a wide variety of industry stakeholders and has been evaluated under multiple research projects. The results of this work have been shared in many venues around the world, including workshops, conferences and scientific publications.

But these statements alone are not enough to make the case for rooftop solar visibility compelling, so let us be quite specific. To further exemplify just how visible rooftop solar can be through this approach, it is important for the reader to note that the above claims are true for:

• Energy market operations (e.g. the NEM, SWIS)
• Transmission networks (i.e. all of the rooftop solar, collectively at the 'end' of a transmission asset)
• Distribution networks (zone substations, feeders, even individual distribution transformers)
• Customised groupings for the operation of Virtual Power Plants or other grid-edge technologies

Let's look at a few examples...

Examples of visibility by scale

Visibility of Energy Markets

The below is a slide from a presentation I made to the Energy Networks Australia "Grid Edge" event in 2017 (this presentation, 'Intermittency as Opportunity' can be viewed on YouTube here). It shows an example of fast moving, fast changing cloud cover over the NSW region at left, with the coinciding drop in rooftop solar PV 'actuals' at right.

These 'Rooftop PV Actuals' are released by AEMO and are updated daily as a part of their "ASEFS2" forecasting model. These are their internal 'best guess' at the actual rooftop solar generation, but the methodology for producing them is not documented clearly. Nevertheless, AEMO has had access to these 'actuals' for a few years, and they are produced partially based on measurements from solar PV systems reporting to PVOutput.org (the Solar Map from APVI is an example of a similar approach).

As we can see in this example, despite aggressive cloud cover growth, the 'PV Actuals' (right hand side) picked up on the aggregate behaviour of the rooftop solar in the NSW region. This is a great example of rooftop solar looking pretty 'visible', even before we consider much more accurate, transparent and scalable technology options for quantifying the daily energy generation profile from rooftop solar. I could easily use multiple Solcast examples here, but it a great opportunity to point to another example of how solar can be 'visible' for the energy market.

Visibility for Transmission Networks

Do you recall fondly the 100-days or its free battery bet and banter of Elon Musk and Mike Cannon-Brookes of 2017? Solcast does - and in fact, we were so excited by the prospect of a large battery system landing in South Australia that we quickly spun up a transmission node based rooftop solar demo, within just a few days. We perhaps were a bit overly keen - but we were super enthused about the potential of combing energy storage with groups of rooftop solar!

This application of our Grid Aggregation data product applies the technology I discussed earlier (satellite data to PV power modelling), and relied upon a quick mapping of Clean Energy Regulator (CER) rooftop solar installation data back to the 'catchment' regions for 7 of ElectraNet's transmission lines (the ones with the highest penetrations of solar at the time). In this example, we follow a more robust approach than the AEMO example above, where the actual cloud cover and radiation conditions at 1km^2 resolution are mapped precisely to the locations of individual rooftop solar PV systems.

Relying on the solar radiation and PV power modelling process I described earlier, it is possible to clear see (e.g. visible) the daily power output profile of the rooftop PV in each transmission catchment zone.

Above: The above chart shows both forecast (to the right of the vertical dotted line) and estimated actual rooftop solar PV data (to the left of the vertical dotted line) for 7x transmission regions in South Australia. It is this 'estimated actual' of rooftop solar PV power output which makes it sufficiently 'visible'.

Notably, we are now operating 'one step below' the 'energy market' regions in the previous example. Could we zoom in further?

Why yes we can...

Visibility in Distribution Networks

From 2016 - 2019, I was the leader of an ARENA funded project whose express purpose was to provide rooftop solar visibility in the distribution network. We had 12 DNSPs onboard this wide-reaching, highly collaborative project, where they were provided with access to the Solcast Grid Aggregations data product, customised for areas of interest in their networks.

What we found, was that providing them with rooftop solar 'visibility' data not only solved the 'invisible' solar problem - it also resulted in several new applications for this type of data. By making rooftop solar visible in the distribution network (zone substations, feeder lines distribution transformers), we enabled the brilliant folks at the DNSPs to come up with many different applications for the data.

SIDENOTE: Examples like those engineers coming up with new ideas once rooftop solar become 'visible' make me wonder what they might come up with if we put our heads together on this whole Off-button thing...

Here's one of my favourites, from Essential Energy. The coloured lines in the graph each represent a different day (total of 6 days) of load for a zone substation. At left, the observed load is seen to be quite 'random' in how it jumps up and down. At right, once the Grid Aggregations data was included to create a 'gross load' that accounted for the rooftop solar generation, things are much 'cleaner', and - most importantly - it is immediately apparent just how much rooftop solar is changing the behaviour of load in the distribution network.

Visibility for any Customised Grouping

Modelling and grouping rooftop solar output is possible for energy markets, transmission networks and even the distribution network. But what about for groups of rooftop solar PV systems which are not directly linked by their location in the electricity network? That's not a problem, in fact, estimates of rooftop solar PV power output can be grouped in any possible permutation.

A great example of this type of customised application is the Virtual Power Plant (VPP), and in a world where VPPs just might be the solution to our Solar Off-button challenges, this one is important.

Now I only have one example of rooftop solar PV power modelling and forecasting being applied in a VPP context, but I think there is a lot of potential here. During the DNSP solar visibility ARENA project cited previously, SA Power Networks was curious to see how the Grid Aggregation data product might work for the Salisbury VPP trial.

While this early stage trial did not use any solar forecasts in its operation, we were able to establish the validity of the rooftop solar estimates, even for relatively small groups of rooftop solar sites (on the scale of a few 10s of rooftop solar PV systems).

Closing Thoughts

Rooftop solar is 'invisible' in a strict sense of being unmetered. But it can be made as 'visible' as is required for the operation of energy markets and our transmission/distribution networks. This thereby full dispels the 'myth' of rooftop solar 'invisibility' - in particular with respect to the need to 'switch off' rooftop solar via that Off-button. Simply put - this Off-button approach may be a perfectly reasonable way to manage high penetration rooftop solar, but do not assert this on the basis of rooftop solar being 'invisible'!

With this first issue out of the way, we're going to progress to the topic of rooftop solar 'reliability', whose sweeping dismissal as being 'unreliable' along with other renewable energies such as utility scale wind and solar, is once again worthy issue to dispute. But we'll save that one for next time.

As I close this off, and in the spirit of making a meaningful change to the discourse on the Solar Off-button subject matter - I'd like to put some skin in the game. If you are the operator of a VPP and are interested to work with my team to further explore the conceptual framework of using solar forecasting to manage VPP operations - I would be happy to give you extended access to a free trial of the technology, in the spirit of solving some of the Solar Off-Button dramas! Just send me a DM.

----------

About the Author

Dr. Nick Engerer is the CTO and co-founder of Solcast, whose mission is to deploy the data and tools needed to build the solar powered future. Follow Nick on LinkedIn for more posts on the topic of rooftop solar integration and energy meteorology.

--------

----------------

Addendum: Anticipating questions from the skeptic/curious

Pre-conceived notions about satellite based technology

If you've been in the energy industry for more than a few years, it is likely that you enter this discussion with pre-conceived notions of what is possible using weather satellite imagery. At a conference or in a white paper a few years back (as recently as 2015 in Australia, 2017 in the US or Europe), 30 to 60 minute updates of coarse (5 x 5km) satellite imagery were the status quo. Solar irradiance modelling was quite biased, timed cloud fronts poorly and was unable to accurately tell the difference between a high thin cloud and a low thick one.

All of this has now changed. A step-change, in fact. For example, in Australia, via the Himawari 8 weather satellite, Solcast now updates our forecasting data every 10 minutes with 1 x 1km resolution data, operating with a near zero bias and 3D modelling of the cloud cover features.

Rooftop solar orientations, shading and losses

In the early days of working with Australia's distribution networks (circa 2016-2017), it became immediately clear that the sharp-as-a-tack engineers working across this great country were some smart folks. And in nearly every introductory meeting, with our project pitch deck presentation, the conversations would inevitably progress to the 'nitty gritty' details of rooftop solar modelling.

What about PV system orientation? And shading? Losses?

Let's cut straight to the chase. It is 100% possible to produce very detailed models of rooftop solar in the LV network, which account for PV system orientations & shading losses. Solcast owns and operates the technology required to do this. We can either 1) learn these characteristics through analytics of PV system power output data or 2) infer them directly through Lidar data.

Above: From top-left, clockwise. A PV system auto-detected from Lidar data, followed by its surrounding shade objects, simulated+manual+actual PV system orientation data for azimuths and tilts, a shading event cross validation with the above Lidar data as detected by time-series PV analytics and the finally auto-detection of PV degradation.

However, despite incredibly high penetrations of rooftop solar in our LV networks and the potential of the above technology - Solcast has yet to encounter an actual need for this level of specificity. To put this more simply it is one of those issues where everyone asks about it, and thinks it is important, but it appears to rarely matter.

Solar contributions to the grid vs. self-consumption

Smart metering in Australia has rightly faced a lot of criticism - they have been expensive to roll-out, and have not been configured properly to measure the actual solar PV profile seperate from the household/business demand.

The lack of this direct measurement data is one of the main reasons solar is labeled as 'invisible', although I've already debunked this through my write-up thus far. However, we have one remaining issue here - one of the major challenges with rooftop solar is that its contributions to the grid are not directly known, leading proponents of the 'invisible solar' argument to push a bit further here and label *this* as the real 'rooftop solar visibility' issue.

However, this is relatively weak excuse. If the individual rooftop solar profiles can be traced with 90-95% accuracy via the satellite to solar methods I've described here, and can easily be scaled up to provide an 'accurate guess' for entire neighbourhoods, zone substations or any other permutation - we're more than halfway to solving this problem. And thankfully, household demand profiles are very well understood and behave predictably at-scale (those new-fangled household battery systems excluded of course); grid operators and networks service providers around the world have been producing household demand estimates for decades with increasing levels of skill.