Slicing 14-dimensional potatoes: a visualization of the Core flow-based domain

In this article, I will visualize 2-dimensional cross-sections of a 14-dimensional flow-based domain and demonstrate its functioning and relevance.

This is ?a fairly complex, abstract concept – at least to my own standards. In what follows, I will attempt to explain the Core day-ahead flow-based capacity domain, by visualizing it in a simple and practical manner.

The capacity domain determines how much transmission capacity is available for the exchange of electricity across borders in a flow-based market coupling process. Its shape is affected by the physical (or virtual) margins on individual network elements and their sensitivity to flows of electricity between different zones.

Why does this matter?

It doesn’t, really – at least not for everyone. It’s a specialized, highly technical topic with a steep learning curve and a very limited potential for casually dropping it in a conversation about electricity markets.

Nevertheless, understanding how it works is crucial to appreciate and assess the benefits of a flow-based market coupling mechanism. The basics of FBMC are fairly simple,[1] yet the actual assessment of how much capacity is available in a concrete (non-abstract) manner is trickier. This last challenge relates to the visualization of the domain, in which the net positions of the 12 Core bidding zones and 2 virtual hubs can move freely without being restricted by the available capacities.

The goal of this article is hence twofold: explaining the concept of the flow-based domains and visualizing it in a simple yet engaging way – both equally important.

Background

The Core Day-Ahead Flow-Based Market Coupling Project (“Core DA FMBC”) brings together the TSOs and NEMOs of 12 bidding zones in 13 European Member States, to jointly calculate and allocate cross-zonal capacities on 19 different borders. It celebrated its successful go-live on 8 June, coupling the markets for delivery on the next day.

These bidding zones are, in alphabetical order: Austria, Belgium, Croatia, Czech Republic, France, Germany/Luxembourg, Hungary, the Netherlands, Poland, Romania, Slovakia and Slovenia. To accommodate exchanges on the DC-border between Belgium and Germany, two "virtual hubs" are added: ALEGrO Belgium and ALEGrO Germany.

Through this market coupling process, zonal prices and net positions for all bidding zones are calculated for each hour of the year.

The Core DA FBMC Project is the successor to the CWE FBMC, coupling the markets of Austria, Belgium, France, Germany/Luxembourg and France, running from 21 May 2015 until 8 June 2021.[2]

The benefits of a flow-based over a coordinated NTC-based approach include a better representation of the actual state of the network (safeguarding operational security) leading to potentially lower security margins (increasing cross-zonal trade possibilities) and a more efficient use of the transmission capacities, increasing overall socio-economic welfare.

On the data

The Core TSOs publish many different datasets related to capacity calculation (in the day-ahead but also the intraday timeframe) in the JAO Publication Tool (?? link). Datasets can be downloaded either manually or automatically via an API - the latter is particularly useful as some datasets are quite large.

The following analyses are based mostly on a limited number of datasets:[3]

• the presolved domains ("Final computation"), listing among others the available margins and sensitivities to cross-zonal exchanges on all network elements;
• the shadow prices, showing which of the elements in the presolved domains actually limit the market (i.e. the active constraints);
• the long-term allocated capacities, which can be used to construct the LTA domain;
• the zonal net positions, or the outcome of the market coupling, determining for each bidding zone whether electricity is imported or exported; and
• the scheduled exchanges, which translate the zonal net position in a bilateral exchange across a border (between two zones).

The flow-based potato

The Core TSOs publish, under the final computation page of the JAO Publication Tool (?? link) the parameters for each of the observed lines (“CNE” or Critical Network Element) and associated contingencies (“C”). These parameters relate to the identification of the CNEC but also on how the margins for cross-zonal exchanges (“RAM” for remaining available margin) are established by substracting security margins, reference flows, long-term nominations, etc. from its maximum allowable power flow (Fmax). In addition to these parameters, the impact of a change in the net position of a bidding zone on the loading of the CNEC is reflected in the PTDF (“Power Transfer Distribution Factor”).

All these elements combined constitute a fairly large and complex dataset: the CNECs of the 16 participating TSOs and their characteristics are listed for each delivery hour. For each hour, +- 20.000 CNECs are listed, even though most of these are irrelevant: they are not restricting the cross-zonal exchanges as other CNECs are more rerstraining. These latter are called the “presolved CNECs” or the “presolved domain”. From the +- 20.000 CNECs, typically only between 100 – 150 are presolved.

These presolved CNECs determine the 14-dimensional flow-based domain (often affectionally called “the flow-based potato”). It is hard to think of this domain in non-abstract terms, but essentially it is a multidimensional area in which the net positions of Austria, Belgium, Croatia, Czech Republic… etc. can move, given the combined margins of all the presolved CNECs.

Alternatively, the domain can also be expressed in dimensions of bilateral exchanges rather then net positions. In that case, it becomes a 19-dimensional area (one for each bidding zone border), in which the bilateral exchanges over these borders can move.

Visualizing this is a challenge: you need to define which two bidding zones (or which two borders) to be portrayed, and take a cross-section of the 14-dimensional domain into the selected two-dimensional surface. This way, you are slicing it up.

Let’s take an example: on 1 Nov. 2022 from 20:00 to 21:00, we can find 127 presolved CNECs for the Core flow-based domain. We can plot the domain for the net positions of Belgium (horizontally, on the x-axis) and France (vertically, on the y-axis). To do this, we need to calculate, for each of these 127 CNECs, the slope and the intercept, based on combination of the PTDFs for Belgium and France and the margins for exchanges on that CNEC. Some basic mathematics learn that:

Intercept = RAM / PTDF_FR

Slope = - PTDF_BE ?/ PTDF_FR ?

This can be done both for the presolved CNECs (in the final computation data set) as well as the active constraints (in the shadow prices data set). The latter are the points which are hit by the market clearing point in one of the 14 dimensions (not necessarily the ones we will visualize). In the following figure, the active constraints are the red lines, the presolved constraints [4] are the blue ones.

The green area in the demonstrates the combination of net positions of France and Belgium that are feasible for the market clearing point to move through, taking into account the margins on the 127 network elements (of which actually only 6 are relevant for the BE – FR combination of positions) and their sensitivity to changing net positions of Belgium and France.

This figure shows one of the 182 (14 bidding zones = n * (n-1) or 182 combinations) possible visualizations of the flow-based potato for this specific hour.[5] The market clearing point (red point) results in an export position for Belgium at 1.984 MW and an import position for France at -2.973 MW.

Not only the margins on the network elements determine how much electricity can be exchanged in the day-ahead timeframe. The capacities that are already sold by the TSOs through the long-term (monthly or yearly) auctions via JAO are guaranteed through the so-called “LTA inclusion process”. As these long-term capacities are calculated through a cNTC-based, bilateral approach, the capacity domains are rectangular and defined by summing the cNTC values on all the borders of a certain bidding zone. Both domains (the presolved flow-based domain as well as the LTA domain) are given to the market coupling algorithm, which takes the convex hull of these (again – to stress the complexity – in all different dimensions) in which the market clearing point can move. The cross-section for Belgium and France is shown below, taking the following cNTC values for the same hour:

• for Belgium: 1.569 MW export; 2.663 MW import
• for France: 2.897 MW export; 1.449 MW import

As you may see, the LTA inclusion is particularly relevant in the export direction for both countries: according to the regular presolved domain, a combination of export for Belgium and France is only possible to a limited extent. However, given that TSOs guarantee already allocated long-term capacities, and these have been sold in the export direction, there is a part of the LTA domain?(top right corner of the red polygon) which allows the market to move to a combination of net export that the presolved domain would not be able to accommodate.

One step further...

This is where it gets interesting. Now that we are able to plot the flow-based domains (presolved and LTA), we can start to imagine use cases where this information is relevant.

• We can compare the sizes of these domains, across time or across other bidding zones. Naturally the domains will be bigger for Germany than for Slovenia, given the size of the bidding zones and the size of the interconnections to their neighbours. The absolute value of the size of the domain is a very abstract metric: we express it in MW^14. We can understand it by calculating the size of the cross-section, which is in MW2 or squared megawatts: in the example above, this is 45.798.724 MW2 for the presolved domain (green) or 18.379.596 MW2 for the LTA domain (red).
• We can highlight all CNECs which exhibit certain characteristics (similar to the distinction between presolved / active CNEC). For example, let’s say we want to know which elements have a RAM value below 20%: we can easily check this in the data and highlight these in the cross-section (see figure below). We could even calculate the effect of the increased application of remedial actions to bring the RAM of CNECs, where below 20%, to this minimum level (shifting the CNEC) and calculate how much the domain increases.?

• We can also work in a bilateral way, focusing on creating a domain on a set of borders (for the bilateral scheduled exchanges to move through rather then the Core net position). For this purpose, we need to change the way we calculate the slopes, no longer basing them on the zone-to-slack PTDFs but calculating the zone-to-zone PTDFs.[6] The following example calculates the domain for scheduled exchanges between Belgium and the Netherlands (horizontally) and between Belgium and France (vertically). The slopes and intercepts are calculated as follows:

Intercept = RAM / (PTDF_BE - PTDFF_R)

Slope = - (PTDF_BE - PTDF_NL)?/ (PTDF_BE – PTDF_FR)

... and then some more

Or we can continue our trip into insanity, down the road of increasingly fascinating but perhaps decreasingly useful combinations of visualizations. Let’s focus on only a couple.

First off, we can attempt to visualize every possible combination of net positions, slicing the flow-based potato, Fruit-Ninja-style, 132 (n * (n-1)) different ways. For a specific hour, we can loop through all (i, j) combinations of bidding zones and visualize the flow-based and LTA domains, with bidding zone [i] horizontally and bidding zone [j) vertically.

Even though this figure might seem like pure chaos and a random collection of polygons, the construction requires quite a bit of precise, meticulous coding and the results are actually surprisingly clear. Some observations:

• While not shown, the scales range from -7.500 to +7.500 MW in both directions. Some countries typically have larger domains, some smaller, but the orders of magnitude are rather similar, perhaps with the exception of Poland.
• Some bidding zones heavily rely on LTA inclusion to provide more trading opportunities. Notable examples are Germany/Luxembourg and Austria (not coincidentally because of large volumes of allocated capacities on their bilateral border).
• One country does not allocate long-term capacities at all. This country – Poland – therefore does not have an LTA domain (red polygon).

Or we can keep our focus on one visualization, but look at its evolution throughout time. This requires a bit of animations. The GIF below shows the 48 domains and the net positions of Belgium and France for two entire days, from 1 November 00:00 until 2 November 23:00.?

We can see that the LTA domain remains stable (as the monthly and yearly long-term allocated capacities where the same on both days) while the presolved domain changes constantly, setting new borders for the market clearing point at each hour, as if playing Pong in an ever-changing environment.

We can also see that the market clearing point is consistently in the bottom-right quadrant, indicating that Belgium is exporting while France is importing (this is pretty standard behaviour nowadays). It’s interesting to see the orientation of the domains: from top-left to bottom-right, meaning that if one country (Belgium or France) is importing, the other (France or Belgium) needs to export – if both countries want to import or export at the same time, the domain is much smaller (in the top-right and bottom-left quadrant) – this makes perfect sense due to the fact that both zones are neighbouring in the same geographical corner of the Core region.

Finally, related to footnote [5], we see hours where the market clearing point falls outside of the domain. Actually, it does not fall outside of the domain (i.e. the 14-dimensional potato), only outside of the visualization which we’ve chosen, as this choice assumes balanced net positions for the other zones.

Conclusion

The use cases for this type of visualization are endless, yet understanding what to show and how to show requires a bit of practice. The 2-dimensional representation of the 14-dimensional flow-based potato is a useful tool to move from the abstract consideration of flow-based market coupling to the concrete determination of possible market exchanges within the Core region.

This visualization is - in my view - very useful to assess the impact of design choices on the net positions for and exchanges between Core bidding zones, and I hope it might help some of you understand better how the flow-based domain works.

?? If you have any questions, comments or suggestions for improvements, feel free to let me know!

Update: the source code

For the sake of transparency, you may find the code used to download, transform and visualize these datasets in my GitHub repository:

https://github.com/nicoschoutteet/Flow-based-potatoes

Also here, any suggestions for improvement are more then welcome!

Footnotes

[1] See also, for a more comprehensive overview, the third installment of CREG's Energy Market Topics, on FBMC: ?? link

[2] The CWE FBMC ran from 20 May 2015 until 8 June 2022 and is the predecessor of the Core DA FBMC, covering Austria, Belgium, France, Germany/Luxembourg and the Netherlands. Some months ago, I wrote a similar article to this one, summarizing some striking observations of the CWE FBMC: ?? link

[3] Some weeks ago, I uploaded a more comprehensive analysis of these different dataset, at the occasion of the 100th business day where the Core DA FBMC was operational: ?? link

[4] Note that there are also redundant CNECs, which do not border the sliced flow-based domain. Being presolved in any of the 14 dimensions does not imply that it is necessarily presolved for each of these bidding zones.

[5] Actually there are infinitely many possible slices. When taking two net positions, we consider that the other 14 net positions are at zero, but this is not necessarily (and probably not) the case. We have to take assumptions (as we can only visualize two dimensions), though, and balancing the other net positions is as good a choice as any. This choice however implies that the market clearing point can fall outside of the presolved or LTA domain, as we have sliced it at the wrong point for any of the other bidding zones’ net positions.

[6] A zone-to-slack PTDF denotes the sensitivity to a change in the general net position of a bidding zone (e.g. “Belgium”). By substracting two zone-to-slack PTDFs, we obtain zone-to-zone PTDFs (e.g. “BE – FR”).?