Flow based in my trainee rotations
I started in Statnett in 2021 and spent the first year-and-a-half as a trainee. Although I have worked in three very distinct departments in Statnett as part of the trainee-program, I have noticed a somewhat common thread throughout my traineeship. This common thread is a mysterious concept known as flow based market coupling (FBMC). I first started working more directly with FBMC in the department of market design during my second rotation, and some of that work continued when I transferred to the data science department. Here, I did several analyses and created a dashboard to visualize some interesting results we found.
In this blogpost, I will go over the basics of the flow based method, the purpose of implementing it in the Nordic synchronous area and shed a light on some of my own analyses on the area.
Market coupling and capacity calculation
The energy and power system in Europe is currently going through large changes. The share of variable renewable energy is rapidly increasing, and there are large digital solutions being implemented in the operational and market side of the power system. Being able to handle more complex and less predictable systems will be paramount to keep the power system stable and reliable. This is one of the main arguments for implementing FBMC.
In short, FBMC is a method of calculating transmission capacities between bidding areas, which are given to the market for trading. This is part of the process that sets hourly spot prices in the power market. The FBMC methodology is set to replace the current methodology in the Nordic power market, Net Transfer Capacity (NTC) , and is currently planned to be implemented in 2024. In order to understand why the shift to FBMC is necessary, we first have to go over the current methodology and how it works.
The current methodology
The electrical power grids in the Nordic countries (except for the western part of Denmark and Iceland) are all part of the same synchronous area. Consequently, power flows given by the physical laws of electricity (Kirchhoff’s laws). However, the power market requires much simpler models than the real physical system. In the current capacity calculation methodology, the power grid is modeled as a set of lines connecting the different bidding zones, but the lines within a bidding zone are not modeled. Two days before the operating day (D-2), each Transmission System Operator (TSO) in the Nordics predict how much surplus or deficit each bidding area in their respective country will have, and also set a maximum capacity of the lines connecting the bidding zones. This capacity calculation is based on thermal and voltage limits of the transmission lines, but is also based on experience and knowledge of the system operators. It is important to note that the capacities are set with the assumption that power can be transferred directly from one zone to another, which is not really the case in a meshed grid. The individual grid models of each TSO are then sent to the Nordic Regional Coordination Center to combine all these models into a common grid model. Here, capacities on all the lines connecting bidding zones in the Nordic synchronous area are modeled, and these capacities are given to the market for electricity trading one day before real-time (D-1).
The basics of the flow based model
In contrast to NTC, the FBMC model is much more similar to the physical grid, and also includes critical network elements (e.g. power lines) within the same zone in the capacity calculation. In the flow based process, the TSOs define a set of critical network elements (CNE) in their respective country, both inside a bidding zone and between two bidding zones. The TSOs also define a maximum flow on each of these CNEs, which is calculated based on the operational security limit. The CNEs and the maximum flows are sent to the Nordic Regional Coordination Center-office which calculates the flow based parameters. These parameters are power transfer distribution factors (PTDF) and remaining available margin (RAM). The PTDFs give the relationship between change in net position and change in flow on the lines in the power system. In other words, they indicate how the flow distributes if an extra MW of power is injected into a bidding zone. Thus, the FBMC model considers how the flow on one line will affect the flow on the other lines in the model. These PTDFs and RAMs are calculated based on a linearization of an AC load flow model around the base case. The base case is a prediction by the TSOs of how the grid topology, net positions and flow will be on the operational day. As shown in the Figure 1, the RAM is the difference between the maximum flow allowed, minus some extra safety margins (flow reliability margin and final adjustment value) and the reference flow (Fref’) of the linearized flow when the net position is zero. If there is remedial actions available, the RAM can be increased, which is what is illustrated below.
The reference flow, Fref’ is flow that is not traded in the spot market, but through other contracts, for example bilateral trade between an industry participant and a power producer. Thus, there will be some flow even when the net position is zero. The PTDFs and RAMs for each CNE and bidding zone are made available to the market, and they define a flow domain, as shown in Figure 2. Consequently, the market is given a domain to trade within, where the capacities between bidding zones depend on each other. This allows the market to maximize trade , and thereby maximizing the socio-economic welfare. Also, since the flow based model more accurately depicts the grid, the flow based domain is larger than the NTC domain, given the same operational security, as seen in Figure 2.
The main advantages of flow based
To summarize and give some more context, the main advantages over the current NTC methodology are the following:
- FBMC is a market model that more closely resembles the physical grid, and can more accurately set capacities that remain within operational security.
- It considers how flow on one line affects the flow on the other lines in the grid, and allows the market to trade more than previously.
- Generally, it should increase the overall socio-economic welfare.
- There should be higher price convergence, i.e. more similar prices across Europe.
- FBMC increases security of supply by reducing the risk of congestion, and also ensure operation inside a secure operational security boundary.
Creating dashboards to illustrate quality measurements
As mentioned above, one of the main arguments for switching to FBMC is the increase in socio-economic welfare. Although this theoretically should be the case, it still needs to be validated. This is something I worked on in the department of market design and data science. FBMC is now in what is called an external parallel run (EPR). In this period, the TSOs must prove that the new market coupling method is operationally secure, and that it provides a gain over the current methodology. If the national regulatory authorities deem the new methodology as stable and beneficial after the external parallel run, it will be implemented as the new CCM.
Along with a small team, we created a dashboard that would keep track of and compare the socio-economic welfare of FBMC and NTC. We also wanted to verify whether the expected result corresponds with reality. The socio-economic welfare was calculated as the sum of consumer surplus, producer surplus, and congestion income. In general, we saw an increased socio-economic welfare with flow based compared to NTC. However, there were weeks where this was not the case. In the flow based regulatory framework, the regulatory authorities require the TSOs to give a justified explanation if there are two or more consecutive weeks where the socio-economic welfare is lower than for NTC. From the dashboard, we noticed that this occurred several times, and a task force was created to investigate why. I was part of this group, and together with a colleague, we wanted to assess the quality of the flow based model with respect to how well it predicts flow in the physical grid. In doing so, we hoped to find out if there were any systematic errors either in the flow based model itself, or in the input data to the model.
The process was as follows:
- We extracted measured flow data from the physical grid.
- We used the measured flow data to calculate net positions for each hour for each bidding area. The net positions are simply the flow exiting the area minus the flow entering the area.
- We extracted the PTDF values mentioned above, along with the Fref’ shown in Figure 1 from the FBMC results.
- We multiplied the PTDFs with the net positions and added the Fref’, which gave us an estimated flow with the flow based model.
- We compared the estimated flow with the measured flow, and we can see the discrepancy.
- Finally, we visualized these data for a certain time period with box plots.
As part of my rotation in data science, I was asked to make an interactive dashboard that would illustrate the points above. The operators could then get an impression of how the flow based model is performing, and the dashboard could also be used for further and deeper analyses on the input data and the modelling itself . The process was very similar to what had already been done, but this time we wanted to use APIs , download flow data and market data to a database, and continuously update the database with new data. We used publicly available data and fetched measured flow from the ENTSO-E transparency platform API, and FBMC results from the Joint Allocation Office platform API.
We implemeted a dashboard using Plotly Dash. By using box plots we illustrated the difference between measured and predicted flow for all border CNEs . In this way, operators and other analysts could see for which areas the flow based model accurately predicts the flow, and for which it does not. This can be used to perhaps change input to the flow based model, or at least be aware of where we can expect larger discrepancies. Below are some screenshots of the dashboard, and what we tried to illustrate.
Looking ahead towards go-live of flow based
On March 13th, we managed to satisfy the requirements of the regulatory authorities for a three-month consecutive period. A report has been written to sum up these three months and is sent to be approved by the regulatory authorities. If there are no other large hiccups along the way, we expect FBMC to go live in Q1 2024. Until then, there will be many more analyses to be performed, and there are also internal KPIs (key performance indicators) we want to measure and achieve. Hopefully, the dashboard we created can be a useful tool in this process.
It has been a very educational 6 months in the data science department at Statnett and learning how to use APIs and databases for these types of analyses have taught me a lot. The first analyses I did were run locally on my computer with python and csv-files. In addition to being limiting on who would have access to the results, the solution was also much less flexible in terms of user input. Now, the dashboard is available to everyone in Statnett, and I hope the results will be used by more people. In this way, it might contribute to help understanding the flow based model and its output more. I will take with me what I learned in the data science department, and it will definitely be useful in my future work.
Leave a Reply