UPDATED 1 Sept: The EI library in London is temporarily closed to the public, as a precautionary measure in light of the ongoing COVID-19 situation. The Knowledge Service will still be answering email queries via email , or via live chats during working hours (09:15-17:00 GMT). Our e-library is always open for members here: eLibrary , for full-text access to over 200 e-books and millions of articles. Thank you for your patience.
New Energy World™
New Energy World™ embraces the whole energy industry as it connects and converges to address the decarbonisation challenge. It covers progress being made across the industry, from the dynamics under way to reduce emissions in oil and gas, through improvements to the efficiency of energy conversion and use, to cutting-edge initiatives in renewable and low-carbon technologies.
Powering up data-driven decarbonisation and flexibility
15/1/2025
10 min read
Feature
A recent energy policy debate at the Energy Institute focused on data-driven decarbonisation in the move towards smarter energy management. The vibrant discussion addressed how new digital technologies are affecting decision making, in the move from centralised power generation towards a smarter, but more complex, two-way supply and demand system. The opportunities arising from artificial intelligence (AI) and new software for improving data sharing and flexibility were also examined. New Energy World Features Editor Brian Davis reports.
The authoritative panel was chaired by Sara Vaughan FEI of Elexon (administrators of the UK electricity grid balancing system), and featured Rebecca Rosling, Head of UK R&D, Future Energy Systems at electricity utility EDF UK; Cathy McClay, Managing Director, National Grid DSO; Gavin Starks, Founder and CEO of data sharing non-profit Icebreaker One, and Jo-Jo Hubbard, Co-founder and CEO of flexibility trading market supplier Electron.
A key theme was the importance of data and collaboration in a rapidly evolving UK energy system. ‘Data is more valuable when it is shared, not hoarded,’ said Vaughan. ‘Especially with the move away from large power stations sending electrons in one direction to suppliers who supply consumers, to a more complicated world of two-way traffic of supply and demand with balance through flexibility and demand reduction at the distribution level.’
There was a proposition that the (energy) trilemma – relating to energy security, affordability and sustainability – would no longer have as much impact because renewables are much cheaper. However, some disagreed, with Vaughan saying the prevailing view was: ‘The trilemma is still a valid framework. Notwithstanding that renewables are falling in price massively, and there is still a need to take into account everything else that sits around the system.’
AI roll-out
Rebecca Rosling of EDF UK emphasised: ‘AI will help roll out net zero technologies at scale.’ She described how AI is being used as a diagnostic tool to assess effects on solar panels, to determine cracks in wind turbines, and use machine vision to ensure that smart meters have been installed correctly.
AI and machine learning are also being used as an advisory tool to improve forecasting, transmission and distribution. But Rosling said AI raised new challenges ‘as a bit of a Black Box’. And asked: ‘How can we trust AI if we don’t know how it works.’
New software
Gavin Starks, Founder of non-profit Icebreaker One, described its scheme for data sharing governance in the financial services sector, named Perseus. The programme enables carbon emissions in small and medium-sized enterprises to be better reported and assured, so banks and others can verify sustainability data to unlock access to green finance through the Perseus framework.
‘There is now a common approach for financial institutions to align data, carbon reporting calculations and implementation, supported by a framework of legal and technical processes,’ he said. ‘We built a remarkable coalition using a cross-sector smart data system, where everybody can collaborate in a trusted network, so loans are available for SMEs at lower interest rates for assured green projects.’
Jo-Jo Hubbard, Co-Founder of Electron, is building software to help scale distributed flexibility markets faster. Working with half of the network operators in the UK, the company is developing a full market platform for the procurement of different flexibility products, for near real-time optimisation of markets, curtailment, allocation and value orchestration.
She said flexibility will require ‘bigger, near real-time asset flexibility’, and suggested that: ‘We are trying to do decarbonisation faster than any other developed economy in the world. I think that the Clean Power 2030 Report [from the UK government] underscores that in a stark way.’ However, she recognised that decarbonisation won’t happen faster unless it is done in a cost-effective manner. This means building responsibly with more reliable data, and demand centralisation of all the data.
Hubbard suggested that three things are essential for faster decarbonisation with flexibility. ‘First is the need to push more value into what you can actually pay with flexibility. The second is to allow flexibility service providers time to invest and bring new volume to these assets… because you can’t do a partial connection or a partial deferral. Third is the idea of reducing friction, so transactions can move faster with a smaller number of volume value.
But Hubbard warned that ‘the difficulty in this sector is that the industry can’t change at the speed of the smallest operator’.
‘Digitalisation is where it’s going to be at in the next stage of distribution,’ remarked Cathy McClay, Managing Director of the National Grid Distribution System Operator (DSO), with 8 million customers and 12 GW peak demand. The DSO has 80,000 heat pumps connected to the network and 148,000 electric vehicles (EVs) today, which is forecast to reach 1.14mn heat pumps and 2.5mn EVs by 2035. ‘Changes in energy generation and the needs of customers are contributing to a more complex distribution network,’ she said.
McClay spoke of the primary (400 and 275 kV) high voltage transmission network and the low voltage, 240 volt secondary distribution network which brings electricity to consumer homes. ‘Good planning is a priority, and the DSO runs [detailed] flow models on these networks. The primary network has 27,316 nodes. The secondary network has 2.3 million nodes – so there’s a lot of data in these models.’
National Grid DSO uses distribution future energy scenarios, working with 120 local authorities and up to 7,200 strategic projects to assess the growth of demand in its network going forwards. The associated file has 13.2 billion rows of data. ‘More is coming in the regional energy system planner. There is a lot going on as we get more and more active assets, so we really can’t keep doing things the same way,’ she admitted.
‘Digitalisation is where it’s going to be at in the next stage of distribution.’ – Cathy McClay, Managing Director, National Grid DSO
Problem is, there’s also gas developments and heat networks to consider, and they all interact with each others’ plans and need to coordinate. ‘There’s a lot of data to exchange and to share and have one version of the truth,’ she said.
As customers will play a much greater role at the distribution level, the role of the DSO in the future cannot be underestimated. ‘I think system operation will change most in the next 10 years, raising issues about the system operational envelope, and what toolbox will be required to manage the network going forward,’ McClay remarked.
Flexibility is key, and the use cases for flexibility are going to change over time as well. ‘The way we want to go is to build out the right amount of network that we can allow all of these assets (for decarbonisation of heat and transport) to access the whole system of markets as well,’ she said.
Here again, AI has a vital role to play.
McClay highlighted opportunities for DSOs to digitalise, including:
- Smart grid automation and sensors
- Self-healing networks
- Automated flexibility dispatch
- Real-time data analytics to support decision making
She discussed regional electricity system planners and the role that the National Energy System Operator (NESO) will play working with DSOs in those regions of the electricity network. ‘NESO wants to control everything down to the distribution level… but they can’t!’ she remarked.
Tariffs, smart meters and other issues
During the Q&A session, one of the attendees raised a question about the attractiveness of different tariffs, which he considered to be one of the greatest drivers of demand-side response, giving the example of the Octopus Agile initiative.
This led to a discussion about how people can access those tariffs if they haven’t got smart meters. A straw poll was held regarding smart meter take-up. A surprising number wanted a smart meter but couldn’t get one because of lack of available access. While some customers are excluded because of affordability issues around electric assets such as EVs.
Another question addressed critical national infrastructure. Some attendees asked whether we were moving too quickly on digitalisation and suggested there should be more thought about the challenges and risks. Though the NESO discussion suggested that ‘not everything should be centralised’, most people were comfortable that governance should be centralised, but felt there should be a balance between centralised and decentralised operations.
The Chair asked: ‘Can the NESO control room manage thousands of assets?’ The common opinion of the panel was ‘no’.
Indeed, Gavin Starks was concerned that people were trying to solve all the problems around decarbonisation and flexibility simultaneously. ‘If you try to do that, you end up solving none.’ Instead, he favoured focus on individual use cases and concluded: ‘It is better to do one thing really well; pick the right use case, solve that and move on to another, [rather] than trying to fix it all simultaneously.’
The Icebreaker head emphasised the importance of collaboration. ‘Don’t criticise, collaborate.’ It’s easy throw brickbats at what somebody else is doing, but far better to work constructively, particularly with a programme of this scale and complexity, said Starks.
Further perspective
In a further discussion with New Energy World, Chair of the energy policy debate Sara Vaughan recognised: ‘There’s a huge amount to do. If you’re trying to hit a big target, it’s better to fail honourably and still have a long way to go, than not having tried in the first place. Decarbonisation is a huge challenge, [one] that we are only going to meet through greater use of digitalisation and data.’
She continued: ‘The energy sector has always been fascinated by digitalisation, and there’s a colossal amount of data. I’m not sure that smart meters have been sold to consumers very well. Octopus do it much better when it comes to communication. You have to show that this will make a difference to people’s lives, and they can do [that] now.’
She then looked at the public perception of energy infrastructure. ‘This is the problem, when you talk about infrastructure, people think surely we’ve got enough pylons around the country… and that sort of thing. But this is about radical change. We are talking about putting real power into the hands of the consumer. I think that’s particularly exciting. We talk about flexibility, but consumers say: “If it’s going to save us money, then we’re interested.”’
Vaughan continued: ‘Most people take energy for granted… knowing the lights come on, or that the heat kicks at six o’clock in the evening. But looking forward, all those things can potentially change. People are beginning to appreciate that they will have more control.’
- Further reading: ‘Hot energy policy debate considers the role of citizen engagement in achieving net zero’. At the Energy Institute’s first energy policy debate of 2024, in a discussion moderated by Mike Gibbons CBE FEI, Senior Independent Director of Bluefield Solar Income Fund, four leading industry and academic speakers discussed the topic: ‘The energy sector can't reach net zero without citizen engagement’.
- Find out how, as the UK advances towards its net zero goals, the onus is on electricity networks to enable the transition. Building more physical infrastructure is a big part of the solution, but flexibility also has a significant role to play.