On a random Tuesday at 2 am, a storm front pushes in from the west and a few wind turbines off the Scottish coast are spinning somewhere near their rated capacity. Nobody’s awake, and demand is low.

Across the transmission networks, prices are sliding towards zero and in recent years, below zero. The turbines get paid to shut down, and energy is curtailed.

On a random Thursday evening, on the other hand, there’s a warm spell in the middle of July. As the sun falls, solar generation that had flooded the grid with cheap electricity at midday drops as the sun sets. People are coming home, more TVs turn on, air conditioning, and cooking starts. This means that demand ramps up just as generation starts to dwindle.

Surplus and storage, cheap and expensive power — these situations are expressions of the same underlying problem. The energy transition has made for enormous progress in building clean generation capacity. In 2024, wind became Britain’s largest single electricity source, contributing around 30% of the national power mix.

The capacity for renewable energy generation is there, the emissions at the point of generation are near zero. The harder problem here is intermittency: energy is abundant at the wrong times when demand is low, and scarce at the right ones, when demand is high.

As you can tell, this problem is very expensive.

In 2025, Britain spent £1.46 billion managing this by paying Scottish wind farms to switch off because the transmission network couldn’t carry their power south, while also paying gas peaker plants in England to fire up instead. This figure is projected to reach £8 billion by 2030¹ if nothing changes. This isn’t only a timing problem, but also a transmission problem: we can generate enough electricity, but we still lack the wires and storage needed to move power to the right place at the right time.

Enter Storage #

It helps to understand how the grid actually stays balanced to better understand the issue.

The grid operates in real-time, and supply and demand must match every second of the day. If there’s too much generation and not enough consumption, automatic protection systems(intelligent devices that monitor abnormalities in the grid) can start to trip because the frequency of the grid rises above its target operating frequency of 50Hz. Too little generation, and the frequency falls, leading to cascading risks of blackouts.

For most of the grid’s history, this balancing process was managed with dispatchable generation; power plants that can be told when to ramp up or ramp down. Natural gas peakers in particular have served as the grid’s shock absorber, usually staying idle most of the year, and ready within minutes to respond to demand spikes². The entire system is designed around the assumption that generators can be switched on or off at will.

We can’t exactly tell the wind when to blow, or the sun when to not shine.

Because grid operators cannot decide when to generate, they can only decide when to curtail. As renewable penetration grows, the peaker plants work harder as the swings between surplus and shortage become wider and more frequent. Gas peakers run more starts, burning fuel and accumulating maintenance costs during the ramp-up phase which is—ironically— the least efficient and most polluting part of their operating cycle³.

The traditional answer to this problem has been pumped hydroelectric storage, where we pump water uphill into a reservoir when power is cheap, release it through turbines when power is expensive. With round-trip efficiencies typically at 70-85%⁴, and larger hydro plants storing huge amounts of energy, this works really well. Britain’s Dinorwig power station in Wales for example, can go from zero to full output(1.7GW) in under 16 seconds⁵.

Pumped hydro’s main drawback is geography. You can’t build a plant in most terrains, and it takes about a decade or so to plan, permit, and build. There’s only so many viable sites. and most of the best ones are already being(or plan to be)used.

With energy, speed matters. The grid needs power that can respond quickly, and the fastest available response is often called first to keep the grid stable, even if it means paying once to curtail energy and again to buy where it’s needed.

Batteries, however, have none of these constraints. A battery energy storage system(BESS) can be sited almost anywhere— next to a substation, on brownfield land, solar farms, neighborhood microgrids— and it is modular, meaning capacity can be added as needed.

And it’s fast.

While a gas peaker takes several minutes to ramp up in response to demand, a battery can respond in milliseconds⁶. Fast Frequency Response services where a battery must react to a drop in grid frequency within 250 milliseconds are already a mature commercial product in Britain.

Batteries have also gotten very cheap. In 2010, lithium-ion battery packs cost around $1474 per kilowatt-hour(kWh). By 2025, that number had dropped all the way to $108/kWh, according to BloombergNEF’s annual Lithium-Ion Battery Price Survey⁷.

That’s a decline of ~93% in 15 years.

For stationary storage specifically(my focus for this series), battery pack prices fell to $70/kWh in 2025, a 40% drop⁸ from the year before, making stationary storage cheaper than electric vehicle packs for the first time.

In Britain alone, operational grid-scale battery capacity increased by 45% in 2025 at around 12.9GWh, and over 440GWh of battery storage capacity is currently in the UK development pipeline in various stages⁹.

A battery sited next to a Scottish substation could absorb surplus wind that would otherwise be curtailed, hold it, and release it into the grid during the evening ramp hours later, solving the mismatch problem on both ends.

Grid-scale batteries are predominantly-owned and operated by independent merchant energy companies. They could be standalone developers, arms of larger utilities, or projects backed by infrastructure funds looking for long-term yield. They site a battery, connect it to the grid, and the battery has to pay for itself entirely through the revenue it can extract from various markets with no subsidy floor in most cases.

Batteries are evidently becoming a very large part of the energy transition, but what stops them from becoming a money sink?

How Batteries Actually Make Money #

When a battery captures the value between electricity price spreads, this is called energy price arbitrage. Buy electricity to charge when prices are low, sell back to the grid when prices are high.

The concept is simple, but the execution is… not.

A day-ahead electricity market is a liquid market where generators and buyers submit bids ahead of time, and prices are set hour by hour based on expected supply and demand for the next 24 hours. In Britain, the day-ahead hourly auctions runs on two exchanges, EPEX and N2EX, each morning at around 09:20 and 09:50 GMT, and each auction clears a single price for each hour of the following power day¹⁰.

The resulting price profile on a typical GB weekday has a recognizable double-peak shape.

Overnight(~11pm-6am) demand is low, industrial users are (mostly) quiet, and whatever wind is blowing or nuclear plants running flood the market, making price cheaper. Morning peaks(~7-9am) as factories start, morning routines begin, and prices spike. Then a solar-driven dip in spring and summer(less apparent in colder months), and the famous evening ramp follows with solar generation falling, people come home, ACs and or heaters come on, and prices spike again often to their daily highs— between around 5-8pm, before falling again towards midnight.

On a good day in a volatile market, the spread a battery tries to capture would yield substantial profit, but might not be worth the battery’s degradation cost on a quiet summer weekend.

Now, for several reasons, this process isn’t simple.

For one, you don’t know tomorrow’s prices when you have to set your charging schedule today. Because the auction closes in the morning, you need to have submitted your charge/discharge plan before prices are known, meaning you make your bids on a forecast.

The battery also has physical limits. The amount of energy you can store is bounded by a certain capacity for each battery, the rate at which you can charge/discharge(called the power rating), and (obviously) you can’t discharge more than you have stored. And arguably the most important, the batteries degrade with each cycle, which reduces the battery’s capacity and lifetime. An operator would take into account these factors to avoid aggressively responding to certain peaks in the day to prolong the health of the battery, and/or save more energy for potentially more expensive peaks in the day.

The market is becoming increasingly competitive. If every operator followed the same strategy and charged/discharged at the same time, they all bid into the same window, which compresses the spread that can be captured per battery. Essentially, the more batteries on the grid, the more saturated the market becomes, which erodes the opportunity that justified their installation in the first place.

But this isn’t completely bad news.

A useful way to think about it is to create a theoretical benchmark for this tradeoff; the perfect foresight value.

What would a battery earn over a given period if it knew every future price exactly and could optimize dispatch perfectly.

That’s the theoretical ceiling on arbitrage revenue, which no real operator can achieve, but the gap between a really smart forecast and that perfect foresight is a good measure of how much value could be left on the table.

Closing that gap is a problem that allows for really interesting engineering.

Intelligent Forecasts #

The forecast problem for a battery operator is largely a supervised machine learning problem that then serves as an input into a dispatch algorithm that determines a battery’s charging schedules and in turn, revenue.

Because electricity prices have a rich, learnable structure with different patterns to learn from, a model can learn from these simultaneously, update its weighting in response to new data, and meaningfully outperform naive baselines like predicting based off of a rolling average of past prices, for example.

There’s a wide variety of model architectures to choose from, but an easily forgotten issue to look out for is temporal leakage with data we choose to train on. Because forecasting is a time-series problem, you can only use information that would have been available at the time of prediction.

So no future prices, weather patterns, or anything that could inadvertently leak tomorrow’s outcome as features or you end up with a model that gives inflated backtest results and pretty bad results in real life scheduling and money lost on bad trades.

Decision tree models and gradient boosting methods like XGBoost or LightGBM are often used in practice as strong baseline models, combined with a robust set of features from data, like lagged prices at multiple time horizons, wind speed, solar irradiance forecasts, and recent gas price levels. For longer-range temporal structures, convolutional networks and transformer-based architectures are starting to show promise in research, along with reinforcement learning in their ability to capture dependencies across hundreds of hours that gradient boosting misses. I started with a naive lag baseline and followed with a LightGBM model while building for reasons I’ll get into in part two of this series, and explore the tradeoffs that came with it.

What I’m trying to build #

I recently built a tool that teaches a battery when to store and sell back electricity to the grid and test how well it would have done using historical electricity price data across different European regions in an attempt to understand this space better.

The energy transition is an economics, engineering and software problem all in one, and framing it as purely(or mainly) a physical engineering issue solved by just increasing renewable generation capacity without considering how best we can optimize what we currently have and what more we build, leaves value on the table and leads to more energy being curtailed down the line.

I believe this is one of the most important technical problems of the next decade, and in part 2, I’ll walk through how I built the tool: how the optimizer is formulated, what the backtesting results look like, the limitations of my current model, and what I’ll be considering going forward.

Check it out here ahead of part 2!

https://bessarbitragedemo.streamlit.app/

Until next time :)