AI Energy Per Query: Same Prompt, Two Different Numbers

In June 2025, Sam Altman put a number on something nobody had pinned down before: a single ChatGPT query, he wrote, uses “about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes.” Google’s own accounting, published two months later, lands nearby: a median Gemini prompt costs 0.24 Wh. Both figures are honestly derived. Both answer a narrower question than the one most readers think they’re asking.

Here’s what almost nobody covering AI energy per query noticed: Google’s own August 2025 report prices the identical median Gemini prompt at either 0.10 Wh or 0.24 Wh — a 2.4x gap — depending only on what gets counted, before anyone disputes a single measurement. That’s not a scandal; it’s a mean-versus-tail problem wearing a press-release headline. And even the honest 0.24 Wh figure describes the cheapest, least representative case: a short, single-turn chat reply, at the exact moment usage is shifting toward reasoning and agentic workloads whose hidden token generation runs many times larger.

What the AI Energy Per Query Number Actually Measures

A watt-hour is a small, useful unit: run a 60-watt appliance for one hour and you’ve used 60 Wh, so a fraction of a watt-hour per query is genuinely tiny. That’s part of why the recent numbers feel reassuring. Independent researchers and the two biggest chat operators have converged on roughly the same ballpark: Epoch AI puts a typical GPT-4o query at about 0.3 Wh, OpenAI cites 0.34 Wh, and Google’s comprehensive figure is 0.24 Wh.

That convergence reads like corroboration — three separate parties checking each other’s math and landing in the same place. It’s really agreement about which question to answer. All three are measuring a short, single-turn chat reply, the cheapest and most common request an AI system handles, not a reasoning chain, an agentic workflow, or a multi-step task — a distinction that’s about to matter a great deal, especially against the backdrop of the grid constraints already reshaping how AI data centers get built.

It wasn’t always this reassuring. The old ~3 Wh per-query figure that circulated for years traces back to a 2023 projection by researcher Alex de Vries, published via his Digiconomist site. Epoch AI’s 2025 reanalysis revised that down roughly tenfold, citing three real changes: newer accelerators (H100 chips rather than A100), models that activate far fewer parameters per token thanks to mixture-of-experts architectures, and a more realistic assumption about how many tokens a typical answer actually contains. It’s a genuinely rigorous piece of reverse-engineering, not industry spin.

Chart it out and the pattern is obvious: four independent estimates cluster between 0.10 and 0.34 Wh, while the old 3 Wh figure sits off in its own compressed lane — a genuine order-of-magnitude correction, and a fast one. That’s worth crediting. The interesting question isn’t whether these numbers are honest; it’s what they’re all quietly agreeing to leave out.

One Prompt, Two Numbers: 0.10 Wh vs 0.24 Wh

Here’s where it gets interesting. Google’s August 2025 report doesn’t give one number for its median Gemini prompt — it gives two, calculated from the exact same interaction. The narrow figure, 0.10 Wh, counts only the electricity the AI accelerator chip draws while actively computing that specific prompt. The comprehensive figure, 0.24 Wh, adds everything a data center actually needs to serve that prompt reliably: idle machines held in reserve for failover and demand spikes, the host server’s CPU and memory, and data-center overhead like cooling and power distribution — similar in spirit to PUE (power usage effectiveness), the multiplier engineers have used for years to describe how much extra energy a facility burns just keeping its servers alive.

Google is unusually candid about which number to trust. In the same report, it calls the narrow accelerator-only method “an optimistic scenario at best” that “substantially underestimates the real operational footprint of AI.” That’s a rarer admission than it should be — most operators would publish the smaller number and let readers assume it’s the whole picture. It also explains why data centers get built the way our earlier look at the AI power bottleneck describes: idle capacity isn’t waste, it’s the thing that makes reliable service possible at all.

None of this erases genuine progress. Over the twelve months from May 2024 to May 2025, Google says the median Gemini text prompt’s energy fell 33x and its total carbon footprint fell 44x — a rate of improvement that would be remarkable in any industry, let alone one this young. Google likes to compare 0.24 Wh to watching television for under nine seconds, and the water accounting moves the same honest direction: 0.12 mL under the narrow method, 0.26 mL once cooling water is included.

Why AI Energy Per Query Hides the Real Tail

Strip away the accounting debate and the underlying physics is simple: energy per query is roughly tokens generated multiplied by energy per token. Energy per token is set by how many floating-point operations (FLOPs) the model needs per token, how efficiently the accelerator converts watts into FLOPs, and how well the serving cluster batches requests — running many users’ prompts through the same GPU at once, which spreads its fixed power draw across more work and flatters the average.

A plain chat reply only generates the words you see. A reasoning model is a different animal: before it answers, it produces a long hidden chain of intermediate tokens — scratch-work the model generates but never shows you — often many times longer than the visible reply. Energy scales with total tokens generated, not with what’s displayed on screen, so a reasoning query can cost several times what a chat query does even when the two answers look identical in length.

Agentic workflows compound the problem further, chaining several such generations — a plan, a tool call, a reflection, another plan — into a single user request, as we’ve argued in looking at what actually drives AI agent memory costs. Published averages also exclude two things that make them systematically optimistic: the embodied manufacturing energy of the chips themselves, and any amortized share of the original training run. None of that makes 0.24 Wh or 0.34 Wh wrong — it makes them descriptions of the cheapest, most common case, not the case that’s growing fastest.

The CERN Rule: Don’t Design for the Average Event

I spent years around particle detectors before I spent them around data centers, and this pattern is deeply familiar. You never characterize a detector by its average event — the average is dominated by cheap, common, uninteresting collisions, while the physics, the trigger design, and the bandwidth budget all live in the tail. A trigger system sized for the mean event rate would be annihilated by the first real burst of interesting data.

Quoting a median chat prompt as “the energy cost of AI” makes the same category error: it sizes the story to the mean while the growth — and the expense — sits in the tail of reasoning chains, agentic loops, and video generation. Data-center capacity planning already works this way, which is exactly why serving architectures built around dense, high-utilization batching — like the wafer-scale approach to inference now reshaping how clusters get provisioned — matter more to the real energy trajectory than any single per-query average ever will.

The Efficiency Gains Are Real, the Question Still Isn’t

None of this is an argument that AI’s energy numbers are fake or that the industry is hiding something sinister. The efficiency curve is genuinely, remarkably steep — a 33x drop in a single year is the kind of number that would make any hardware engineer’s decade. Google’s willingness to publish its narrow figure, its comprehensive figure, and an explicit warning about which one to trust is more transparency than this genre usually gets.

What needs to change is simpler than a methodology fight: operators should start reporting energy by workload class — short chat, reasoning, agentic, multimodal — the way they already report latency by percentile rather than by mean. I’d bet within two years that breakdown becomes standard disclosure, not because regulators demand it, but because the tail is where the real infrastructure spending is going, and investors will want it modeled honestly.

Until then, the next time someone tells you an AI query costs a quarter of a watt-hour, ask them which query. The answer they give back will tell you more about where this technology is headed than the number itself ever could.