Nobody Can Honestly Compare Cloud and Local AI Energy. Here's Why.

A claim you'll hear from cloud providers: datacenter inference is more energy-efficient than running AI on your laptop, because batching amortizes cost across thousands of users. A claim you'll hear from local-AI advocates: your MacBook draws 40 watts while a datacenter GPU node draws kilowatts, so local obviously wins.

Both arguments are unfalsifiable as stated — because no shared methodology exists to test them. This note explains the four measurement problems that make the comparison dishonest today, and what an honest benchmark would require. Building that benchmark is the first thrust of our AEOLI research program.

Problem 1: The boundaries aren't the same

When a cloud provider reports energy per token, the number typically covers the accelerator doing the math. It typically does not cover: power usage effectiveness (the cooling, power conversion, and facility overhead layered on every datacenter watt), idle provisioning (capacity kept warm to absorb demand spikes, burning energy whether or not you're using it), network transit between you and the facility, and the redundancy inherent in high-availability infrastructure.

When a local-AI advocate reports laptop wattage, the number typically ignores the opposite set of sins: the device was already on (so what's the marginal draw of inference?), the model may be smaller and lower-quality than the cloud model it's being compared against, and single-user hardware amortizes nothing.

Neither side is lying, exactly. They're measuring different systems and calling them the same thing.

Problem 2: Tokens are not a unit of usefulness

Energy per token is the standard metric, and it's broken for comparison purposes. A 3-billion-parameter local model and a frontier cloud model both emit "tokens," but not tokens of equal value: the smaller model may need three attempts, longer outputs, or human correction to complete the same task. If the local model burns half the energy per token but requires twice the tokens to produce an acceptable answer, nothing was saved.

An honest comparison must normalize by task outcome, not token count — energy per quality-adjusted unit of completed work. That requires held-out task suites (document summarization, retrieval-backed question answering, data transformation) with defined acceptance criteria, scored blind. This is a measurement-science problem before it is a systems problem, and to our knowledge no published benchmark does it end-to-end.

Problem 3: The interactive workload is not the batched workload

Batching is real: a datacenter serving thousands of concurrent requests genuinely amortizes cost in ways a single laptop cannot. But the workloads where local AI matters most — a lawyer analyzing privileged documents, a clinician summarizing a record, a school processing student work — are interactive, intermittent, and single-user. For these, the relevant cloud-side number includes the idle and provisioning overhead of keeping capacity ready for you, not the theoretical throughput of a saturated cluster. Comparing a saturated datacenter's best case against a laptop's worst case is the most common rhetorical move in this debate, in both directions.

Problem 4: Nobody's numbers are verifiable

Cloud energy figures arrive via sustainability reports and press statements, unauditable by design. Local measurements arrive via screenshots of activity monitors, unreproducible by accident. An honest benchmark needs instrumented, wall-power-measured device runs with published harnesses anyone can rerun — and cloud-side estimates built transparently from public datacenter literature, with every assumption stated and adjustable.

What we're building

The EQAT benchmark (Energy per Quality-Adjusted Token) is our attempt at the honest version: open methodology, open harness, reproducible device measurements across multiple hardware classes, cloud-side estimates with stated assumptions, and task-outcome normalization. We expect the results to be less flattering to both camps than their marketing suggests — which is roughly how you know a measurement is honest.