What consumer hardware can already do
Modern consumer silicon is architecturally suited to inference in ways that were not true five years ago. Unified memory architectures (Apple M-series) let models occupy tens of gigabytes of memory shared across CPU, GPU, and Neural Engine without the transfer penalties of discrete-GPU systems — which is why 30-billion-parameter-class models now run interactively on a Mac Studio, and why our own products run production LLM workloads on ordinary MacBooks today. On the Windows side, the NPU has gone from novelty to default: AI-capable PCs are projected to dominate shipments within the refresh cycle. The substrate is not hypothetical. It is being purchased at retail, right now, for reasons unrelated to AI — and most of its AI capability sits idle.
What advancements are actually necessary
We try to be precise about the gaps, because they define our research agenda:
- Heterogeneous orchestration. Today's local runtimes mostly use one accelerator well. Splitting inference optimally across CPU, GPU, and NPU under unified memory — dynamically, per workload — is unsolved. (Our AEOLI Objective 2.)
- Quality assurance without the cloud. Local deployment currently means trusting a small model blind. Calibrated confidence, bounded-hallucination guarantees, and local escalation decisions are open problems. (Objectives 2–3, and the routing seam described on the Architect page.)
- Verifiable privacy. "It's local, trust me" must become a checkable artifact before compliance-bound institutions can adopt at scale. (Objective 3 — egress attestation.)
- Honest energy accounting. No accepted methodology compares cloud and local energy per unit of completed work. Without it, every claim on this page — including ours — is unfalsifiable. (Objective 1 — the EQAT benchmark. See Research Note 01.)
- Fleet manageability. Enterprises deploy software to thousands of managed devices routinely; doing so for models, updates, and attestation policies needs tooling that doesn't exist yet.
The grid dividend, stated conservatively
We will not publish a savings number we have not measured — that is precisely the dishonesty our benchmark exists to end. But the shape of the argument can be stated with published inputs: U.S. data centers are projected to consume 325–580 TWh by 2028, inference constitutes a growing majority share of AI compute, and the routing literature indicates most interactive queries do not require frontier capability. If even a modest fraction of routine interactive inference migrated to marginal draw on already-powered consumer devices, the avoided load would be measured in whole power plants — along with the associated ~1.2 gallons of water per kWh of generation that never has to be consumed. Establishing the actual size of that dividend, with reproducible measurements, is the core purpose of the EQAT program. We expect the honest number to be smaller than advocates hope and far larger than incumbents concede.