Reviewers always ask the right question: if this matters, why haven't the incumbents done it? Here is our honest answer, including the parts that are inconvenient for us.
The local inference ecosystem in 2026 is genuinely excellent. We build on it, we depend on it, and we have no interest in pretending otherwise. What follows is what each layer does well and what specifically none of them provide.
The runtimes: llama.cpp, MLX, Ollama
These are the foundation, and they are good. llama.cpp remains the broadest in model support. MLX, Apple's array framework, has won the Apple Silicon performance race — when Ollama switched its Metal backend to MLX in version 0.19 (March 2026), decode speed on identical hardware roughly doubled, with larger gains on some models. Ollama is the distribution story: 52 million monthly downloads, airgapped deployment support, and metadata logging that enterprises can wire into monitoring.
What they provide: fast, private-by-default inference on hardware you own, with no telemetry once models are downloaded.
What none of them provide, per the published comparative literature and their own documentation:
- No attestation. These runtimes execute locally, but they produce no artifact proving they did. "It ran on-device" is an assertion, not evidence a compliance officer can verify.
- No energy accounting. None instrument their own power draw or report energy per unit of completed work.
- Residency is subtler than "local." KV caches live in memory for the process lifetime; some frameworks persist prompt caches to disk unless explicitly disabled; streaming endpoints, request logs, and crash reports can capture user text. Local is not automatically leak-free.
- No heterogeneous orchestration. Each uses one accelerator well. Splitting work dynamically across CPU, GPU, and Neural Engine under unified memory remains unsolved.
- No built-in authentication. Serving beyond localhost requires bolting on a reverse proxy for TLS and access control.
Apple's on-device features and Private Cloud Compute
Apple deserves specific credit here, because Private Cloud Compute is the most important prior art for our attestation work — and it is genuinely impressive. PCC publishes the measurements of all code running on its nodes to an append-only, cryptographically tamper-proof transparency log, makes those logs and the associated binary images publicly available for inspection, and has user devices refuse to send request payloads to any node whose attested measurements don't match a published release.
That is the correct shape of the idea. Our work differs in where it points:
- PCC attests that a remote server is running inspectable software. Our thrust attests that no remote server was involved at all — that a session ran entirely on-device with zero egress. Different claim, different artifact, complementary architectures.
- PCC is Apple's infrastructure for Apple's models, integrated with iCloud account metering. It is not a general mechanism a law firm can apply to an arbitrary local model on hardware it already owns.
- Apple's fully on-device models are budgeted to run on every supported iPhone — a constraint that has meant roughly 2GB-class models with a small context window, while a 36GB Mac can budget 20GB+ and run models an order of magnitude larger.
Confidential computing broadly
Trusted execution environments and confidential-computing approaches solve a real and adjacent problem: protecting workloads from the infrastructure they run on. They are largely oriented toward the datacenter, where the threat model is a hostile or curious host. Our threat model is the inverse and simpler: the user owns the machine and trusts it — what they need is proof to show a third party that the data stayed there.
So what's actually missing
Assembling the honest list: nobody publishes a reproducible methodology for comparing cloud and local energy per unit of completed work (Objective 1). Nobody orchestrates inference across all three accelerators under unified memory (Objective 2). Nobody produces a portable, hardware-backed artifact proving an inference session had zero data egress on consumer hardware (Objective 3). Each gap is individually plausible to close. Together they are the difference between "local AI works" and "local AI is adoptable by an institution that must prove what it did."
We would be glad to be wrong about any of this. If you know of work that closes one of these gaps, tell us and we will cite it here: support@yforest.ai.