Realtime voice AI bills for every second it listens. So we moved it onto the phone.

Build a voice assistant into your app for smart glasses and you hit a cost shape that gets worse the better your product does.

Realtime speech-to-speech models bill per token like any other model, except audio tokens cost roughly eight to seventeen times what text tokens cost on the same model. And an assistant on your face isn't a chat window someone opens twice a day. It listens. That's the entire point of it. Every second of audio in and every second of speech out is metered, so the bill scales with engagement, which is exactly the number you were trying to grow.

You can't optimise your way out of that. Shorter replies and tighter prompts move it a few percent. The cost is structural: you are renting a conversation by the second from someone else's datacentre.

So we moved the conversation onto the phone.

Can you run realtime AI models locally on mobile?

Yes. A realtime voice assistant — speech in, a model thinking, speech out — runs entirely on a modern phone today, with no server and no per-token cost. We ship it on both iOS and Android.

The practical limits are worth stating plainly, because most write-ups skip them. Local models on mobile are small models: we run Qwen 3 at 1.7B, 4B and 8B parameters, not frontier-scale. They hold a conversation and call tools reliably; they are not GPT-5. Memory is the real constraint — a 4 GB phone runs the 1.7B comfortably, 8 GB reaches the 4B — and the weights are a download the user has to accept, typically one to two gigabytes.

What you get in exchange is a conversation that costs nothing to run, works with no signal, and never sends audio anywhere.

What running the model on mobile actually means

Select a local-* model in Extentos and the assistant's brain runs on the mobile device itself. Speech recognition on the phone, an open-weights model generating the reply on the phone, the phone speaking it back. No tokens leave. Nothing is billed, because there is nothing to bill — not a discount, not a free tier with a ceiling, just no meter.

It also keeps working in airplane mode, and no conversation content reaches us or any model vendor. Those are real, but they follow from the architecture rather than being the pitch.

Your code doesn't change. The same block you'd write against a cloud model:

glasses.assistant.start(provider) {
    instructions = "You help the user capture notes hands-free."
    tool("save_note", "Save a note the user dictates") { args -> notes.save(args) }
}

Same tools, same events, same wake and sleep. Only the brain changes. That's deliberate — a local tier you have to write a second app for isn't a local tier, it's a second product.

Both platforms are published, so running local models on mobile is a dependency line rather than a research project: com.extentos:glasses-local on Maven Central for Android, and the GlassesLocal product from swift-glasses for iOS.

Why running models locally on mobile is hard (and it isn't the model)

Downloading weights onto a mobile device is the easy half. Anyone can do that. The hard half is that a conversation is a concurrency problem, and small models make it worse by being slower.

Someone interrupts mid-sentence. A tool call takes two seconds while the user keeps talking. Speech recognition returns a partial that turns out to be the tail of the assistant's own voice coming back through the microphone. Do that with mutable flags and background jobs and you get a system that works in the demo and fails in the seventh conversation — we know, because that's what our first attempt did, across seven hardware sessions and a new failure mode per patch.

So the second attempt isn't a patch. The entire turn loop is a pure state machine in our Rust core, shared by iOS and Android:

• One turn at a time by construction. Not enforced by a lock, but by there being one state rather than N concurrent jobs.
• Overlapping speech is topologically impossible. Re-entering the speaking state passes through an exit whose first effect cancels the previous utterance. You can't get two voices out because there's no path through the graph that produces them.
• Barge-in is a transition, not a feature. Speech during thinking cancels inference. Speech during speaking cancels speech. It isn't special-cased anywhere.
• A wedged component cannot deafen the assistant. Listening is never gated on the mouth or the mind, so a stuck effect can't stop the machine consuming events.

The platform shells hold no turn logic at all. They execute effects and feed events. Which is why the local tier landed on iOS and Android with the same behaviour rather than two subtly different implementations that drift for a year.

The voice is on-device too. Eleven neural speakers, running on the phone, no cloud round trip.

What we changed this week

Two things from the last few days, because they're the honest illustration of how this gets better — not a roadmap, just what happened.

Eleven voices appeared out of nothing. Kokoro is a multi-speaker model and the file already on every phone contains all eleven. Our synthesizer passed a hardcoded 0 and threw the other ten away. Fixing that cost one parameter. No extra download, no extra memory, no extra load time — the voices had been sitting on users' phones the whole time, unreachable.

A 4 GB iPhone was beating a 5.4 GB Android. Same model, and the Android was falling back to cloud while the iPhone ran it happily. The cause was quantisation: our Android floor model shipped 8-bit where every other model on both platforms was 4-bit, so it carried double the weights. Measured on a Galaxy A25: 30% less memory for identical generation speed. Mid-range Android phones went from cloud fallback to running locally.

Neither of those was new technology. Both were using what we already had properly. That's most of what improving an on-device tier looks like, and we intend to keep doing it as the models, the runtimes and the phones move — because all three are moving fast, and none of it requires you to change a line.

The optional part: Automatic

Pinning a model gives you zero cost and complete predictability. But it puts a question on you that you can't really answer: which model can each of my users' phones run? You're shipping to thousands of devices with wildly different memory.

So we added Automatic. Select it and the SDK reads the device at session start, runs the best model that phone can genuinely sustain, and falls back to the managed cloud when a device can't run one at all. It reports which model it chose, every session, so you always know where a conversation ran.

It's an addition, not a replacement. Pin a model and you keep absolute zero. Choose Automatic and you trade that for "works on every phone I ship to."

Common questions

Can you run realtime AI models locally on mobile? Yes. A realtime voice assistant — speech in, a model thinking, speech out — runs entirely on a modern phone today, with no server and no per-token cost. Extentos ships this on both iOS and Android. The models are small ones: Qwen 3 at 1.7B, 4B and 8B parameters rather than frontier-scale. They hold a conversation and call tools reliably, but they are not GPT-5.

What does it cost to run a local model on a phone? Nothing per token. The model runs on the device, so there is no meter — not a discount or a capped free tier. The costs are the user's battery and a one to two gigabyte model download they have to accept.

How much memory does an on-device LLM need on a phone? Memory is the binding constraint. A 4 GB phone runs a 1.7B model at 4-bit quantisation comfortably; 8 GB reaches a 4B model. Quantisation matters as much as parameter count — measured on a Galaxy A25, moving a 1.7B model from 8-bit to 4-bit cut peak memory by 30% with no change in generation speed.

Does a local voice assistant work offline? Yes. With the model and voice downloaded, the whole conversation runs on the phone, so it keeps working in airplane mode. No conversation audio or text leaves the device.

A question for you

Automatic currently falls back to cloud when a device can't run a local model, because we assumed a working assistant beats a free one.

That assumption might be wrong for some of you. If you're building something where cost must be zero by construction — or where audio must never leave the device, no exceptions — you'd want Automatic to pick the best local model and then simply refuse rather than reach for the cloud.

Should Automatic let you switch the cloud fallback off entirely? And if you'd want that, is it a per-project setting, or something you'd flip per user?

Tell us. That answer decides what we build next here.

Start with the on-device models guide — the model ladder, the download flow, and the voices.