The managed AI gateway

The assistant routes voice AI through Extentos's managed gateway on Extentos's keys — content is relayed but never stored, and usage is metered. Handler-code AI you call yourself never touches the gateway at all.

There are two distinct ways AI shows up in an Extentos app, and the privacy and billing story is different for each. Getting them straight is the whole point of this page.

The rest of this page is about the managed gateway (the left column). For the right column — calling your own provider from a handler — see credentials; Extentos is not involved in those calls at all.

What the managed gateway is

When your app uses the Phase 4 assistant runtime (glasses.assistant.start(...)), the voice-AI traffic does not carry an API key in your app. Instead, the SDK opens a WebSocket to Extentos's gateway (wss://api.extentos.com/v1/realtime), and the gateway relays the session to the AI provider on Extentos's key. The default provider is OpenAI Realtime (default model gpt-realtime-2); the gateway also routes xAI Grok and Google Gemini Live voice models — switch to one in the dashboard's Agent settings and the gateway relays to that vendor on Extentos's key, with no change to your app. Gemini 3.1 Flash Live additionally accepts streaming video input (sendVideoFrame) — the only realtime lineup entry that can see what the glasses see. Each realtime model brings its own voices (they aren't interchangeable across vendors — the dashboard scopes the voice picker to the model you pick).

This is deliberate: it means a developer can build and ship a voice-AI glasses app with zero AI configuration — no key to obtain, store, rotate, or leak. The assistant just works.

// No API key anywhere in your code — the gateway supplies it.
val session = glasses.assistant.start(AssistantProvider.Managed()) {
    instructions = "You are a running companion…"
    tool("get_pace", "The runner's current pace") { ToolResult.Ok(pace()) }
}

Heads-up: the assistant runtime has shipped in com.extentos:glasses since 1.4.0 (install the current release — see below) and in the iOS SDK's GlassesCore product from github.com/extentos/swift-glasses. See assistant, Android install, and iOS install.

Compaction-model providers

The gateway also proxies the assistant's memory-compaction model — the chat model that summarizes older turns when a conversation outgrows the context window (chosen per project in the dashboard's Agent section). Alongside OpenAI you can pick Google Gemini or Anthropic Claude compaction models: the gateway routes each to its provider on Extentos's managed keys (Gemini via Google's OpenAI-compatible endpoint, Claude via Anthropic's Messages API) and meters the usage exactly like any other gateway call, at the provider's list price.

What the gateway does and doesn't do with your data

The gateway is a transient relay, not a data store:

• Audio frames, transcripts, and model responses are forwarded verbatim in both directions and never persisted. Binary audio is never even read.
• The only thing recorded is the provider's usage object at the end of each response: token counts (input/output, text/audio/cached splits), the model, and the computed cost_usd — or, for per-minute-billed voice models (xAI Grok voice), the billed session seconds. No conversation content is stored.

So the precise framing is: for the managed gateway, Extentos is in the path but never inspects or stores the content — it reads the usage object and forwards everything else. (The older docs claimed Extentos is "never in the path / never proxies AI." That was only ever true for handler-code BYOK — the right column above — and it's the framing this page corrects.) See security for the full data-handling commitments and the content/PII boundary.

There is no bring-your-own-key option

The assistant's voice AI always runs on Extentos's provider key, metered to your account. You cannot supply your own key, and this is not a roadmap item — it is the design. Holding one provider relationship is what lets the gateway meter uniformly, add models without per-customer key management, and keep the SDK free of any credential.

If the provider relationship needs to be yours — an existing commitment, a specific tier, a compliance posture — talk to us about whether Extentos is the right fit, rather than waiting for a feature that isn't coming.

Calling a non-assistant AI from your own handler code (a vision model over a captured photo, a translation API) is a separate thing and always works — that is your app making its own request with its own key, which Extentos neither routes nor meters.

Identity: how the gateway knows who's calling

Every gateway connection presents one of three credentials, and which one you get is not something you configure.

In the browser simulator the backend mints a session-scoped token. Everywhere else — a debug build on real glasses, a TestFlight build, an App Store or Play release — your app presents its baked project key. That is the normal production path, not a development shortcut.

Platform attestation (App Attest, Play Integrity) is preferred automatically wherever it is available, but it is an upgrade, never a requirement, and you should not architect around it. On Android it is not available to customer apps at all: verifying a Play Integrity token means calling Google's decodeIntegrityToken with Extentos's service account against your package, which would require you to grant us access to your Play Console.

A build that can't attest — a sideloaded development build, or any Android app not distributed through Play — presents its baked project key instead. That key is written by generateConnectionModule, and it is the whole setup: there is nothing for you to enable, register, or renew, on either platform.

The key lives in your binary, so treat it like any public API key. Spend is bounded whatever happens to it: usage is metered to your project against the per-project cap and your prepaid balance, and you can revoke a key at any time, which rotates it on the next scaffold.

Metering and billing

Metering and billing are live. Every managed-gateway response is priced at the provider's list price (no markup) and recorded against your project — token counts for token-billed models, or connected minutes for per-minute voice models (xAI Grok voice). You can see usage per project in the dashboard's Agent section, and combined across projects in the account Billing hub.

It's a prepaid credit model. Every account starts with $2 of free managed-gateway credit; after that, usage draws down a prepaid balance you top up in the Billing hub (buy any amount; optional auto-reload). When the balance reaches zero, managed-gateway sessions stop until you add credits — prepaid, so there's never a surprise invoice. All managed usage counts toward the balance regardless of environment (simulator, dev, and real hardware alike).

The free-forever surfaces (discovery, validation, and guidance MCP tools, on-device simulation, real-hardware DAT testing) stay free regardless. See pricing for the full money story.

When the gateway refuses a session

A refusal is not a network problem, and your app should not present it as one. There is one reason a healthy app gets refused — the balance ran out — and the gateway names it. The SDK carries that reason to your app: the connect failure message is <reason>: <detail>, so you can branch on the prefix.

On SDK 2.3.0 and earlier all three arrive instead as realtime WebSocket open timed out, roughly ten seconds after the attempt, which reads as a dead network. If a voice agent on an older SDK has stopped opening sessions, check the balance in the Billing hub before you suspect connectivity.

When to use which

• Managed gateway (the assistant's AI): voice AI with zero config, running on Extentos's provider with usage metered against your credit balance. This is the path every voice app takes today.
• Your own AI calls (a different thing): you're calling a non-assistant AI — a vision model on a captured photo, a translation API — from your own handler. That never touches the gateway; wire the key yourself per credentials.

Related

• Assistant runtime — the glasses.assistant.* API the gateway powers
• Pricing — free surfaces, metered gateway, the prepaid credit model
• Security — the data-handling boundary and what's collected
• Credentials — Meta DAT setup and handler-code provider keys