Choose your path

Extentos has two integration paths, and most apps need only the first. The difference is not how much of Extentos you get — the voice runtime is identical on both. The difference is whether you need vendor credentials at all.

Which path are you on?

If you can't tell yet, start on the voice path. Adding camera later is additive — your handlers, tools and conversation logic don't change.

The voice path

Smart glasses with a microphone and speaker present themselves to the phone as a standard Bluetooth hands-free audio device. Extentos reaches them through the operating system's own audio routing, so there is no vendor SDK in the picture.

What you need:

• The Extentos SDK (com.extentos:glasses). com.extentos:glasses-ui is optional on this path — it only provides the packaged connection page, which a voice app doesn't render.
• Microphone permission (RECORD_AUDIO / NSMicrophoneUsageDescription), requested at runtime before you start the assistant
• INTERNET, unless you run entirely on local models — the managed gateway is a network call
• A foreground service if the assistant must keep listening while your app is backgrounded: FOREGROUND_SERVICE + FOREGROUND_SERVICE_MICROPHONE on Android, UIBackgroundModes: [audio] on iOS
• On iOS only, usedCapabilities: [.microphone, .speaker] in your config — it picks the transport there, and leaving it empty resolves to real glasses
• Your project key, if you use the assistant: buildConfigField("String", "EXTENTOS_PROJECT_KEY", …) on Android, EXTENTOSProjectKey in the Info.plist on iOS. Neither causes a build error when missing — the assistant just fails at runtime with AssistantError.NoApiKey
• On Android, buildFeatures { buildConfig = true } — AGP 8 defaults it off and the init snippet reads BuildConfig.DEBUG

Full sequencing for both platforms is in the voice-assistant guide.

What you do not need: a Meta Wearables Developer Center account, a Meta App ID or Client Token, a GitHub Packages token, a connection page, or a device pairing flow. None of it applies.

val glasses = ExtentosGlasses.create(
    ExtentosConfig(applicationContext = applicationContext)
)

// `start` and `wake` are suspend functions, so they run inside a coroutine.
scope.launch {
    val session = glasses.assistant.start(provider = AssistantProvider.Managed()) {
        instructions = "You are a running coach. Speak briefly."

        tool("log_split", "Record a lap split for the current run.") {
            ToolResult.Ok("split recorded")
        }
    }
    // Any trigger that calls session.wake() works — here, a wake phrase.
    glasses.voice.onPhrase("hey coach") { session.wake() }
}

You don't call connect() yourself — assistant.start(...) brings the transport up. (glasses.connection.connect() exists for apps that want to drive connection explicitly, and is what the packaged connection page calls.)

That is the integration — five short items, none of them a vendor account. You get the complete agent runtime: turn-taking, barge-in, tool calling, conversation memory, local and cloud models, and automatic routing between them.

You also get the primitives on their own, without an assistant session — including the high-quality on-device voice, which reads any text your app has in a natural voice rather than the platform's robotic one:

glasses.audio.speak(summary, SpeakConfig(voice = "kokoro-emma"))

That works on this path with nothing paired at all. See on-device models.

What it runs on

Anything the phone can route voice through. The SDK pins whatever exposes a Bluetooth hands-free (HFP) route and does no device-identity checking at all, so all of these are the same code path:

That last row matters more than it looks: your app never dead-ends because the glasses are in their case. The phone always has a microphone and a speaker, so the agent stays usable and picks the glasses back up when they reconnect.

The device path

Camera and display go through the vendor's own SDK, which means vendor credentials and a device connection flow. This is where the Meta setup lives.

Add the vendor module alongside the core:

implementation("com.extentos:glasses:2.1.2")
implementation("com.extentos:glasses-meta:2.1.2")   // camera + display

The vendor transport registers itself, so TransportChoice.Auto resolves to real glasses with no extra code. Meta needs glasses-meta because its artifacts sit behind Meta's own credentialed repo; Brilliant Labs needs nothing at all — it ships inside the SDK on both platforms, though it remains preview and unverified on hardware. Then follow Vendors: Meta Ray-Ban for the credential setup, or Android XR for projected glasses.

Moving from voice to device later

Your conversation logic does not change — handlers, tools and prompts are written against capabilities (glasses.audio.*, glasses.assistant.*), never against a vendor. What does change is the bootstrap around them, because the vendor transport needs things a voice app never needed:

1. Add the vendor dependency.
2. Complete the vendor's credential setup.
3. Request BLUETOOTH_CONNECT and CAMERA before ExtentosGlasses.create(...). TransportChoice.Auto decides the transport inside create(...) and keeps it; the vendor arm can only claim if it can see that glasses are bonded, and that read needs BLUETOOTH_CONNECT.
4. Pass an activityProvider to ExtentosConfig — vendor registration needs a live Activity, and it defaults to null.
5. Guard the new calls on the capability:

if (glasses.capabilities.camera) {
    val photo = glasses.camera.capturePhoto()
}

Steps 3 and 4 are the ones that surprise people; the voice-assistant guide has the worked bootstrap for a voice and camera app.

That guard is the normal shape for every capability on every path — one build ships to devices with different hardware, and absent capabilities are safe, never thrown. See capabilities.

What a device app does when there are no glasses

Once you're on the device path, your app still has to run for a user whose glasses are in a drawer. It degrades rather than breaking:

• The voice agent keeps working over whatever audio the phone has routed — earbuds, or the phone itself.
• glasses.capabilities.camera reports false, so guarded code simply skips the camera branch.
• An unguarded capturePhoto() returns CaptureError.PlatformError with code = "system_audio_no_camera". It does not crash, and it does not hang. (Not NotConnected — on this transport the connection genuinely is up; there is simply no camera behind it.)

So a camera app and a voice app are the same app on a phone with no glasses connected — you lose exactly the capabilities the hardware was providing, and nothing else.

Glasses paired, but in their case

This is the common one, and it resolves differently from "never paired" — a Bluetooth bond is a stored pairing record that survives the glasses being powered off. So the vendor arm still claims, and transportChosen reads REAL_META even though there is nothing switched on to talk to.

Voice still works. The glasses' microphone and speaker were never reached through the vendor SDK on either platform — they are ordinary Bluetooth hands-free audio, routed by the OS. The Meta transport uses the same audio path as the vendorless baseline, so when the glasses are off the OS routes to whatever else is available: earbuds, or the phone itself. Your assistant keeps running and the user keeps talking to it.

Camera doesn't, and says so. There's no device session, so capturePhoto() returns CaptureError.NotConnected — the connection genuinely isn't Active.

The two cases stay distinguishable because capabilities reports what the transport's vendor provides, not what is switched on right now: on REAL_META with the glasses in their case, capabilities.camera is still true while connection.state is not Active. Never-paired is the opposite — capabilities.camera is false and the state is Active. So a camera app guards on both:

val canCapture = glasses.capabilities.camera &&
    glasses.connection.state.value is GlassesState.Active

When the user takes the glasses out of the case, the device session comes back on its own and capture starts working again — no restart, because the transport was already REAL_META. That's the difference from the never-paired case above, which does need a restart.

Already shipping? Upgrading from an older SDK

If your app was built before com.extentos:glasses-meta existed, the Meta transport used to live inside com.extentos:glasses. It doesn't any more, so a camera or display app needs one added line:

implementation("com.extentos:glasses:2.1.2")
implementation("com.extentos:glasses-ui:2.1.2")
implementation("com.extentos:glasses-meta:2.1.2")   // ← add this if you use camera or display

No Kotlin change is needed. The module registers itself at app start, so TransportChoice.Auto resolves to real glasses exactly as before.

If you skip the new dependency, nothing fails loudly. Your build succeeds and your voice features keep working, because the SDK falls back to the audio baseline — but capabilities.camera becomes false and camera calls start returning errors. The SDK logs a warning at startup naming the missing module (adb logcat -s Extentos:W), and glasses.transportChosen will read SYSTEM_AUDIO where you expect REAL_META. Check both after upgrading.

The honest boundary: the code delta is small, but the vendor's own credential setup (developer account, app registration, access token) is real work that Extentos cannot remove. Doing it later rather than up front is the saving.

Related