Runtime internals (iOS)

The structure below is verified against the current iOS source. Package install: Install.

Shell over a shared core

The Extentos iOS SDK is a thin Swift shell over a shared Rust core. The same extentos-core powers both platforms: vendor-agnostic logic — transport selection, state machines, data shapes, event vocabularies, protocol handling, display strings — lives once in the core and is exposed to Swift through uniffi-generated bindings. The Swift shell keeps only the platform-bound parts: iOS OS APIs, the SwiftUI connection page, audio device wiring, and the Meta DAT bridge.

Concretely, GlassesCore compiles the uniffi-generated Swift bindings together with the compiled Rust, shipped as a binary xcframework (extentos_coreFFI). That's why a fix in the core lands on iOS and Android at once, and why the generated bindings are never hand-edited — they're regenerated from the Rust source.

The transport bridge

Extentos.create(config:) resolves one of four transports behind a single abstraction, all sharing the core's selection logic:

• System audio — the vendorless baseline: microphone and speaker over the phone's own Bluetooth routing (AVAudioSession .playAndRecord / .voiceChat), with no vendor SDK in the path. Serves the complete agent runtime on any hands-free audio route — smart glasses, ordinary earbuds, or the phone itself. No camera, no display.
• Real Meta — production glasses via the Meta DAT iOS toolkit (meta-wearables-dat-ios, floor 0.8.0). This is the iOS-specific bridge: it adapts DAT's accessory channel and the custom-URL-scheme auth callback (forwarded through handleUrl(_:)) to the core's transport interface.
• Browser simulator — a WebSocket to the Extentos backend; the hardware surrogate runs in a browser tab. This is the dev loop.
• Local simulator — Extentos's own in-memory deterministic transport (no DAT SDK, no network), for a fast inner loop.

transport: .auto picks among them: the injected build-time / env-var session URL (debug builds) → a bonded Meta device → a pending socket the MCP localhost bridge binds (debug builds without a URL or bonded device) → .systemAudio when the app declared capabilities and none of them are camera or display (a voice app) → otherwise real Meta glasses. A release build always targets real hardware or the audio baseline — it never silently falls back to a simulator. An app that declares NO capabilities keeps the real-Meta default rather than being downgraded out of camera support. The simulator paths run the same SDK code as hardware with only the transport and I/O swapped — a simulator failure is a real failure. See transport vs app simulation.

Where the shell stops

The boundary is deliberate: anything cross-platform belongs in the Rust core, not duplicated in Swift, so the two platforms can't drift. The iOS shell exists for the things only iOS can do — SFSpeechRecognizer for on-device STT, AVFoundation audio routing, the External Accessory session, SwiftUI views, and the .onOpenURL auth handoff. There are no customer-facing extension points beyond the public sub-client API; customize behavior through ExtentosConfig and your own handler code, not by reaching into the runtime.

For the cross-platform architecture in full, see architecture.

Related