---
title: iOS API reference
description: The Extentos iOS SDK API reference — entry point, ExtentosConfig fields, and the GlassesUI surface (ExtentosConnectionPage, theming, escape hatch). Hand-maintained until the generated DocC reference replaces it.
type: reference
platform: ios
related:
  - /docs/sdk/ios
  - /docs/sdk/ios/install
  - /docs/sdk/ios/initialization
  - /docs/getting-started/with-agent
---

The iOS SDK installs from
[`github.com/extentos/swift-glasses`](https://github.com/extentos/swift-glasses)
([install](/docs/sdk/ios/install)). This page is a **hand-maintained** reference of
the surface you touch first — the entry point, the configuration struct, and the
`GlassesUI` types; a generated symbol-level DocC reference will replace it as the
package line grows. For per-primitive call shapes (camera, audio, voice, …), your
agent's `getCapabilityGuide(feature)` serves Swift snippets generated from source.

## Entry point

```swift
import GlassesCore

// usedCapabilities picks the transport on iOS — declare your real footprint.
let glasses: any ExtentosGlasses = Extentos.create(config: ExtentosConfig(
    usedCapabilities: [.microphone, .speaker]   // + .camera / .display if used
))
// Camera/display apps forward the Meta auth callback from their App scene.
// A voice app has no DAT pairing, so it needs neither of these lines:
.onOpenURL { url in Task { _ = await glasses.handleUrl(url) } }
```

`ExtentosGlasses` exposes the typed sub-clients (`connection`, `camera`, `audio`,
`runtime`, `toggles`, `voice`, `telemetry`, `observability`, `assistant`,
`display`), plus `capabilities: DeviceCapabilitySet`, `usedCapabilities:
[DeclaredCapability]`, and a root `connect()` convenience — what the generated
bootstrap calls, equivalent to `connection.connect()`. See
[initialization](/docs/sdk/ios/initialization).

Return shapes vary by kind of call: lifecycle and capture ops return an
`ExtentosResult` to pattern-match ([threading](/docs/sdk/ios/threading));
continuous primitives are `AsyncStream` / `AsyncThrowingStream`; display
`show` / `clear` return `Void`; assistant methods `throw`; and
`registerSound` / `soundNames` are synchronous.

`camera.videoFrames(config:)` is an `AsyncThrowingStream`, and the two errors it
can throw both arrive at the start of iteration:

| Error | Meaning |
|---|---|
| `CameraStreamPaused` | The wearer paused the camera with the temple button. Prompt a tap and retry; a pause *mid*-stream is not an error — frames halt and resume. |
| `CameraUnavailable` | This transport cannot serve a live frame STREAM, so no frame will ever arrive. Carries `.error`, the same typed `CaptureError` the discrete paths return. A transport can throw this and still have a working `capturePhoto` — Brilliant is exactly that: stills work, while neither device has a video primitive or codec. |

```swift
do {
    for try await frame in glasses.camera.videoFrames() { render(frame) }
} catch let e as CameraUnavailable {
    log("no camera on this transport: \(e.message)")
} catch is CameraStreamPaused {
    showHint("Tap the right temple of your glasses to resume the camera")
}
```

To branch *before* starting a stream, read `glasses.capabilities.camera`. Note the
Android counterpart adds one cause iOS cannot have — a build missing
`com.extentos:glasses-meta` — because on iOS every vendor transport ships inside
`GlassesCore`, so there is no vendor dependency to omit.

## ExtentosConfig

All fields have defaults — `ExtentosConfig()` is a valid development configuration.

| Field | Type (default) | What it does |
|---|---|---|
| `appId` | `String?` (`nil`) | App identity for telemetry + the dashboard. `nil` → falls back to the bundle identifier |
| `accountId` | `String?` (`nil`) | Optional Extentos account identity |
| `transport` | `TransportChoice` (`.auto`) | Transport selection; `.auto` resolves simulator vs real glasses vs the vendorless audio baseline ([how](/docs/concepts/transport-vs-app)) |
| `progressiveResponse` | `ProgressiveResponseConfig` (`.default`) | Advanced: progressive-response delivery tuning |
| `logLevel` | `LogLevel` (`.warn`) | SDK log verbosity |
| `externalTaskGroup` | `(any TaskGroupHandle)?` (`nil`) | Advanced: run SDK tasks in a host-owned task group |
| `interceptors` | `[any ExtentosInterceptor]` (`[]`) | Advanced: request/event interception hooks |
| `debug` | `Bool` (`false`) | Development mode — feeds `.auto` transport resolution (production builds leave this `false` so bonded glasses resolve to the real transport) |
| `telemetryConsent` | `Bool` (`true`) | Master telemetry switch — `false` sends no events at all |
| `dataSharingConsent` | `Bool` (`true`) | Developer-level Do-Not-Sell: `false` keeps your dashboard analytics but excludes your events from cross-account vendor aggregates ([security](/docs/resources/security)) |
| `environment` | `ExtentosEnvironment` (`.development`) | `.development` / `.beta` / `.production` — routes analytics to the right bucket. Production apps must set `.production` explicitly |
| `telemetryEndpoint` | `URL?` (`nil`) | Override the telemetry ingest endpoint; `nil` → production endpoint |
| `premiumVoice` | `PremiumVoiceConfig` (`.none`) | Premium voice configuration for the assistant |
| `usedCapabilities` | `[DeclaredCapability]` (`[]`) | The app's declared capability footprint. **On iOS this decides the `.auto` transport**: non-empty and free of `.camera`/`.display` → the vendorless system-audio baseline; empty or vendor-declaring → real glasses. Also drives the connection page's capability tiles. Cases: `.camera`, `.microphone`, `.speaker`, `.display`, `.location`, `.notifications`, `.custom(String)`. A voice app should pass `[.microphone, .speaker]`. |
| `hasBondedMetaDevice` | `(@Sendable () -> Bool)?` (`nil`) | Advanced: override bonded-device detection during `.auto` resolution. Note the semantics differ from Android: on iOS `nil` means "no bonded device", because there is no platform API to enumerate bonds. On Android `null` means "use the SDK's built-in bonded-device detector". |

## Display

`glasses.display` renders declarative trees on display-capable glasses
(Ray-Ban Display). `display.isAvailable` reports whether the connected device
has a display — a display call on a no-display device is a silent no-op, never
a crash. Display ships in both SDKs and is fully simulator-testable; on-glasses
rendering via Meta DAT is live on Android, with iOS on-glasses delivery in
progress.

```swift
if glasses.display.isAvailable {
    await glasses.display.show(onBack: { /* back gesture for this show */ }) { scope in
        scope.flexBox(direction: .column, gap: 8) { col in
            col.text("Hello from the app")
            col.button("Done") { /* fires on a real tap or a sim-injected select */ }
        }
    }
}
await glasses.display.clear()
```

Each `show` replaces the whole display (latest wins); tappable nodes get
auto-assigned ids, stable within a show. The node vocabulary is `text`,
`image(url:)` (the glasses fetch http(s) themselves), `button`, `icon`, and
`flexBox(direction:)` containers; the root scope adds `video(url:)` — a hosted
video as the sole, full-surface node. Android-only today: the local-media roots
(`image(photo:)`, `video(clip:)`) and the hosted-video helpers (`prepareVideo`,
`forgetHostedVideo`).

## Named sounds

`glasses.audio` plays named sounds through the glasses speaker alongside
`speak` and `earcon`:

| Call | Shape |
|---|---|
| `playSound(_:volume:)` | `async` → `ExtentosResult<Void, AudioError>`; unknown name → `.platformError(code: "sound_not_found")` |
| `registerSound(_:pcm16:sampleRate:)` | Synchronous — registers (or replaces) mono PCM16-LE bytes at `sampleRate` Hz, for the process lifetime |
| `soundNames()` | Synchronous — the registered names, sorted |

Dashboard-uploaded sounds register automatically at assistant start; code
registrations win on name collisions.

## VideoConfig and Photo.loadImage

`captureVideo(config:)` takes a `VideoConfig` with five fields: `resolution`,
`maxDurationSeconds` (`nil` = no time cap — the capture ends on cancellation),
`format`, `includeAudio` (default `true`), and `frameRate` (default 24 — video
and live view share one warm camera stream, so it takes effect only when the
recording is the session's first camera use).

`Photo.loadImage()` is the one iOS photo helper: `async`, decodes `file://` and
`data:` URIs uniformly across transports, and returns a `PlatformImage?`
(`UIImage` on iOS) — `nil` on unrecognized scheme, missing file, or decode
failure.

## Assistant

`glasses.assistant` is the voice-AI runtime, routed through the Extentos
managed gateway (no API key in code). Two forms: the sugar `start(provider:_:)`
(builds the config inline, returns a started session) and the raw
`createSession(config:)` + `session.start()`. Sessions are singleton-active per
instance; assistant methods `throw` (`AssistantError`).

```swift
let session = try await glasses.assistant.start(provider: .managed()) {
    $0.instructions = "You are a helpful assistant on smart glasses."
    $0.tool("take_picture", description: "Take a photo with the glasses camera.") {
        if case .success = await glasses.camera.capturePhoto() { return .ok("photo saved") }
        return .err("camera failed")
    }
}
```

An `AssistantSession` has a wake/sleep cycle inside it: `start()` lands it
Dormant, `wake()` opens the realtime connection, `sleep()` closes it but keeps
history/tools ready, `stop()` tears it down for good. In-session ops: `say(_:)`,
`greet(_:)`, `includeImage(uri:prompt:)`, `setVoice(_:)`, `setModel(_:)`,
`updateInstructions(_:)`, `cancelSpeak()` (barge-in), and the history ops
(`conversationHistory(limit:)`, `clearHistory()`, `appendHistory(_:)`,
`replaceHistory(_:)`). `session.state` observes the 8-state lifecycle: `idle`,
`dormant`, `activating`, `active`, `reconnecting`, `sleeping`, `stopping`,
`stopped`.

## Voice and audio helpers

- `glasses.voice.onPhrase(phrase:label:stops:handler:)` — the library matches
  the phrase (case-insensitive substring on final transcripts) and runs your
  handler. While it runs, hearing any `stops` phrase cancels the handler's
  `Task` — plain structured concurrency, so your `defer` cleanup runs. Returns
  a `VoiceRegistration` whose `cancel()` removes the hint.
- `glasses.voice.registerHint(phrase:label:stops:)` — announce the phrase to
  the simulator + connection-page UI while doing your own matching.
- `glasses.audio.cancelSpeak()` — barge-in: stops in-flight `speak` TTS
  immediately; idempotent.
- `glasses.audio.startPushToTalk()` — returns a `PushToTalkSession`;
  `await session.stopAndFlush()` ends the capture and returns the concatenated
  final transcripts.

## GlassesUI

| Type | Shape | What it is |
|---|---|---|
| `ExtentosConnectionPage` | `init(glasses:config:)` — `config` defaults to `ConnectionPageConfig()` | The drop-in SwiftUI connection page: status, capability tiles, toggles |
| `ConnectionPageConfig` | `init(sections: SectionVisibility = .init())` | Structural visibility control for the page |
| `SectionVisibility` | `init(capabilities: Bool = true, voiceCommands: Bool = true, toggles: Bool = true)` | Which page sections render |
| `ExtentosTheme` | `init(appearance: Appearance = .default) { content }` | Wrap Extentos views to restyle them with brand tokens |
| `Appearance` | `init(colors:typography:shapes:)` | The token set `ExtentosTheme` applies |
| `ExtentosPairingScreen` | `init(code:expiresAtMs:appearance:)` | The simulator pairing-code screen — shown automatically by the connection page when the localhost auto-bind probe doesn't resolve; the developer enters the code in the browser simulator to claim the socket |
| `rememberExtentosState(_:)` | `(any ExtentosGlasses) -> ExtentosUiState` | Escape hatch: snapshot the UI state and build fully custom UI — you forfeit library-driven UI updates |

The escape hatch — `rememberExtentosState(_:)` and its `ObservableObject` form
`ExtentosStateModel` — is gated behind an SPI import:
`@_spi(ExtentosEscapeHatch) import GlassesUI`. The opt-in is deliberate: once
you bypass `ExtentosConnectionPage`, you're responsible for rendering any
surfaces the library adds later.

```swift
import GlassesUI

ExtentosConnectionPage(glasses: glasses)

// Restyled + trimmed:
ExtentosTheme(appearance: myAppearance) {
    ExtentosConnectionPage(
        glasses: glasses,
        config: ConnectionPageConfig(sections: SectionVisibility(voiceCommands: false))
    )
}
```

## Local tier

Separate SPM products — see [on-device models](/docs/concepts/local-models). Linking `GlassesLocal` raises the package's deployment floor to **iOS 17**.

```swift
// GlassesLocal
public enum ExtentosLocalTier {
    public static func register()                       // BEFORE Extentos.create(...)
    public static func models() -> [LocalModelInfo]
    public static func autoChoice() -> AutoChoice
    public static func deviceFit(for dashboardId: String) -> DeviceFit
    public static func download(
        modelId: String,
        onProgress: @escaping @Sendable (Double) -> Void = { _ in }
    ) async throws
    @discardableResult
    public static func delete(modelId: String) throws -> Bool

    public struct LocalModelInfo {
        public let id: String
        public let displayName: String
        public let requiredMb: Int
        public let isInstalled: Bool
        public let autoEligible: Bool
    }
    public struct AutoChoice {
        public let modelId: String?
        public let isLocal: Bool
        public let cloudReason: String?
        public let downloadTarget: String?
    }
    public struct DeviceFit {
        public let modelId: String
        public let requiredMb: Int
        public let availableMb: Int
        public var fits: Bool
    }
    public enum LocalTierError: Error { case unknownModel(String) }
}

// GlassesLocalVoice
public enum ExtentosLocalVoice {
    public static func register()
    public static var modelDirectory: URL? { get }
}

public enum KokoroVoiceModel {
    public static var totalBytes: Int { get }
    public static func isInstalled() -> Bool
    public static func delete() throws
    public static func download(
        progress: @escaping @Sendable (Double) -> Void
    ) async throws
}
```

Two shape differences from Android: **no `Context` parameter** on any call, and `download`'s progress is a single `Double` fraction rather than Android's `(bytesDownloaded, totalBytes)` pair.

`register()` must run **before** `Extentos.create(...)`. Because Swift runs stored-property initializers ahead of `init()`, declare the handle without an initializer and assign it inside `init()` after the `register()` calls — see [on-device models](/docs/concepts/local-models#setup).

## Differences from Android

Shipping both platforms? These are the ones that bite. Names and shapes first:

| | Swift | Kotlin |
|---|---|---|
| OpenAI provider | `.managed(model:voice:turnDetection:reasoningEffort:)` | `AssistantProvider.Managed(...)` |
| Assistant config block | `(AssistantConfigBuilder) -> Void` — `$0.instructions = …` | receiver lambda — bare `instructions = …` |
| Typed tool | `schema:` is **required** | schema inferred from `@Serializable` |
| Tool result | `.ok` / `.err` | `ToolResult.Ok` / `.Err` |
| Capability dial | `DeviceCapabilitySet` | `GlassesCapabilities` |
| `usedCapabilities` | `[DeclaredCapability]` (array) | `Set<CapabilityKind>` |
| `sleepAfterSilence` | `TimeInterval` seconds | `kotlin.time.Duration` |
| Toggles | `state` + `update(_:)` | `state` + `put(...)` / `get(...)` |
| Layout builder | `flexBox(direction:…)` | `column(…)` / `row(…)` |
| Local tier | no `Context` parameter | every call takes `context: Context` |
| Streams | `AsyncStream` / `AsyncThrowingStream` | `Flow` |
| `includeImage` | `prompt` has no default | `prompt: String? = null` |

<Callout type="warn">
  **Two defaults differ silently — same call, different behaviour.**

  - `earcon(_:volume:)` defaults to **`0.8`** on Swift and **`1.0`** on Kotlin.
  - `SpeakConfig.pitch` defaults to **`0.0`** on Swift and **`1.0`** on Kotlin
    (and Swift's `SpeakConfig` fields are `Float`, Kotlin's are `Double`).

  Neither produces an error — your iOS build just sounds different. Pass the
  value explicitly on both platforms if it matters to you.
</Callout>

**Swift has, Android doesn't:** `progressiveResponse`, `premiumVoice`, `interceptors`, `externalTaskGroup` on `ExtentosConfig`; a root `glasses.connect()`; `Extentos.default()`; `Extentos.requestSpeechRecognitionAuthorization()`. (`KokoroVoiceModel` — `isInstalled` / `download` / `delete` for the on-device voice — exists on **both** platforms; Swift takes no `Context`.)

<Callout type="warn">
  **iOS has no `transportChosen`, so the one-line startup assertion the Android
  guides recommend has no direct equivalent.** Use what iOS does expose:
  `glasses.capabilities` (a voice app on the audio baseline reads `camera:
  false`) together with `glasses.connection.state`. If you need certainty about
  which transport resolved, force it — set `transport: .systemAudio` or
  `.realMeta` explicitly rather than `.auto` — and let `connect()`'s typed error
  tell you when it can't be served.
</Callout>

**Android has, Swift doesn't:** `VoiceScope` / `onPhrase(firesWhen:)` — so an iOS wake phrase can re-fire mid-conversation and you gate it in your handler; `camera.stopVideo()`; `camera.preferredStreamConfig`; `display.prepareVideo` / `forgetHostedVideo` / `forgetHostedImage`; the `video(clip:)` and `image(photo:)` display overloads; `applicationContext` / `activityProvider`; and `transportChosen` / `selectionSource` / `device` / `connectionConfig` on the root handle.

Neither platform has an assistant-runtime language setting — see [what language does the assistant speak](/docs/concepts/assistant#what-language-does-the-assistant-speak-and-hear).

## Related pages

- [Initialization](/docs/sdk/ios/initialization) — entry points, sub-clients, auth handoff, speech authorization
- [Threading](/docs/sdk/ios/threading) — the concurrency contract
- [Info.plist setup](/docs/sdk/ios/info-plist) and [lifecycle](/docs/sdk/ios/lifecycle)
- [Android API](/docs/reference/android-api) — the same surface in Kotlin