iOS quickstart

The Extentos iOS SDK installs from github.com/extentos/swift-glasses via Swift Package Manager — .package(url: "https://github.com/extentos/swift-glasses", from: "2.8.0") (current version on the install page). This page is the by-hand path; the agent-driven flow automates every step below.

1. Add the package

In Xcode: File → Add Package Dependencies… → https://github.com/extentos/swift-glasses → dependency rule Up to Next Major Version, starting at 2.0.0. Link GlassesCore; add GlassesUI only if you render the packaged connection page (a voice app doesn't). Other manifest forms: Install. Requirements: the Swift 6 toolchain, Meta DAT (meta-wearables-dat-ios) 0.8.0+ (resolved transitively), and deployment target iOS 17.0+. The package declares that floor for every product, because SwiftPM applies a package's platform floor package-wide and GlassesLocal pulls MLX, which requires 17.

2. Add the Info.plist keys

Which keys you need depends on your path — see choose your path.

Voice apps need the privacy strings your capabilities imply — NSMicrophoneUsageDescription and NSSpeechRecognitionUsageDescription — plus UIBackgroundModes: ["audio"] if the assistant must keep listening while your app is backgrounded, plus the two Extentos keys EXTENTOS_APP_ID and EXTENTOSProjectKey. No Meta keys and no Meta Developer account.

The project key is not a Meta credential. It is how your app authenticates to the managed gateway in every environment, App Store builds included, and it is what meters usage to your account.

generateConnectionModule mints it and writes it for you, and there is no other source — an app that skips that call has no key at all, however complete the rest of its integration looks. If you are wiring by hand, still run the tool once and take the value from the Info.plist entries it emits (you can discard the rest of what it generates), or copy the key from your dashboard project. Without it an assistant app fails at runtime with AssistantError.NoApiKey after a perfectly clean build.

Camera or display apps additionally need the Meta DAT set: the MWDAT dict (including an explicit DAMEnabled), CFBundleURLTypes (a custom scheme, not a universal link), LSApplicationQueriesSchemes: ["fb-viewapp"], UISupportedExternalAccessoryProtocols: ["com.meta.ar.wearable"], and the Bluetooth/external-accessory UIBackgroundModes.

Full set with values: Info.plist setup.

Camera apps also need two entitlements, and they are not Info.plist keys. Camera frames do not travel over Bluetooth: they ride a higher-bandwidth link the Meta SDK obtains by asking iOS to join the glasses' own Wi-Fi network. That request is gated by Access Wi-Fi Information (com.apple.developer.networking.wifi-info) and Hotspot Configuration (com.apple.developer.networking.HotspotConfiguration), which must be on your app target and enabled on your App ID. An entitlement can only be granted to an app, never to a framework, so no SDK can supply these for you.

Camera streaming also needs NSLocalNetworkUsageDescription and a non-empty NSBonjourServices, which gate the socket layer once the network join has happened. Both are necessary and neither is sufficient on its own.

Get them wrong and there is no error to read: the session starts, the glasses report healthy, capture returns no frame, and nothing is logged. generateConnectionModule emits the entitlements file for you whenever your footprint includes camera. Confirm what actually shipped with codesign -d --entitlements - YourApp.app.

Expect a one-time-per-launch iOS prompt asking to join the glasses' network — the wearer must allow it, and the first capture afterwards takes roughly 10-15 seconds while the link comes up. Every capture after that is sub-second. iOS discards the join when your app exits, so the prompt returns on the next cold start; arm the camera during your connect flow if you would rather the wearer not pay that wait on a shutter press.

3. Initialize

Create the SDK entry point once, at your App entry point, and forward the Meta auth callback:

import GlassesCore

@main
struct MyApp: App {
    // usedCapabilities is load-bearing on iOS — it picks the transport.
    // A voice app declares [.microphone, .speaker]; add .camera / .display
    // if you use them. Leaving it empty resolves to real glasses.
    let glasses: any ExtentosGlasses = Extentos.create(config: ExtentosConfig(
        usedCapabilities: [.microphone, .speaker]
    ))
    // `debug` means "prefer the browser simulator over real I/O", not "is a
    // debug build". `debug: true` opens a pending simulator socket: a coding
    // agent's MCP bridge binds it automatically, and with no bridge the app
    // shows a 5-character pairing code you type into the simulator. Leave it
    // at its false default when you want the phone's own mic and speaker.

    var body: some Scene {
        WindowGroup {
            ContentView(glasses: glasses)
                .onOpenURL { url in Task { _ = await glasses.handleUrl(url) } }
        }
    }
}

ExtentosConfig's full field list (transport choice, environment, consent flags, …) is in the iOS API reference; details: Initialization.

4. Make your first capability call

Every call is async and returns an ExtentosResult — pattern-match it (if case .success(let photo) = await glasses.camera.capturePhoto()); don't wrap in do/catch. See Threading.

5. Drop in the connection page — camera and display apps only

Skip this step if you're building a voice app: it renders no connection page, so it doesn't need GlassesUI at all. See choose your path.

import GlassesUI

ExtentosConnectionPage(glasses: glasses)   // status, capabilities, toggles — library-owned UI

6. Verify it in the simulator

Run against the browser simulator — the same library code as production with only the transport swapped, so a green run here is meaningful. With debug: true, the SDK's Auto transport resolution picks the simulator transport during development.

There are two ways to get a session, and they end at the same place.

With a coding agent. Ask it to mint one; it calls createSimulatorSession and your running app attaches through the local bridge with nothing further to do.

Without one. Create a simulator from your projects, then pair your app to it:

1. Sign in and hit New simulator, then pick iOS. The simulator opens and waits.
2. Run your app with debug: true. With no agent to discover, it shows a 5-character pairing code on the packaged connection page.
3. Type that code into the simulator's Enter the pairing code field. The app attaches immediately — no rebuild, no session URL to paste.

If you don't render the packaged connection page — a voice app usually doesn't — the same code arrives on glasses.connection.simulatorHint as SimulatorHint.AwaitingPair.

Related