From simulator to real glasses

You built your app against the browser simulator and the agent-driven test loop closes: you inject an utterance, the right tool fires, the event log shows the expected sequence. That gives you high confidence the protocol, tool wiring, dispatch, and event reporting are correct — because those bytes are identical to what real glasses send and receive. It does not tell you whether the app will sound clear on a noisy street, recover from a dropped Bluetooth link, or survive a 30-minute session on the glasses' battery. Real hardware is the only thing that proves those.

This page is the transition between the two. The simulator needs zero credentials and no Meta involvement — that's the point of it. Moving to real glasses is where your app first acquires a production identity and flips off the simulated transport. There are four moves, all of them one-time setup, and an emulator can't do any of it (no real Bluetooth radio, no Meta companion app), so you need a physical Android device.

The four moves

The rest of this page walks each one, then covers what the simulator already proved (so you know what not to re-test) and what only hardware can confirm.

What the simulator already proved — and what it didn't

Don't re-test on hardware what the simulator already guarantees, and don't assume the simulator covered what it can't. The split is sharp.

Byte-identical with real glasses (proven in sim — trust it):

Only real hardware can confirm (the simulator does not model these):

If your app passes the agent loop in sim, the logic is correct. The verification pass below is about the second table, not the first.

Move 1 — Get your Meta production identity

Real-hardware traffic is gated by Meta. The simulator's placeholder credentials only ever route to the simulator; a real session needs your own Meta registration.

1. Create an app in the Meta Wearables Developer Center — the App ID and Client Token are minted there (Manage projects).

2. Copy the App ID and Client Token from the DAT configuration.

3. Register your app's identity: its package name and its release SHA-256 signing signature. Meta binds the DAT entitlement to that exact pair — your app will not connect to real glasses until it's registered. (On iOS the pair is bundle ID + Team ID instead.)

4. Enroll yourself as a tester. Meta gates registration on release-channel membership: create a version, assign it to a channel, and invite the Meta account that owns the glasses you're about to test with — including your own. Until that invite is accepted, registration fails even with correct credentials and a correctly signed build. Requires glasses firmware v125+. See release channels.

Because the entitlement is bound to your package and signature, this registration is per-app and can't be shared — Extentos can deliver credentials into your build, but it can't substitute its identity into your signed app.

Move 2 — Store credentials in your dashboard

Open your project's Credentials section and paste in the App ID and Client Token. These are build-time vendor identity — they ship inside your binary and are extractable, so they're build identity, not a server secret. (That's a different role from your managed-gateway AI key, which stays server-side and never ships. The section separates the two.)

You don't hand-edit any files. The next time you run generateConnectionModule, it reads your stored credentials over an owner-authenticated request and bakes them in:

• Android — writes app/src/main/res/values/extentos_meta_credentials.xml with the meta_application_id and meta_client_token string resources, referenced by two <meta-data> elements (com.meta.wearable.mwdat.APPLICATION_ID / …CLIENT_TOKEN) the MWDAT SDK reads at init. (Values go through @string refs because a bare numeric App ID is typed as an int by the resource compiler.)
• iOS — fills the Info.plist MWDAT dict (MetaAppID, ClientToken, plus the iOS-only TeamID and URL scheme).

If you haven't stored them yet, the scaffold leaves REPLACE_ME placeholders you can fill by hand. Either way, the simulator and emulator dev need none of this — credentials matter only for real hardware and release builds.

Move 3 — Resolve the Meta DAT SDK (the GitHub token)

This is the one step that surprises people, so it's worth understanding why it exists. Camera and display come from com.extentos:glasses-meta, which depends on Meta's DAT SDK (com.meta.wearable:mwdat-*) — and Meta publishes that SDK only through GitHub Packages, not Maven Central, not Google's Maven repository. com.extentos:glasses itself has no vendor dependency and resolves from Maven Central like any normal artifact.

Concretely you need two things in your project:

1. The repository, in settings.gradle.kts (inside dependencyResolutionManagement.repositories). generateConnectionModule emits this block for you — it's the only repository you add by hand, because the Extentos library itself resolves from its normal Maven coordinates:

   maven {
     url = uri("https://maven.pkg.github.com/facebook/meta-wearables-dat-android")
     credentials {
       val localProps = java.util.Properties().apply {
         val f = File(rootDir, "local.properties")
         if (f.exists()) f.inputStream().use { load(it) }
       }
       username = localProps.getProperty("github_username") ?: System.getenv("GITHUB_USERNAME") ?: "token"
       password = System.getenv("GITHUB_TOKEN") ?: localProps.getProperty("github_token")
     }
   }

2. A GitHub personal access token with the read:packages scope, provided either as a GITHUB_TOKEN environment variable or as github_token in your local.properties (which is in the default Android Studio .gitignore — keep it that way). The username can be the literal token.

Validate the whole chain before your first build — token, scope, repo block, and live reachability — with the bundled diagnostic, run from your project root:

npx -p @extentos/mcp-server extentos-mcp setup

It does a ground-truth fetch against the actual mwdat artifact using the same auth scheme Gradle uses, so a green result genuinely predicts a successful build. Running it first saves the 1–3 minute build cycle that otherwise fails late at :app:checkDebugAarMetadata with a bare 401 Unauthorized that never mentions Extentos or Meta. If you skipped it and you're staring at exactly that 401, this is the cause.

Move 4 — Flip the transport, pair, and run

Flip off the simulator

During development the library auto-selects the browser simulator when a session URL is baked in (BuildConfig.EXTENTOS_SESSION_URL on Android, extentos.session.plist on iOS). The resolver checks that baked URL before it checks for bonded glasses — so an APK built with a simulator URL stays in simulator mode even on a phone with glasses paired. For your hardware build:

• Build a release variant that leaves EXTENTOS_SESSION_URL null (and doesn't set the env var), so the resolver falls through to real Meta DAT when glasses are bonded. A release build also omits the debug-only pairing path.
• Or, for a quick spike, pass TransportChoice.RealMeta explicitly in ExtentosConfig.

validateIntegration will flag a release build that still has a simulator URL baked, and a missing GitHub Packages repo block — run it before you build.

Pair and install

1. A physical Android 12+ device (the library's minSdk is 31). No SIM needed.
2. Install the Meta AI companion app from the Play Store, sign in, and pair your Ray-Ban Meta. Take a photo from the glasses to confirm the pairing works outside your app.
3. adb install your release-variant APK, grant the runtime permissions (microphone, camera, Bluetooth) on first launch, and let the DAT registration flow hand off to the Meta AI app for approval. Once it returns REGISTERED, the session opens.

Run the verification pass

Assert the transport first, once, before you judge anything else:

Log.i("Extentos", "transport=${glasses.transportChosen} source=${glasses.selectionSource} camera=${glasses.capabilities.camera}")

REAL_META is what you're after. SYSTEM_AUDIO means Auto fell through to the vendorless audio baseline — voice will work, capture won't, and nothing will throw. The usual causes are a missing com.extentos:glasses-meta dependency, glasses that aren't bonded, or BLUETOOTH_CONNECT not granted before create(...). BROWSER_SIM means the simulator flip in the previous step didn't take.

If the assistant fails to start with AssistantError.NoApiKey, that's the gateway, not the glasses: a sideloaded development build can't attest, so it needs either a simulator-session binding or the dev-tier project key. See gateway identity. A beta or production build attests on its own.

Then walk 5–10 real voice interactions covering every tool you registered, and watch getEventLog(filter: "voice"). You're confirming two things: that the assistant.* sequence matches what the simulator showed (session_started → user_spoke → tool_called → tool_result → assistant_spoke), and how the app behaves on the three things the simulator couldn't model:

• Audio character — do your tools still fire reliably with real HFP audio, including outdoors? If a tool that was solid in sim gets flaky on hardware, the fix is almost always a clearer tool description, not a code change.
• Hardware timing — does the first utterance feel right, or is it clipped while Bluetooth switches profiles? Playing an earcon or a short say("ready") on assistant.session_started covers the gap. Check music coexistence if your app needs it.
• Failure modes — does the app recover from walking out of Bluetooth range, from an incoming call pausing the session, and does it stay usable within the ~30-minute battery ceiling of continuous use?

Record what you find in your app's release notes — these are customer-facing UX expectations, not library bugs. When the pass is clean, getProductionChecklist covers the remaining ship gates (permissions audit, foreground service, store listing).

First hardware run — telling the four failures apart

Four different things block a first real-glasses run, and three of them can look identical from the app's side. Run adb logcat -s Extentos:* while you launch, and read in this order — each line is emitted by the SDK and is greppable.

1. Did you reach the vendor transport at all?

transport=REAL_META source=VENDOR_REGISTRY

Anything else and you never got to Meta. SYSTEM_AUDIO means the vendor arm didn't claim — missing com.extentos:glasses-meta, no bonded glasses, or BLUETOOTH_CONNECT not granted before create(...). Fix that first; nothing below can succeed until this line reads REAL_META.

2. Did the assistant open?

AssistantError.NoApiKey thrown from assistant.start(...) is the project key, not the glasses. A sideloaded build can't attest, so it needs the baked EXTENTOS_PROJECT_KEY or a simulator-session binding. This failure happens before any camera work and is easy to mistake for a hardware problem — it isn't. See install § the project key.

3. Did Meta registration complete?

registration: app not registered — starting handoff
registration: registered

If you see the first line and never the second, the handoff started but didn't land. The SDK also logs, after a stall:

registration: still waiting — are the glasses awake and connected in the Meta AI app?

Three causes, in the order worth checking:

4. Did the device session start?

device session: STARTED (start gated)

If instead you see device session: did NOT reach STARTED in 12s, or:

device session: createSession failed (dat=NO_ELIGIBLE_DEVICE)

…the glasses are registered but the session didn't open. A createSession failed line carries a hint when the SDK can identify the cause — for example — camera permission not granted; grant Camera access to capture photo/video. NO_ELIGIBLE_DEVICE with no hint usually means the glasses went to sleep between steps; wake them and retry.

Only after line 4 does capturePhoto() have a session to run on. A capture before that returns CaptureError.NotConnected, which is a symptom of one of the four above and never a problem in your capture code.

Troubleshooting

Frequently asked questions

Can I just keep using the simulator?

For building and iterating, yes — that's the primary surface, and the agent-driven loop closes there without humans or hardware. But before you put the app on a customer's head, at least one real-hardware pass is required: the simulator is byte-accurate for protocol and logic, and deliberately silent on audio character, Bluetooth timing, and failure modes.

Can I test on an Android emulator instead of a real phone?

No. Emulators have only a virtual Bluetooth stack, so the glasses can't bond, Bluetooth SCO audio can't start, and the DAT registration has no companion app to hand off to. Cloud device farms are real phones but they're in a datacenter — they can't reach the glasses on your desk. Real-hardware testing needs a physical device with the glasses paired to it.

Do I need to create a GitHub personal access token?

You need a token with the read:packages scope so Gradle can pull Meta's DAT SDK from GitHub Packages — see Move 3. It can be a brand-new token used only for this; it never goes into your app binary or your git history. If you've already built the library locally, you likely have one set already (extentos-mcp setup will tell you).

Why does Meta's SDK need a token at all?

Because Meta distributes the Device Access Toolkit exclusively through GitHub Packages, and GitHub Packages requires authentication even for public packages. There's no unauthenticated mirror today. The token authenticates to GitHub, not to Meta or Extentos.

My build worked all through development and suddenly 401s — what changed?

Nothing about your code. The most common trigger is building on a fresh machine or CI runner where the read:packages token isn't configured, or a settings.gradle.kts where the GitHub Packages repo block was dropped. Run extentos-mcp setup to pinpoint which. (Note this applies only to apps that depend on com.extentos:glasses-meta for camera or display — those artifacts are transitive through it, so a missing token bites simulator builds too. A voice-only app never pulls them.)

Is the transition different on iOS?

The shape is the same — get a Meta identity, store credentials, flip the transport, pair and verify — but the concrete wiring uses Swift Package Manager and the Info.plist MWDAT dict instead of Gradle and resource files. See the iOS quickstart. The assistant-runtime walkthrough on this page is written against the Android library.

Related

• Transport vs app simulation — what the browser simulator covers versus Meta's Mock Device Kit, and why the same glasses.* code runs against both the simulator and real hardware
• Quickstart with an AI agent — the MCP flow that scaffolds the project and bakes your credentials
• Vendors: Meta Ray-Ban — the capability matrix and what DAT supports on real hardware
• Glasses won't connect — the connection-failure decision tree

Related