Quickstart with an AI agent

Extentos is the smart-glasses infrastructure layer for native mobile apps. An AI coding agent installs the Extentos MCP server, the server exposes a tight set of deterministic tools (capability discovery, code-example reference, project scaffold, validation, browser simulator), and the agent uses them to wire a typed Kotlin/Swift SDK into an existing iOS or Android app — your handler classes then subscribe to primitives like glasses.audio.transcriptions(), glasses.camera.capturePhoto(), and glasses.audio.speak(). Targets Meta Ray-Ban (supported in production today), with other vendors on the roadmap. A browser-based simulator covers the bulk of the dev loop without physical glasses.

This page is the agent-driven path. If you're integrating by hand, see the iOS or Android quickstart instead.

Install

There's no single host-agnostic install command — every MCP host (Claude Code, Cursor, Windsurf, Cline) stores its server config differently. Three paths, in order of recommended:

1. Agent prompt (recommended, host-agnostic)

Open your MCP-capable agent and paste:

Install @extentos/mcp-server in my MCP config.

The agent runs the right command for its host (claude mcp add ... for Claude Code, or edits mcp.json for Cursor / Windsurf / Cline). Restart the agent when it reports done. The Extentos tools appear in the agent's tool list.

2. Claude Code one-liner

If you're on Claude Code and prefer a shell command:

claude mcp add extentos -- npx -y @extentos/mcp-server@latest

Restart Claude Code. Run /mcp — you should see extentos listed with the full tool surface.

3. Manual JSON

For any MCP host, drop this into the host's MCP config file (~/.cursor/mcp.json for Cursor, ~/.codeium/windsurf/mcp_config.json for Windsurf, or equivalent):

{
  "mcpServers": {
    "extentos": {
      "command": "npx",
      "args": ["-y", "@extentos/mcp-server@latest"]
    }
  }
}

Restart the host. The Extentos tools should appear in the agent's tool list.

Prerequisites: Node.js 20 or newer on PATH. That's the only requirement — npx fetches and caches the package on first run.

Pinning a version (optional): Replace @latest with an exact version (the current one is on npm) if you need a reproducible install across sessions. Extentos is pre-1.0; APIs may shift between minor versions until the hardware test loop closes.

First prompt to your agent

After install, ask the agent to do the actual work. Two prompt patterns that work well:

Add Extentos smart-glasses voice triggers to my Android app.
The user should be able to say "summarize this" while wearing
Meta Ray-Ban glasses, and the app should run my existing
summarize() function on whatever the camera sees.

Use the Extentos MCP server to add photo capture from
Meta Ray-Ban to my iOS app. When the user double-taps the
glasses, capture a frame, send it to my analyzeImage()
function, and speak the result back.

Be specific about: which platform, which capability (voice / photo / video / sensor / audio), and which existing function in your app should consume the data. The agent composes against the SDK primitives — wake-phrase matching, business logic, error handling — all in plain Kotlin / Swift in a handler class you own.

What the agent does

The MCP server has no planning tool — agents are better planners than regex. The agent composes capability primitives itself, then calls a deterministic sequence. You can watch it happen in the tool log:

The numbering is a recommended order, not a required one — and minting early is fine. createSimulatorSession sits last because that is when a session is most useful, but agents routinely mint first to see the simulator before committing to a project layout, and nothing stops them. The only difference: a sim minted before generateConnectionModule has no extentos.manifest.json to key on, so it is held against your account and platform rather than bound to a project, and every unscaffolded directory you mint from resumes that same sim. Scaffolding binds it. Either order works and neither loses your session.

One-time credential setup (Android) — only if you use camera or display. Those live in com.extentos:glasses-meta, which pulls Meta's DAT SDK from GitHub Packages and needs a free read:packages GitHub token. A voice-only app needs none of this — com.extentos:glasses has no vendor dependency and resolves from Maven Central. See choose your path.

For iteration: edit handler code → rebuild + reinstall → the app auto-attaches to the same simulator session. No remint required.

For debugging: getEventLog (filter by errors first to surface every failure across modalities, then narrow to a capability chip — voice / camera / display / ai / lifecycle / custom), getSimulatorStatus. For ship-readiness: getProductionChecklist. For credentials (BYOK providers + Meta DAT registration): getCredentialGuide.

The full tool reference is at /docs/mcp-server/tools.

Verifying it worked

When the agent calls createSimulatorSession, the MCP server auto-opens a browser tab to your session at extentos.com/s (specifically extentos.com/s/<sessionId>). You should see:

• A glasses surrogate with camera + mic permission prompts (browser-side; no physical hardware)
• A live event log showing your app initializing, the connection opening, and your trigger firing when you exercise it
• A WebSocket-synced view of your app's responses, exactly as they'd appear on real Meta Ray-Ban glasses

Auto-bind is shipped on both Android and iOS: once the Extentos library is in your app, the MCP and library coordinate via a 127.0.0.1:31337/whoami localhost bridge. Subsequent createSimulatorSession calls auto-attach the running app to the new session — no rebuild, no URL paste.

Pricing

Extentos's tools, simulator, code generation, validation, and SDK are free — there's no per-seat or subscription fee to build and ship. The one surface that meters usage is the managed AI gateway behind the Phase-4 voice assistant. Specifically:

Why account-required for the simulator? The browser simulator runs on Extentos's backend (WebSocket hub, voice proxy, event log persistence) — that's the one piece that costs us per-session compute. Asking for a free email-only account is the lightest possible gate. Most other Extentos surfaces (discovery, guidance, validation, on-device sim, real-hardware DAT testing) don't touch the backend and stay free without an account. The exceptions are createSimulatorSession, generateConnectionModule's scaffold step, and the account-scoped project tools — all free, but they need the sign-in. See pricing.

Account linking happens through the device-code flow: the first createSimulatorSession call returns auth_required with a verification URL, your agent surfaces it, you sign in (Google one-click or email + password + ToS acceptance), and the original tool call retries automatically. No manual token paste, no payment.

Real hardware testing (running your app on actual Meta Ray-Ban glasses) requires your own Meta Wearables Developer Center registration — a one-time, free, ~15–30 minute setup using your personal Meta credentials. No Extentos account required for this path.

Full pricing details: /docs/resources/pricing.

Multi-platform projects (Android + iOS = one project)

Your Android and iOS versions share one project on the dashboard when you scaffold both with the same Extentos app id. That id is config.appId — conventionally your reverse-DNS package name, but decoupled from your store bundle id, so the two platforms share a project even when their store identifiers differ.

Give both platforms the same Extentos app id (the appPackage you scaffold with — it's config.appId, decoupled from your store bundle id) and minting a simulator from each automatically joins the same project: one card, two platform chips, no manual setup. This works even when your Android and iOS store bundle ids differ.

The full story — how project identity is derived, what's preserved across a Merge, what the Settings page surfaces — lives at /docs/concepts/projects.

What's actually supported (be honest with your agent)

The Meta DAT public toolkit is the substrate Extentos sits on. It has real boundaries — your agent should know them so it generates apps that actually run:

• Strong public capabilities: photo capture, video capture, camera streams, sensor data
• Voice triggers: custom phrases are recognized on the phone (on-device Vosk on Android hardware, Apple's SFSpeechRecognizer on iOS; the platform recognizer on the dev path), fed from the glasses mic over Bluetooth (HFP/SCO). Meta AI's "Hey Meta" wake word is not accessible to third-party apps
• Audio I/O: mic capture and TTS playback ride mobile-platform Bluetooth profiles. Extentos wires the audio session; your platform's native STT/TTS does the work (zero Extentos runtime cost)
• Not available: raw Neural Band gestures and custom gesture recognition. Captouch is system-defined (taps pause/resume sessions); on Meta Ray-Ban Display, wearer interactions reach your glasses.display.* UI as abstracted tap events
• Distribution: public publishing is partner-gated while Meta's toolkit is in developer preview — users enroll as testers via Meta release channels today. Android export works normally; Meta doesn't support iOS App Store publishing yet

See /docs/concepts/capabilities for the full matrix, and /docs/concepts/transport-vs-app for the framing of what Meta provides vs what Extentos adds.

Troubleshooting first-run issues

/mcp doesn't list extentos in Claude Code. Restart Claude Code completely (Cmd+Q / quit from system tray, not just close window). The MCP host loads servers at startup.

npx errors with "command not found". Install Node.js 20 or newer from nodejs.org. Verify with node -v.

The agent's first tool call is slow or times out. npx is fetching the package on first run — wait 10–30 seconds and retry. Subsequent calls hit the cache and are instant.

createSimulatorSession returns auth_required immediately. That's the expected flow on first run — the browser simulator requires a free account. The response includes a verification URL; open it, sign in (Google one-click or email), and the agent retries automatically.

Auto-bind not connecting on Android. Check that nothing else is bound to localhost port 31337. The MCP server logs a warning if it can't bind the bridge (port 31337 in use); the app then falls back to URL-paste mode.

iOS app not picking up the session. Check that nothing else is bound to localhost port 31337; the library logs a warning if the bridge fails to start. Falls back to URL-bake (extentos.session.plist) if the probe times out.

More: /docs/troubleshooting.

Next steps

• MCP server reference — full tool catalog, configuration, auth model
• Concepts: architecture — how the MCP, the library, the simulator, and your app fit together
• Concepts: transport vs app simulation — what Meta provides, what Extentos adds, and why both matter
• Guides — task-shaped recipes (voice triggers, photo capture, sensor streams, background sessions)
• From simulator to real glasses — when you're ready for hardware: Meta identity, credentials, and the transport switch
• Vendors: Meta Ray-Ban — the primary production target and its capability matrix

Status

Pre-1.0 (current version on npm). Android is the longer-proven path on real hardware; both SDKs are published in lockstep (see iOS install). APIs may shift between minor versions until the hardware test loop closes — pin to an exact version if you need cross-session reproducibility. Published on npm (MCP server, MIT), Maven Central (Android library), and github.com/extentos/swift-glasses (iOS Swift package).

Related