MCP server
The Extentos MCP server (`@extentos/mcp-server`) is an npm package an AI agent (Claude Code, Cursor, Windsurf, Cline) installs once and then uses to add Meta Ray-Ban smart-glasses capabilities to a native iOS or Android app. It exposes a tight set of deterministic tools across 7 categories — discovery, generation, guidance, validation, simulation, production-readiness, and search — plus a CLI for account linking, telemetry consent, and update checks. This is the agent's operating manual.
The MCP server is how an AI agent — Claude Code, Cursor, Windsurf, Cline, or any Model Context Protocol-compatible host — operates Extentos. The agent invokes deterministic tools; the server primes the agent on what capabilities the glasses expose, returns canonical SDK code patterns in Kotlin and Swift, scaffolds the project, brokers simulator sessions, and queries debugging traces. The server itself has no planning tool — agents are better planners than regex bundles. Tools are typed primitives the agent composes in sequence.
This page is the section landing — what the server is, the tools at a glance, the canonical agent-driven flow, configuration knobs, the CLI, and how the auth model works. Sub-pages cover each in depth.
Install
claude mcp add extentos -- npx -y @extentos/mcp-server@latestFor non-Claude-Code hosts, see the agent prompt or manual JSON install paths. Full install reference at /docs/mcp-server/install.
The tools, by category
The server exposes a deterministic tool surface (verified in mcp-server/src/tools/definitions.ts), organized into 7 categories. The categories are the agent's mental map; an agent that understands which category a tool lives in can decide when to call it.
1. Discovery and SDK reference (3 tools)
The first calls in any new task. Cheap, all local, no side effects.
| Tool | What it does |
|---|---|
getPlatformInfo | Returns the static platform catalog — library version, the list of SDK capabilities the glasses expose, per-vendor tiers. Always the right first call. |
getCapabilityGuide | Per-feature minimal usage in Kotlin + Swift — call shape, config args, gotchas. Pairs with getPlatformInfo (which names the features) to tell the agent how to invoke each. |
getCodeExample | Full canonical compositions in both languages for the 6 voice-glasses patterns: voice_qa_assistant, barge_in_speak, photo_describe_voice, live_transcription_ui, voice_notes, connection_page_setup. Peel from these when writing handler code. |
2. Setup and Generation (1 tool)
| Tool | What it does |
|---|---|
generateConnectionModule | One-shot scaffold — bootstrap module, Gradle/SPM wiring, dependencies, permissions, manifest. Two-call flow: first call without placement returns a question asking where ExtentosConnectionPage should live; second call with the chosen placement returns the full file set. |
After scaffolding, the agent writes its own handler classes against the SDK primitives surfaced by getCapabilityGuide / getCodeExample. Handler code is the customer's authoring surface — there's no initSpec or DSL-population step.
3. Implementation Guidance (2 tools)
Side-quest tools the agent calls during composition.
| Tool | What it does |
|---|---|
getVoiceCommandGuidance | Analyze proposed wake / command phrases for UX issues (collisions, ambiguity, hard-to-recognize words, Meta wake-word conflicts) before wiring them into a glasses.audio.transcriptions() consumer. |
getPermissions | Derive exact platform permissions, Meta DAT requirements, and foreground-service needs from the capability list. Run when adding or removing a primitive from your handler. |
4. Validation (2 tools)
Correctness gates. Run after structural changes (new capability declared, dependency bumped, manifest edited).
| Tool | What it does |
|---|---|
inspectIntegration | Read-only project snapshot — manifest, generated-file hashes, dependency list, connection-page config. Run before manual edits to understand current state. |
validateIntegration | Whole-project correctness check — manifest, generated files, dependency declared, permissions cover declared capabilities, bootstrap calls ExtentosGlasses.create(...), toolchain versions, foreground-service hints for continuous-capture flows. The pre-test gate. |
5. Simulation (4 tools)
Provision and operate browser-based simulator sessions.
| Tool | What it does |
|---|---|
createSimulatorSession | Get-or-create a browser-mode session at extentos.com/s. Returns the saved sim for this project if one exists (status: "resumed"), or mints a new one (status: "active"). Auto-attaches the running app via the local bridge when reachable; otherwise emits a BuildConfig.EXTENTOS_SESSION_URL snippet (Android) or extentos.session.plist payload (iOS). Pass resetFresh: true to rotate the sessionId for a clean slate. |
completeAuthLink | After createSimulatorSession returns status: "auth_required" (anonymous install needs to link to mint sessions), polls the backend until the user finishes signup, then persists the bearer token to ~/.extentos/auth.json. |
getEventLog | Fetch structured event traces from a session. Six-chip taxonomy: all, errors, voice, camera, ai, lifecycle, custom — one chip per event, with errors absorbing severity≥warn regardless of modality. Plus since, limit for trace-level scope. The primary debugging tool. |
getSimulatorStatus | Read a live session's current state — phase, hardware-ready, attached roles, active capability streams, current toggle values. |
6. Production (2 tools)
Pre-ship checks.
| Tool | What it does |
|---|---|
getProductionChecklist | Personalized production-readiness checklist based on declared capabilities + handler names — credential wiring, permission audit, foreground-service requirements (when continuous capture is used), simulator-URL removal from release builds, store-listing readiness. |
getCredentialGuide | Step-by-step credential setup for production AI providers — anthropic, openai, google_cloud_vision, google_translate, google_gemini, deepl, azure_cognitive, aws_bedrock, huggingface, or custom — plus Meta DAT registration. |
7. Search (1 tool)
| Tool | What it does |
|---|---|
searchDocs | Search Extentos documentation by topic or keyword. Aligned post-pivot topics: getting_started, custom_handlers (the canonical SDK-composition doc), voice_integration, connection_ui_placement, host_app_scaffold, auto_bind_session_lifecycle, local_bridge_discovery, device_code_flow, connection_state_model, permissions, production_checklist, concurrency_modes, multi_platform_projects, library_api, toggles, audio_video_coexistence, simulator_browser_mode, event_log_schema, file_actions, connection_ui, manifest_format. Topic IDs are stable. |
Full per-tool reference with input schemas, response shapes, and worked examples: /docs/mcp-server/tools.
The canonical agent-driven flow
In a fresh project, the agent invokes the tools in this order:
1. getPlatformInfo({ sections: ["version", "capabilities"], glasses: "meta_rayban" })
2. getCodeExample({ pattern: "voice_qa_assistant" }) // or whatever pattern fits
3. getCapabilityGuide({ feature: "<each primitive the handler will use>" })
4. generateConnectionModule({ platform, glasses, appPackage })
→ returns "needs_placement" question
5. generateConnectionModule({ ... placement: "<chosen>" })
→ writes scaffold files (ExtentosBootstrap, manifest, etc.)
6. <agent writes handler class(es)> against the SDK primitives
<agent updates extentos.manifest.json's `capabilities` array>
7. validateIntegration()
→ ✓ all good (or returns structured errors to fix)
8. createSimulatorSession({ glasses })
→ returns sessionId; auto-opens browser at extentos.com/s/<id>
→ if running app is reachable via local bridge, it auto-attaches
9. <developer interacts with the simulator; capability events flow into the backend>
10. getEventLog({ sessionId, filter: "errors" }) → debug
getSimulatorStatus({ sessionId }) → statusFor iteration: edit handler code → rebuild + reinstall → the app auto-attaches to the same simulator session (no remint, the URL is stable). Before shipping: getProductionChecklist and getCredentialGuide.
Configuration
The MCP server reads these environment variables (verified in mcp-server/src/):
| Variable | Default | What it does |
|---|---|---|
EXTENTOS_BACKEND_URL | Production backend | Override the backend URL (tools/util/backendClient.ts). For local development of Extentos itself. |
EXTENTOS_CONFIG_DIR | ~/.extentos | Override the config/auth directory (telemetry/consent.ts). |
EXTENTOS_TELEMETRY | unset (consent-default) | Set to 0 to decline telemetry without running the CLI consent command. |
EXTENTOS_NO_AUTO_OPEN | unset | Set to 1 to disable browser auto-open on simulator-session creation (useful in headless environments). |
Full configuration reference: /docs/mcp-server/configuration.
CLI subcommands
Running npx @extentos/mcp-server@latest with no arguments starts the MCP server over stdio (the path the agent uses). With a subcommand, it acts as a developer CLI:
| Subcommand | What it does |
|---|---|
login | Link this install to an Extentos account via device-code flow (proactive — useful before the first simulator session, or after logout to re-link). |
logout | Clear ~/.extentos/auth.json. Install returns to the anonymous tier; the next simulator session call will re-trigger the device-code flow. |
whoami | Print installId, accountId (if linked), tier, auth expiry. |
accept-privacy | Record privacy consent (enables telemetry upload). |
decline-privacy | Record privacy decline (disables telemetry upload). |
status | Print consent state, install ID, linked account, MCP/library versions. |
update | Check for MCP server updates (no-op on npx @latest installs). |
Full CLI reference: /docs/mcp-server/auth.
Auth model
The MCP server is anonymous-first. Every tool except simulator-session minting works with no account. Code generation, validation, real-hardware testing, on-device simulation, and the discovery / capability-guide / code-example surface all work without ever signing in.
The browser simulator is the one exception. The first createSimulatorSession call returns status: "auth_required" with a verification URL. The agent calls completeAuthLink to poll the backend; the developer signs up with a free email-only account at the URL (Google or email + password, no payment); the backend issues a token; the original tool call retries automatically. After linking, simulator sessions are unlimited.
There is no paid tier. Full auth model: /docs/mcp-server/auth and /docs/resources/pricing.
Privacy and telemetry
On first run, the MCP server injects a one-time privacy notice into the response. Telemetry is anonymous (tagged with installId, no source code or personal data) and is dismissed-by-continuing by default — same pattern as Vercel CLI, Astro, Vite. Decline at any time:
npx @extentos/mcp-server@latest decline-privacy
# or
EXTENTOS_TELEMETRY=0 (env var, persistent for the shell)Privacy notice content is in mcp-server/src/index.ts (PRIVACY_NOTICE constant). Notice is shown once per install via claimFirstPrivacyNotice — never repeats.
Compatible MCP hosts
Verified to work with:
- Claude Code — primary target. One-line install via
claude mcp add. - Cursor — JSON config in
~/.cursor/mcp.json. - Windsurf — JSON config in
~/.codeium/windsurf/mcp_config.json. - Cline — JSON config in Cline's MCP settings.
- Any MCP-compatible host — drop the standard
mcpServers.extentosJSON block in.
The MCP server speaks the standard MCP protocol over stdio (@modelcontextprotocol/sdk); no host-specific code paths exist on the server side. Per-host install steps: /docs/mcp-server/agents.
The local bridge — auto-bind dev loop
When the server starts, it opens a 127.0.0.1:31337/whoami HTTP listener (mcp-server/src/localBridge.ts). The Extentos library on the developer's app probes this endpoint at runtime to learn its host MCP's installId. The result: every createSimulatorSession call from the agent auto-attaches the running app to the new session — no rebuild, no URL paste.
Reach paths:
- Android emulator:
http://10.0.2.2:31337/whoami(host-loopback NAT alias) - iOS Simulator:
http://localhost:31337/whoami(shares host network namespace) - Physical Android phone via USB:
adb reverse tcp:31337 tcp:31337once, thenlocalhost:31337from the device - Cellular phone or cloud-hosted agent: probe times out. The agent uses the URL-bake path instead —
createSimulatorSessionreturns aBuildConfig.EXTENTOS_SESSION_URLsnippet (Android) orextentos.session.plistpayload (iOS) the developer pastes in, then rebuilds the app once. Less elegant than auto-bind but works on any topology.
Bound to 127.0.0.1 only. The installId is not a secret — it's the same value the MCP sends to api.extentos.com on every tool call. No auth needed at this layer.
If port 31337 is in use (rare; another MCP instance already running), startup logs a warning and continues. Auto-bind silently fails for that session; the developer uses the URL-bake path until the port frees up.
Status
- Package:
@extentos/mcp-server@0.0.57on npm (MIT license) - Engines: Node.js 20+
- Source: github.com/Asgermolgaard/vibe-hardware/tree/master/mcp-server
- Pre-1.0 — APIs may shift between minor versions until the hardware test loop closes. Pin to an exact version if you need cross-session reproducibility.
Related
- Quickstart with an AI agent — install the server and walk through a real dev loop
- Tools reference — full per-tool API
- Configuration — environment variables, config files, install-time settings
- Auth — device-code flow, account linking, CLI auth commands
- Supported agents — per-host install instructions
- Architecture — how the MCP server fits into the broader Extentos system
- Transport vs app simulation — what the simulator the MCP brokers actually does
From simulator to real glasses
The browser simulator validates your protocol, tool wiring, and audio bytes; real Ray-Ban Meta glasses are the final gate for audio character, Bluetooth timing, and hardware failure modes. This guide walks the full transition — getting a Meta production identity (App ID, Client Token, package + SHA-256 registration), storing credentials in your dashboard so generateConnectionModule bakes them into the build, resolving the Meta DAT SDK from GitHub Packages with a read:packages token, flipping the transport off the simulator, pairing glasses to an Android device, and running the real-hardware verification pass — plus exactly what the simulator already proved and what it can't.
Install the MCP server
How to install the Extentos MCP server (@extentos/mcp-server) in any MCP-compatible AI coding agent — Claude Code, Cursor, Windsurf, Cline, and others. Per-host install commands, config file locations, copy-pasteable JSON snippets, restart and verify steps, version pinning, updating, troubleshooting common errors, and uninstall instructions. Verified install paths for each supported host.