MCP server
The Extentos MCP server (`@extentos/mcp-server`) is an npm package an AI agent (Claude Code, Cursor, Windsurf, Cline) installs once and then uses to add Meta Ray-Ban smart-glasses capabilities to a native iOS or Android app. It exposes 18 deterministic tools across 7 categories — discovery, generation, guidance, validation, simulation, production-readiness, and documentation — plus a CLI for account linking, telemetry consent, and update checks. This is the agent's operating manual.
The MCP server is how an AI agent — Claude Code, Cursor, Windsurf, Cline, or any Model Context Protocol-compatible host — operates Extentos. The agent invokes deterministic tools; the server reads project state, generates code, mutates the AppSpec, brokers simulator sessions, and queries debugging traces. The server itself has no planning tool — agents are better planners than regex bundles. Tools are typed primitives the agent composes in sequence.
This page is the section landing — what the server is, the 18 tools at a glance, the canonical agent-driven flow, configuration knobs, the CLI, and how the auth model works. Sub-pages cover each in depth.
Install
claude mcp add extentos -- npx -y @extentos/mcp-server@latestFor non-Claude-Code hosts, see the agent prompt or manual JSON install paths. Full install reference at /docs/mcp-server/install.
The 18 tools, by category
The server exposes exactly 18 tools (verified in mcp-server/src/tools/definitions.ts), organized into 7 categories. The categories are the agent's mental map; an agent that understands which category a tool lives in can decide when to call it.
1. Discovery and Planning (2 tools)
The first calls in any new task. Cheap, all local, no side effects.
| Tool | What it does |
|---|---|
getPlatformInfo | Returns the static platform catalog — library version, spec schema, glasses capabilities — for the target vendor (meta_rayban). Always the right first call. |
getExampleSpec | Reference library — retrieve one of 8 worked example specs (voice_assistant, live_translation, scene_description, notification_reader, navigation_guide, video_call, object_detection, voice_notes). Use to study complex compositions; not as a starting template. |
2. Setup and Generation (4 tools)
The mutation tools that scaffold the project and the AppSpec. Run in this order on a fresh project.
| Tool | What it does |
|---|---|
generateConnectionModule | One-shot scaffold — bootstrap module, empty spec, manifest entries, dependency wiring, permission stubs. Two-call flow: first call without placement returns a question; second call with the chosen placement returns the full file set. |
initSpec | Create the FIRST populated spec for a fresh project. Refuses if spec is already non-empty. |
generateConsumer | Stub app_callback handlers (or stream consumers) for whatever the spec declares. |
updateSpec | Mutate an existing spec. Requires the basedOn freshness token from inspectIntegration to detect stale concurrent edits. |
3. Implementation Guidance (2 tools)
Side-quest tools the agent calls during composition.
| Tool | What it does |
|---|---|
getVoiceCommandGuidance | Analyze proposed voice-command phrases for UX issues (collisions, ambiguity, hard-to-recognize words) before adding voice_command triggers. |
getPermissions | Derive exact platform permissions, Meta DAT requirements, and foreground-service needs from a spec. Run after creating or updating the spec. |
4. Validation (3 tools)
Correctness gates. Run after every mutation.
| Tool | What it does |
|---|---|
validateSpec | Validate a detached spec JSON against the 11 schema rules. Pure JSON check, no filesystem. |
inspectIntegration | Read-only project snapshot — manifest, spec, generated-file hashes, dependency list, plus the basedOn freshness token. Run before every updateSpec or generateConsumer. |
validateIntegration | Whole-project correctness check — spec, files, manifest, dependencies, permissions, handler coverage, bootstrap wiring. The pre-test gate. |
5. Simulation (4 tools)
Provision and operate browser-based simulator sessions.
| Tool | What it does |
|---|---|
createSimulatorSession | Provision a browser-mode session at extentos.com/s. Auto-attaches the running app via the local bridge if reachable; otherwise emits a BuildConfig.EXTENTOS_SESSION_URL snippet (Android) or extentos.session.plist payload (iOS). Calling this with a session already live supersedes it cleanly. |
completeAuthLink | After createSimulatorSession returns status: "auth_required" (1000-event meter exhausted), polls the backend until the user finishes signup, then persists the bearer token to ~/.extentos/auth.json. |
getEventLog | Fetch structured event traces from a session. Filters: all, errors, transport, triggers, blocks, callbacks, toggles, streams, system, spec. Plus flowId, since, limit for trace-level scope. The primary debugging tool. |
getSimulatorStatus | Read a live session's current state — phase, hardware-ready, roles, active streams/triggers. |
6. Production (2 tools)
Pre-ship checks.
| Tool | What it does |
|---|---|
getProductionChecklist | Personalized production-readiness checklist based on what the spec uses. |
getCredentialGuide | Step-by-step credential setup for production AI providers — anthropic, openai, google_cloud_vision, google_translate, google_gemini, deepl, azure_cognitive, aws_bedrock, huggingface, or custom. |
7. Documentation (1 tool)
| Tool | What it does |
|---|---|
searchDocs | Search Extentos documentation by topic or keyword. Catalog topics (trigger_types, action_types, block_types, stream_types, spec_format, template_syntax, app_callback_guide, library_api) carry inline minimal examples — most specs can be authored from these without ever fetching a getExampleSpec pattern. |
Full per-tool reference with input schemas, response shapes, and worked examples: /docs/mcp-server/tools.
The canonical agent-driven flow
In a fresh project, the agent invokes the tools in this order:
1. getPlatformInfo({ sections: ["version", "capabilities"], glasses: "meta_rayban" })
2. searchDocs({ topic: "trigger_types" | "action_types" | "block_types" })
3. generateConnectionModule({ platform, glasses, appPackage })
→ returns "needs_placement" question
4. generateConnectionModule({ ... placement: "<chosen>" })
→ writes scaffold files
5. initSpec({ ... })
→ first AppSpec written
6. generateConsumer({ kind: "callback" })
→ handler stubs written
7. validateIntegration()
→ ✓ all good (or returns structured errors to fix)
8. createSimulatorSession({ glasses, spec })
→ returns sessionId; auto-opens browser at extentos.com/s/<id>
→ if running app is reachable via local bridge, it auto-attaches
9. <developer interacts with the simulator; events flow into the backend>
10. getEventLog({ sessionId, filter: "errors" }) → debug
getSimulatorStatus({ sessionId }) → statusFor iteration, the loop is inspectIntegration (for the freshness token) → updateSpec → generateConsumer (if handlers/streams added) → validateIntegration. Before shipping, getProductionChecklist and getCredentialGuide.
Configuration
The MCP server reads these environment variables (verified in mcp-server/src/):
| Variable | Default | What it does |
|---|---|---|
EXTENTOS_BACKEND_URL | Production backend | Override the backend URL (tools/util/backendClient.ts). For local development of Extentos itself. |
EXTENTOS_CONFIG_DIR | ~/.extentos | Override the config/auth directory (telemetry/consent.ts). |
EXTENTOS_TELEMETRY | unset (consent-default) | Set to 0 to decline telemetry without running the CLI consent command. |
EXTENTOS_NO_AUTO_OPEN | unset | Set to 1 to disable browser auto-open on simulator-session creation (useful in headless environments). |
Full configuration reference: /docs/mcp-server/configuration.
CLI subcommands
Running npx @extentos/mcp-server@latest with no arguments starts the MCP server over stdio (the path the agent uses). With a subcommand, it acts as a developer CLI:
| Subcommand | What it does |
|---|---|
login | Link this install to an Extentos account via device-code flow (only needed after the 1000-event meter exhausts). |
logout | Clear ~/.extentos/auth.json. Install returns to the anonymous tier. |
whoami | Print installId, accountId (if linked), tier, meter remaining, auth expiry. |
accept-privacy | Record privacy consent (enables telemetry upload). |
decline-privacy | Record privacy decline (disables telemetry upload). |
status | Print consent state, install ID, linked account, MCP/library versions. |
update | Check for MCP server updates (no-op on npx @latest installs). |
Full CLI reference: /docs/mcp-server/auth.
Auth model
The MCP server is anonymous-first. All 18 tools work with no account. The first browser-simulator session is anonymous, keyed to a per-machine installId at ~/.extentos/install_id.
The 1000-event meter is the only gate. When it exhausts, createSimulatorSession returns status: "auth_required" with a verification URL. The agent calls completeAuthLink to poll the backend; the developer signs up with email (no payment) at the URL; the backend issues a token; the original tool call retries automatically.
There is no paid tier at launch. Full auth model: /docs/mcp-server/auth and /docs/resources/pricing.
Privacy and telemetry
On first run, the MCP server injects a one-time privacy notice into the response. Telemetry is anonymous (tagged with installId, no source code or personal data) and is dismissed-by-continuing by default — same pattern as Vercel CLI, Astro, Vite. Decline at any time:
npx @extentos/mcp-server@latest decline-privacy
# or
EXTENTOS_TELEMETRY=0 (env var, persistent for the shell)Privacy notice content is in mcp-server/src/index.ts (PRIVACY_NOTICE constant). Notice is shown once per install via claimFirstPrivacyNotice — never repeats.
Compatible MCP hosts
Verified to work with:
- Claude Code — primary target. One-line install via
claude mcp add. - Cursor — JSON config in
~/.cursor/mcp.json. - Windsurf — JSON config in
~/.codeium/windsurf/mcp_config.json. - Cline — JSON config in Cline's MCP settings.
- Any MCP-compatible host — drop the standard
mcpServers.extentosJSON block in.
The MCP server speaks the standard MCP protocol over stdio (@modelcontextprotocol/sdk); no host-specific code paths exist on the server side. Per-host install steps: /docs/mcp-server/agents.
The local bridge — auto-bind dev loop
When the server starts, it opens a 127.0.0.1:31337/whoami HTTP listener (mcp-server/src/localBridge.ts). The Extentos library on the developer's app probes this endpoint at runtime to learn its host MCP's installId. The result: every createSimulatorSession call from the agent auto-attaches the running app to the new session — no rebuild, no URL paste.
Reach paths:
- Android emulator:
http://10.0.2.2:31337/whoami(host-loopback NAT alias) - iOS Simulator:
http://localhost:31337/whoami(shares host network namespace) - Physical Android phone via USB:
adb reverse tcp:31337 tcp:31337once, thenlocalhost:31337from the device - Cellular phone or cloud-hosted agent: probe times out. The agent uses the URL-bake path instead —
createSimulatorSessionreturns aBuildConfig.EXTENTOS_SESSION_URLsnippet (Android) orextentos.session.plistpayload (iOS) the developer pastes in, then rebuilds the app once. Less elegant than auto-bind but works on any topology.
Bound to 127.0.0.1 only. The installId is not a secret — it's the same value the MCP sends to api.extentos.com on every tool call. No auth needed at this layer.
If port 31337 is in use (rare; another MCP instance already running), startup logs a warning and continues. Auto-bind silently fails for that session; the developer uses the URL-bake path (paste the EXTENTOS_SESSION_URL Android snippet or write the extentos.session.plist iOS payload that createSimulatorSession returns, then rebuild). The agent and tools keep working — only the zero-rebuild auto-attach is lost until the port frees up.
Status
- Package:
@extentos/mcp-server@0.0.16on npm (MIT license) - Engines: Node.js 20+
- Source: github.com/Asgermolgaard/vibe-hardware/tree/master/mcp-server
- Pre-1.0 — APIs may shift between minor versions until the hardware test loop closes. Pin to an exact version (
@0.0.16) if you need cross-session reproducibility.
Related
- Quickstart with an AI agent — install the server and walk through a real dev loop
- Tools reference — full per-tool API
- Configuration — environment variables, config files, install-time settings
- Auth — device-code flow, account linking, CLI auth commands
- Supported agents — per-host install instructions
- Architecture — how the MCP server fits into the broader Extentos system
- Transport vs app simulation — what the simulator the MCP brokers actually does
Handle disconnects
Gracefully detect when the glasses disconnect and reconnect — UX patterns, retry policy, error states.
Install the MCP server
How to install the Extentos MCP server (@extentos/mcp-server) in any MCP-compatible AI coding agent — Claude Code, Cursor, Windsurf, Cline, and others. Per-host install commands, config file locations, copy-pasteable JSON snippets, restart and verify steps, version pinning, updating, troubleshooting common errors, and uninstall instructions. Verified install paths for each supported host.