---
title: MCP server
description: The Extentos MCP server (`@extentos/mcp-server`) is an npm package an AI agent (Claude Code, Cursor, Windsurf, Cline) installs once and then uses to add Meta Ray-Ban smart-glasses capabilities to a native iOS or Android app. It exposes a tight set of deterministic tools across 10 categories — discovery, generation, agent configuration, credentials, analytics, guidance, validation, simulation, production-readiness, and documentation — plus a CLI for account linking, telemetry consent, and update checks. This is the agent's operating manual.
type: overview
platform: mcp
related:
  - /docs/getting-started/with-agent
  - /docs/concepts/architecture
  - /docs/concepts/transport-vs-app
  - /docs/mcp-server/install
  - /docs/mcp-server/tools
  - /docs/mcp-server/configuration
  - /docs/mcp-server/auth
  - /docs/mcp-server/agents
---

The MCP server is how an AI agent — Claude Code, Cursor, Windsurf, Cline, or any [Model Context Protocol](https://modelcontextprotocol.io)-compatible host — operates Extentos. The agent invokes deterministic tools; the server primes the agent on what capabilities the glasses expose, returns canonical SDK code patterns in Kotlin and Swift, scaffolds the project, brokers simulator sessions, and queries debugging traces. The server itself has **no planning tool** — agents are better planners than regex bundles. Tools are typed primitives the agent composes in sequence.

This page is the section landing — what the server is, the tools at a glance, the canonical agent-driven flow, configuration knobs, the CLI, and how the auth model works. Sub-pages cover each in depth.

## Install

```bash
claude mcp add extentos -- npx -y @extentos/mcp-server@latest
```

For non-Claude-Code hosts, see the [agent prompt or manual JSON](/docs/getting-started/with-agent#install) install paths. Full install reference at [/docs/mcp-server/install](/docs/mcp-server/install).

## The tools, by category

The server exposes a deterministic tool surface (verified in `mcp-server/src/tools/definitions.ts`), organized into 10 categories. The categories are the agent's mental map; an agent that understands which category a tool lives in can decide when to call it. The always-current full catalog is the generated [tools reference](/docs/reference/mcp-tools).

### 1. Discovery and SDK reference (4 tools)

The first calls in any new task. Cheap, all local, no side effects.

| Tool | What it does |
|---|---|
| `getPlatformInfo` | Returns the static platform catalog — library version, the list of SDK capabilities the glasses expose, per-vendor tiers. Always the right first call. |
| `getCapabilityGuide` | Per-feature minimal usage in Kotlin + Swift — call shape, config args, gotchas. Pairs with `getPlatformInfo` (which names the features) to tell the agent how to invoke each. |
| `getCodeExample` | Full canonical compositions in both languages. Start with `assistant_agent_loop` (the Phase-4 canonical voice-assistant flow) and `agent_driven_e2e_full_loop` (the agent-driven E2E test). Also covers `voice_qa_assistant`, `barge_in_speak`, `photo_describe_voice`, `live_transcription_ui`, `voice_notes`, `connection_page_setup`, `byok_anthropic`, `display_browse_detail`, `display_media_gallery`, and `video_frames_ml`. Peel from these when writing handler code. The full enumerated list is generated at [/docs/reference/mcp-tools](/docs/reference/mcp-tools). |
| `getMigrationGuide` | For apps already built on raw Meta DAT — returns a map keyed by your existing DAT symbols to the Extentos primitives that replace them, plus an ordered cutover plan. |

### 2. Setup and Generation

| Tool | What it does |
|---|---|
| `generateConnectionModule` | One-shot scaffold — bootstrap module, Gradle/SPM wiring, dependencies, permissions, manifest. Two-call flow: first call without `placement` returns a question asking where `ExtentosConnectionPage` should live; second call with the chosen placement returns the full file set. |
| `getConnectionPageConfig` / `setConnectionPageConfig` | Read / write the per-project connection-page config (theming tokens + section visibility) the dashboard/server holds. |
| `regenerateConnectionPageFile` / `adoptConnectionPageFile` | Sync the committed `extentos.connection-page.json` with the server config — regenerate (server→file) or adopt (file→server). |

After scaffolding, the agent writes its own handler classes against the SDK primitives surfaced by `getCapabilityGuide` / `getCodeExample`. Handler code is the customer's authoring surface — there's no `initSpec` or DSL-population step.

### 3. Agent configuration & usage (5 tools)

Account-scoped — require a linked account and are gated per-project by the [MCP access grant](/docs/mcp-server/auth) (default Read+Write).

| Tool | What it does |
|---|---|
| `getAssistantConfig` / `setAssistantConfig` | Read or change a project's dashboard-managed assistant settings — the OpenAI Realtime model, the voice, the memory (compaction) model, and the within-session memory mode. `set` is a partial update, validates each value against the catalog, and echoes the cost impact of a model change. |
| `listProjectSounds` / `addProjectSound` | List or upload a project's named-sound library — MP3/M4A/AAC/WAV files under 1 MB that app code plays with `glasses.audio.playSound(name)` (iOS today; the Android playback ride is landing) — the dashboard can also pick one as the wake sound. Gated by the `assistant_config.sounds` grant. |
| `getGatewayUsage` | Read the project's managed-gateway usage + exact cost over a recent window — token counts and list-price USD from the billing ledger, broken down by model. Metadata only, never transcripts or content. |

### 4. Credentials (2 tools)

Account-scoped + per-project gated. Write-without-knowing — the secret never passes through the agent.

| Tool | What it does |
|---|---|
| `getCredentialStatus` | Read whether a project's Meta DAT build identity is set — a masked hint + updated-at only, never the value. |
| `setCredential` | Start write-without-knowing entry of a credential — returns a dashboard link where the signed-in owner pastes the secret straight into the encrypted vault. Takes no secret argument by design. |

### 5. Analytics (1 tool)

| Tool | What it does |
|---|---|
| `getProjectAnalytics` | Read a project's **production** analytics — aggregate telemetry from shipped App Store / Play Store installs (events, active installs, by event / day / vendor / platform). Metadata only, account-scoped, ownership-checked, gated by the Analytics grant. Empty until the app ships and sends prod-attested events (use `getEventLog` for the live dev/sim stream). |

### 6. Implementation Guidance (2 tools)

Side-quest tools the agent calls during composition.

| Tool | What it does |
|---|---|
| `getVoiceCommandGuidance` | Analyze proposed wake / command phrases for UX issues (collisions, ambiguity, hard-to-recognize words, Meta wake-word conflicts) before wiring them into a `glasses.audio.transcriptions()` consumer. |
| `getPermissions` | Derive exact platform permissions, Meta DAT requirements, and foreground-service needs from the capability list. Run when adding or removing a primitive from your handler. |

### 7. Validation (2 tools)

Correctness gates. Run after structural changes (new capability declared, dependency bumped, manifest edited).

| Tool | What it does |
|---|---|
| `inspectIntegration` | Read-only project snapshot — manifest, generated-file hashes, dependency list, connection-page config. Run before manual edits to understand current state. |
| `validateIntegration` | Whole-project correctness check — manifest, generated files, dependency declared, permissions cover declared capabilities, bootstrap calls `ExtentosGlasses.create(...)`, toolchain versions, foreground-service hints for continuous-capture flows. The pre-test gate. |

### 8. Simulation

Provision and operate browser-based simulator sessions, plus the agent-driven testing tools that close the end-to-end loop without a human.

| Tool | What it does |
|---|---|
| `createSimulatorSession` | Get-or-create a browser-mode session at [extentos.com/s](https://extentos.com/s). Returns the saved sim for this project if one exists (`status: "resumed"`), or mints a new one (`status: "active"`). Auto-attaches the running app via the local bridge when reachable; otherwise emits a `BuildConfig.EXTENTOS_SESSION_URL` snippet (Android) or `extentos.session.plist` payload (iOS). Rotating the sessionId is `deleteSimulatorSession` then mint — there is no force-fresh flag. |
| `ensureSimulatorBrowser` | Open + confirm a connected simulator browser tab — the precondition for camera and inject flows. |
| `completeAuthLink` | After `createSimulatorSession` returns `status: "auth_required"` (anonymous install needs to link to mint sessions), polls the backend until the user finishes signup, then persists the bearer token to `~/.extentos/auth.json`. |
| `getEventLog` | Fetch structured event traces from a session. Filter values: `all` (no filter) plus the seven chips `errors`, `voice`, `camera`, `display`, `ai`, `lifecycle`, `custom` — one chip per event, with `errors` absorbing severity≥warn regardless of modality. Plus `cursor`, `follow`, `limit` for trace-level scope. The primary debugging tool. |
| `getSimulatorStatus` | Read a live session's current state — phase, hardware-ready, attached roles, active capability streams, current toggle values. |
| `injectTranscript` / `injectAssistantUtterance` / `assertToolCalled` | Drive a wake phrase or an assistant turn, then assert which tool the model called — the agent-driven E2E loop, no human needed. |
| `injectHardwareButton` | Press the simulated glasses' hardware capture button — tap pauses/resumes a live camera stream, hold stops it — so the agent can exercise the wearer's privacy gestures (and test the resulting `CaptureError.StreamPaused`) with no human. |
| `setSimVideo` / `setSimDevice` | Pipe a test video into the simulated camera; switch the simulated device model (e.g. `rayban_display` to exercise the display path). |
| `getDisplayState` / `injectInput` | Read the display tree currently rendered + drive display input (`select` / `navigate` / `back`). |

### 9. Production (2 tools)

Pre-ship checks.

| Tool | What it does |
|---|---|
| `getProductionChecklist` | Personalized production-readiness checklist based on declared capabilities + handler names — credential wiring, permission audit, foreground-service requirements (when continuous capture is used), simulator-URL removal from release builds, store-listing readiness. |
| `getCredentialGuide` | Step-by-step credential setup for production AI providers — `anthropic`, `openai`, `google_cloud_vision`, `google_translate`, `google_gemini`, `deepl`, `azure_cognitive`, `aws_bedrock`, `huggingface`, or `custom` — plus Meta DAT registration. |

### 10. Documentation & search (1 tool)

| Tool | What it does |
|---|---|
| `searchDocs` | Search Extentos documentation by topic or keyword. For voice assistants read `assistant_runtime` first; `conversation_runtime` is the deprecated Phase-3 runtime (migration reference only). Other aligned topics: `voice_integration`, `agent_e2e_testing`, `managed_gateway`, `conversation_memory`, `display`, plus the stable conceptual set — `getting_started`, `custom_handlers` (the canonical SDK-composition doc), `simulator_browser_mode`, `simulator_session_lifecycle`, `event_log_schema`, `toggles`, `library_api`, `permissions`, `multi_platform_projects`. Topic IDs are stable; the live tool input is authoritative. |

Full per-tool reference with input schemas, response shapes, and worked examples: [/docs/mcp-server/tools](/docs/mcp-server/tools).

## The canonical agent-driven flow

In a fresh project, the agent invokes the tools in this order:

```text
1. getPlatformInfo({ sections: ["version", "capabilities"], glasses: "meta_rayban" })
2. getCodeExample({ pattern: "assistant_agent_loop" })  // Phase-4 voice assistant; or whatever pattern fits
3. getCapabilityGuide({ feature: "<each primitive the handler will use>" })
4. generateConnectionModule({ platform, glasses, appPackage })
   → returns "needs_placement" question
5. generateConnectionModule({ ... placement: "<chosen>" })
   → writes scaffold files (ExtentosBootstrap, manifest, etc.)
6. <agent writes handler class(es)> against the SDK primitives
   <agent updates extentos.manifest.json's `capabilities` array>
7. validateIntegration()
   → ✓ all good (or returns structured errors to fix)
8. createSimulatorSession({ glasses })
   → returns sessionId; auto-opens browser at extentos.com/s/<id>
   → if running app is reachable via local bridge, it auto-attaches
9. <developer interacts with the simulator; capability events flow into the backend>
10. getEventLog({ sessionId, filter: "errors" })  → debug
    getSimulatorStatus({ sessionId })             → status
```

For iteration: edit handler code → rebuild + reinstall → the app auto-attaches to the same simulator session (no remint, the URL is stable). Before shipping: `getProductionChecklist` and `getCredentialGuide`.

## Configuration

The MCP server reads these environment variables (verified in `mcp-server/src/`):

| Variable | Default | What it does |
|---|---|---|
| `EXTENTOS_BACKEND_URL` | Production backend | Override the backend URL (`tools/util/backendClient.ts`). For local development of Extentos itself. |
| `EXTENTOS_CONFIG_DIR` | `~/.extentos` | Override the config/auth directory (`telemetry/consent.ts`). |
| `EXTENTOS_TELEMETRY` | unset (consent-default) | Set to `0` to decline telemetry without running the CLI consent command. |
| `EXTENTOS_NO_AUTO_OPEN` | unset | Set to `1` to disable browser auto-open on simulator-session creation (useful in headless environments). |

Full configuration reference: [/docs/mcp-server/configuration](/docs/mcp-server/configuration).

## CLI subcommands

Running `npx @extentos/mcp-server@latest` with no arguments starts the MCP server over stdio (the path the agent uses). With a subcommand, it acts as a developer CLI:

| Subcommand | What it does |
|---|---|
| `login` | Link this install to an Extentos account via device-code flow (proactive — useful before the first simulator session, or after `logout` to re-link). |
| `logout` | Clear `~/.extentos/auth.json`. Install returns to the anonymous tier; the next simulator session call will re-trigger the device-code flow. |
| `whoami` | Not yet implemented (Phase-0 stub). Will print `installId`, `accountId` (if linked), tier, auth expiry. |
| `setup` | Preflight the local build environment — checks the GitHub Packages PAT (`read:packages`) the Meta DAT artifacts need, for apps that depend on `com.extentos:glasses-meta`, plus other dependency prerequisites. |
| `accept-privacy` | Record privacy consent (enables telemetry upload). |
| `decline-privacy` | Record privacy decline (disables telemetry upload). |
| `status` | Print consent state, install ID, linked account, MCP/library versions. |
| `update` | Check for MCP server updates (no-op on `npx @latest` installs). |

Full CLI reference: [/docs/mcp-server/auth](/docs/mcp-server/auth).

## Auth model

The MCP server is **anonymous-first**. Discovery, capability guides, code examples, validation, docs search, on-device simulation, and real-hardware testing all work without ever signing in. Three things link a free account: minting browser-simulator sessions (`createSimulatorSession`, HTTP 402), the `generateConnectionModule` scaffold step (it mints your account-bound project key — same 402 device-code flow; the informational first call is anonymous), and the account-scoped project tools (assistant config, credentials, connection-page writes, analytics — HTTP 401).

The device-code flow: the first gated call returns `status: "auth_required"` with a verification URL. The agent calls `completeAuthLink` to poll the backend; the developer signs up with a free email-only account at the URL (Google or email + password, no payment); the backend issues a token; the original tool call retries automatically. After linking, simulator sessions are unlimited.

Extentos's tools, codegen, validation, SDK, and the browser simulator are free — there's no per-seat or subscription fee to build and ship. The one surface that meters usage is the **managed AI gateway** behind the Phase-4 voice assistant. Full auth model: [/docs/mcp-server/auth](/docs/mcp-server/auth); pricing: [/docs/resources/pricing](/docs/resources/pricing).

## Privacy and telemetry

On first run, the MCP server injects a one-time privacy notice into the response. Telemetry is anonymous (tagged with `installId`, no source code or personal data) and is dismissed-by-continuing by default — same pattern as Vercel CLI, Astro, Vite. Decline at any time:

```bash
npx @extentos/mcp-server@latest decline-privacy
# or
EXTENTOS_TELEMETRY=0 (env var, persistent for the shell)
```

Privacy notice content is in `mcp-server/src/index.ts` (`PRIVACY_NOTICE` constant). Notice is shown once per install via `claimFirstPrivacyNotice` — never repeats.

## Compatible MCP hosts

Verified to work with:

- **Claude Code** — primary target. One-line install via `claude mcp add`.
- **Cursor** — JSON config in `~/.cursor/mcp.json`.
- **Windsurf** — JSON config in `~/.codeium/windsurf/mcp_config.json`.
- **Cline** — JSON config in Cline's MCP settings.
- **Any MCP-compatible host** — drop the standard `mcpServers.extentos` JSON block in.

The MCP server speaks the standard MCP protocol over stdio (`@modelcontextprotocol/sdk`); no host-specific code paths exist on the server side. Per-host install steps: [/docs/mcp-server/agents](/docs/mcp-server/agents).

## The local bridge — auto-bind dev loop

When the server starts, it opens a `127.0.0.1:31337/whoami` HTTP listener (`mcp-server/src/localBridge.ts`). The Extentos library on the developer's app probes this endpoint at runtime to learn its host MCP's `installId`. The result: every `createSimulatorSession` call from the agent auto-attaches the running app to the new session — no rebuild, no URL paste.

Reach paths:
- **Android emulator:** `http://10.0.2.2:31337/whoami` (host-loopback NAT alias)
- **iOS Simulator:** `http://localhost:31337/whoami` (shares host network namespace)
- **Physical Android phone via USB:** `adb reverse tcp:31337 tcp:31337` once, then `localhost:31337` from the device
- **Cellular phone or cloud-hosted agent:** probe times out. The agent uses the URL-bake path instead — `createSimulatorSession` returns a `BuildConfig.EXTENTOS_SESSION_URL` snippet (Android) or `extentos.session.plist` payload (iOS) the developer pastes in, then rebuilds the app once. Less elegant than auto-bind but works on any topology.

Bound to `127.0.0.1` only. The `installId` is not a secret — it's the same value the MCP sends to `api.extentos.com` on every tool call. No auth needed at this layer.

If port `31337` is in use (rare; another MCP instance already running), startup logs a warning and continues. Auto-bind silently fails for that session; the developer uses the URL-bake path until the port frees up.

## Status

- **Package:** [`@extentos/mcp-server`](https://www.npmjs.com/package/@extentos/mcp-server) on npm (MIT license)
- **Engines:** Node.js 20+
- **Pre-1.0** — APIs may shift between minor versions until the hardware test loop closes. Pin to an exact version if you need cross-session reproducibility.

## Related

- **[Quickstart with an AI agent](/docs/getting-started/with-agent)** — install the server and walk through a real dev loop
- **[Tools reference](/docs/mcp-server/tools)** — full per-tool API
- **[Configuration](/docs/mcp-server/configuration)** — environment variables, config files, install-time settings
- **[Auth](/docs/mcp-server/auth)** — device-code flow, account linking, CLI auth commands
- **[Supported agents](/docs/mcp-server/agents)** — per-host install instructions
- **[Architecture](/docs/concepts/architecture)** — how the MCP server fits into the broader Extentos system
- **[Transport vs app simulation](/docs/concepts/transport-vs-app)** — what the simulator the MCP brokers actually does