MCP tools reference
The complete, always-current catalog of @extentos/mcp-server tools — 38 deterministic tools across 10 categories for discovering smart-glasses capabilities, scaffolding native iOS/Android integrations, driving the browser simulator, and gating production readiness. Generated from the live tool registry.
Generated from source. This page is emitted from
mcp-server/src/tools/definitions.ts— the exact tool registry your agent loads. It lists every tool currently shipped. The authoritative live surface is what your installed agent sees via the MCPtools/listcall; this page is the static mirror for pre-install evaluation and search.
The Extentos MCP server exposes 38 deterministic tools (zero LLM inference on the Extentos side) across 10 categories. Install it with:
claude mcp add extentos -- npx -y @extentos/mcp-server@latestCurrent published version and changelog: npmjs.com/package/@extentos/mcp-server. Each tool's parameters below are generated from its input schema; for the full narrative on when to reach for each, follow the per-category guide links.
Discovery & SDK reference
Learn what the glasses can do and the canonical call shape for each primitive before writing handler code — plus the migration path for apps already built on raw Meta DAT. Narrative guide: /docs/mcp-server/tools/discovery.
getPlatformInfo
Return static platform metadata: library version + the list of SDK capabilities the glasses expose.
| Parameter | Type | Required | Description |
|---|---|---|---|
sections | string[] (enum) | yes | One of: version, capabilities. |
glasses | string (enum) | no | Target glasses vendor. One of: meta, android_xr, brilliant, meta_rayban. |
expand | string[] (enum) | no | Optional opt-in expansions on top of the compact default. One of: schema, capabilities.full, capabilities.advanced. |
getCodeExample
Reference library — retrieve a complete SDK code example (Kotlin + Swift) for a use case.
| Parameter | Type | Required | Description |
|---|---|---|---|
pattern | string (enum) | yes | Which compositional pattern to fetch. One of: agent_driven_e2e_full_loop, agent_test_loop, assistant_agent_loop, barge_in_speak, byok_anthropic, connection_page_setup, conversation_agent_loop, display_browse_detail, display_media_gallery, live_transcription_ui, …(+4). |
getCapabilityGuide
Per-feature SDK usage guide — minimal Kotlin + Swift snippet + gotchas + which getCodeExample patterns exercise the feature.
| Parameter | Type | Required | Description |
|---|---|---|---|
feature | string (enum) | yes | Feature name from getPlatformInfo.features[].name. One of: assistant_provider_openai, assistant_runtime, assistant_session_runtime, assistant_start, assistant_tool, assistant_vision, audio_chunks, cancel_speak, capture_photo, capture_video, …(+14). |
getMigrationGuide
The entry point for a developer who ALREADY built their app against raw Meta DAT (the Device Access Toolkit / Wearables SDK) and wants to move onto the Extentos SDK.
| Parameter | Type | Required | Description |
|---|---|---|---|
from | string (enum) | no | Which SDK the app is migrating FROM. One of: meta_dat. |
Setup & generation
Scaffold the connection module into a host app and manage the per-project connection-page config. Narrative guide: /docs/mcp-server/tools/generation.
generateConnectionModule
One-shot project scaffold — emits the bootstrap module that wires Extentos.create(...) into the host app, build-script changes, dependencies, permissions, and the integration manifest.
| Parameter | Type | Required | Description |
|---|---|---|---|
platform | string (enum) | yes | One of: android, ios. |
glasses | string (enum) | yes | Target glasses vendor. One of: meta, android_xr, brilliant, meta_rayban. |
appPackage | string | yes | The Extentos project identity (reverse-DNS). |
libraryVersion | string | no | — |
responseFormat | string (enum) | no | One of: concise, detailed. |
placement | string (enum) | no | Where ExtentosConnectionPage should live in the app. One of: dedicated_route, settings_subscreen, bottom_tab, modal_sheet, headless. |
projectPath | string | no | Absolute path to the project root. |
capabilities | string[] | no | The SDK capabilities your app uses (feature names from getPlatformInfo — e.g. |
getConnectionPageConfig
Read the per-project ExtentosConnectionPage config (theming tokens + section visibility) the dashboard/server holds for an app.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id), e.g. |
setConnectionPageConfig
Persist the per-project ExtentosConnectionPage config (theming tokens + section visibility) to the dashboard/server, making the app MANAGED — the dashboard/server becomes the single source of truth and the SDK fetches it at render time (se…
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id). |
config | object | yes | The config to persist: { schemaVersion?, sections?, appearance? |
regenerateConnectionPageFile
Regenerate the committed extentos.connection-page.json FROM the dashboard/server config (file ← server) — the generated mirror that makes the dashboard's tokens real in the repo and lets the app theme correctly offline.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id). |
platform | string (enum) | no | Defaults to android. One of: android, ios. |
projectPath | string | no | Absolute path to the project root. |
adoptConnectionPageFile
Adopt the committed extentos.connection-page.json UP to the dashboard/server (file → server) — the deliberate one-time seed of an existing code/file theme into the managed config.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id). |
platform | string (enum) | no | Defaults to android. One of: android, ios. |
projectPath | string | yes | Absolute path to the project root (required — reads app/src/main/assets/extentos.connection-page.json). |
confirm | boolean | no | Set true to overwrite an existing, DIFFERING server config with the committed file. |
Agent configuration & usage
Read or change a project's dashboard-managed assistant settings (Realtime model, voice, memory model + mode), and read its managed-gateway usage + exact cost. Account-scoped; writes are gated per-project by the MCP access grant (default Read+Write) and echo the cost impact. Narrative guide: /docs/mcp-server/tools.
getAssistantConfig
Read the per-project "Agent" (assistant) settings the dashboard holds for an app — the OpenAI Realtime model, the voice, the memory (compaction) model, and the within-session memory mode — plus the catalog of valid options for each, the da…
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id), e.g. |
setAssistantConfig
Change a project's dashboard-managed "Agent" (assistant) settings via MCP — any of the OpenAI Realtime model, voice, memory (compaction) model, or within-session memory mode (PARTIAL update; pass only what you're changing).
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id). |
realtimeModel | string (enum) | no | OpenAI Realtime model id. One of: gpt-realtime-2.1, gpt-realtime-2.1-mini, gpt-realtime-2, gpt-realtime-1.5, gpt-realtime-mini, grok-voice-think-fast-1.0, gemini-3.1-flash-live-preview, gemini-2.5-flash-native-audio-preview-12-2025, local-auto, local-qwen3-8b, …(+4). |
voice | string (enum) | no | Voice id. One of: marin, cedar, ash, ballad, coral, sage, verse, alloy, echo, shimmer, …(+25). |
compactionModel | string (enum) | no | Memory (compaction) model id. One of: gpt-4o-mini, gpt-5.4-nano, gpt-5.4-mini, gpt-5.4, gpt-5.5, gemini-3.1-pro-preview, gemini-3.5-flash, gemini-3.1-flash-lite, gemini-2.5-flash-lite, claude-opus-4-8, …(+2). |
withinSessionMemory | string (enum) | no | Within-session memory mode. One of: smart, basic. |
listProjectSounds
List a project's named-sound library — the sounds uploaded in the dashboard's Agent section or via addProjectSound.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id), e.g. |
addProjectSound
Upload a local audio file into a project's named-sound library.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id), e.g. |
name | string | yes | The play-by-name key, e.g. |
filePath | string | yes | Path to a local MP3/M4A/AAC/WAV file under 1 MB (relative to the working directory, or absolute). |
getGatewayUsage
Read account-scoped managed-AI-gateway usage + exact cost for a project over a recent window.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id). |
days | number | no | Look-back window in days. |
Credentials
Read whether a project's API keys are set (masked hint only, never the value) and start write-without-knowing entry — the agent hands off a dashboard link where the owner pastes the secret, which never passes through the agent. Account-scoped; gated per-project by the MCP access grant. Narrative guide: /docs/mcp-server/tools.
getCredentialStatus
Read whether a project's Meta DAT build identity is set — plus a MASKED hint and when it was updated.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id). |
setCredential
Start WRITE-WITHOUT-KNOWING entry of the project's Meta DAT build identity.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id). |
credentialType | string (enum) | yes | Which credential to set. One of: meta_dat. |
Analytics
Read a project's production analytics — aggregate telemetry from shipped App Store / Play Store apps (events, active installs, by event/day/vendor/platform) over a window. Metadata only, never content. Account-scoped + ownership-checked; gated per-project by the MCP access grant. Narrative guide: /docs/mcp-server/tools.
getProjectAnalytics
Read a project's PRODUCTION analytics — aggregate telemetry from the app's shipped (App Store / Play Store) installs over a recent window.
| Parameter | Type | Required | Description |
|---|---|---|---|
appPackage | string | yes | The app's package / bundle id (project_install_id). |
days | number | no | Look-back window in days. |
Implementation guidance
Derive permissions and pressure-test voice phrases before wiring them in. Narrative guide: /docs/mcp-server/tools/guidance.
getVoiceCommandGuidance
Analyze proposed voice phrases for UX issues (length, homophones, digit-usage, Meta wake-word collision, ambiguity with existing phrases) before wiring them into a wake trigger.
| Parameter | Type | Required | Description |
|---|---|---|---|
phrases | string[] | yes | The proposed wake / trigger phrases to analyze (at least one). |
existingPhrases | string[] | no | Phrases already wired into the app. |
getPermissions
Derive the Android permissions, iOS Info.plist keys, and Meta DAT scopes a list of SDK capabilities requires.
| Parameter | Type | Required | Description |
|---|---|---|---|
capabilities | string[] | yes | Capability names from getPlatformInfo.features[].name. |
platform | string (enum) | yes | Selects how the one-line summary is phrased (Android vs iOS framing). One of: android, ios. |
Validation
Read the current integration state and gate correctness before you test. Narrative guide: /docs/mcp-server/tools/validation.
inspectIntegration
Read-only snapshot of the current Extentos integration at a project path.
| Parameter | Type | Required | Description |
|---|---|---|---|
projectPath | string | no | Absolute (or cwd-relative) path to the project root holding extentos.manifest.json. |
validateIntegration
Pre-test gate that runs ~15 severity-tiered checks over the whole project — manifest present + parses, generated-file markers + hashes, dependency declared, library-version freshness, Android Meta-DAT repo declaration, bootstrap wiring (Ex…
| Parameter | Type | Required | Description |
|---|---|---|---|
projectPath | string | no | Absolute (or cwd-relative) path to the project root. |
Simulation & agent-driven testing
Drive the browser simulator and close the agent-driven end-to-end loop — inject input, read the event log and display tree, assert tool calls — without a human or physical hardware. Narrative guide: /docs/mcp-server/tools/simulation.
createSimulatorSession
Get-or-create: provision a browser-based simulator session for this project, OR return the existing saved one.
| Parameter | Type | Required | Description |
|---|---|---|---|
glasses | string (enum) | yes | Target glasses vendor. One of: meta, android_xr, brilliant, meta_rayban. |
platform | string (enum) | no | One of: android, ios. |
projectPath | string | no | — |
recordBinary | boolean | no | — |
autoOpenBrowser | boolean | no | — |
autoLink | boolean | no | When true (default), if the backend returns auth_required, this handler polls completeAuthLink internally for autoLinkSeconds and re-mints on success. |
autoLinkSeconds | number | no | Max seconds to wait for the user to approve the verification URL when autoLink is true. |
ensureSimulatorBrowser
Ensure a connected simulator browser tab for the session — opens one if needed and confirms the browser's WebSocket actually attached before returning.
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
timeoutMs | number | no | Max ms to wait for the browser tab's WebSocket to attach after auto-opening. |
autoOpen | boolean | no | When true (default): if no browser is connected the MCP server opens the session URL in the developer's default browser and polls until it attaches. |
completeAuthLink
Poll the backend until the user finishes signup at the verificationUrl, then persist the bearer token to ~/.extentos/auth.json.
| Parameter | Type | Required | Description |
|---|---|---|---|
deviceCode | string | yes | The deviceCode from createSimulatorSession's auth_required (HTTP 402) response — the opaque handle the backend correlates the developer's signup with. |
maxWaitSeconds | integer | no | Max seconds to keep polling for the developer to finish signup. |
pollIntervalSeconds | integer | no | Seconds between backend polls. |
getEventLog
Fetch structured event trace inside a simulator session.
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
filter | string (enum) | no | Chip to filter the event stream by. One of: all, errors, voice, camera, display, lifecycle, custom. |
limit | integer | no | — |
cursor | string | no | Opaque keyset cursor from a prior getEventLog response. |
follow | boolean | no | When true, turns this call into a live watch: if no events are newer than cursor, the call BLOCKS until something happens (or timeoutMs elapses) instead of… |
timeoutMs | integer | no | Max time (ms) a follow call blocks waiting for new events. |
responseFormat | string (enum) | no | One of: concise, detailed. |
redactBinary | boolean | no | — |
collapseRepeats | boolean | no | When true (default), consecutive payload-identical events (same layer/severity/type/message/details — timestamps excluded) collapse into ONE entry carrying `re… |
injectTranscript
Inject a synthetic STT transcript into a live simulator session, the same way the simulator browser tab's click-to-fire chips do.
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
text | string | yes | The transcript text to inject. |
isFinal | boolean | no | When true (default for type=stt_transcript), the frame is a final transcript that fires onPhrase matchers. |
partial | boolean | no | Convenience: when true, send as stt_partial (a not-yet-final transcript). |
confidence | number | no | STT confidence value (0-1). |
injectAssistantUtterance
Phase 4: drive an assistant turn from outside the live session for agent-driven E2E tests of `glasses.assistant.start { tool(...) { ...
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
text | string | no | Utterance text the agent wants the assistant to react to. |
audioWavBase64 | string | no | DEFERRED to v1.1 — passing this field returns not_implemented. |
assertToolCalled
Phase 4: wait for an assistant.tool_called event matching name (and optionally argsMatch partial-match).
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
name | string | yes | Exact tool name to wait for. |
sinceCursor | string | no | Strongly recommended.: The watchCursor value returned by the injectAssistantUtterance that should trigger this tool. |
argsMatch | object | no | Optional top-level partial match against the tool's parsed args. |
timeoutMs | number | no | Max time to wait, in ms. |
getSimulatorStatus
Read a live simulator session's current state — phase (active/paused/closed), hardware-ready, attached roles (app + browser), active capability streams (which videoFrames/audioChunks/transcriptions subscriptions are open right now),…
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
deleteSimulatorSession
Retire a simulator session on purpose.
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
setSimVideo
Pipe a Test Video into the simulator's camera input so capture_photo / capture_video / videoFrames run against a known scene instead of a blank viewport — closes the agent-driven test loop for camera-driven flows the same way injectTranscr…
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
videoId | string | yes | ID of the test video to load. |
setSimDevice
Switch the simulated glasses DEVICE MODEL — one of the eight simulatable models: rayban_meta, oakley_meta_hstn, oakley_meta_vanguard, rayban_meta_optics, meta_glasses (camera + audio, NO display), rayban_display (display + Neural Band), an…
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
device | string (enum) | yes | The device model to simulate. One of: rayban_meta, oakley_meta_hstn, oakley_meta_vanguard, rayban_meta_optics, meta_glasses, rayban_display, android_xr_audio_glasses, android_xr_display_glasses, brilliant_halo, brilliant_frame. |
getDisplayState
Read what's currently rendered on the simulated glasses DISPLAY (track 5 — the native glasses.display.* capability).
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
injectInput
Drive the simulated glasses' DISPLAY input — the agent's way to "click" the display with no human and no hardware (track 5).
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
action | string (enum) | yes | 'select' fires the target (or focused) node's onClick — pass targetId (from getDisplayState.interactiveIds) for a deterministic click. One of: navigate, select, back. |
targetId | string | no | The node id to target. |
injectHardwareButton
Press the simulated glasses' hardware CAPTURE BUTTON (the right-temple controls) — the agent's way to exercise the wearer's hardware privacy gestures with no human and no hardware.
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId | string | yes | — |
button | string (enum) | no | Which hardware button to press. One of: capture. |
action | string (enum) | no | The press gesture. One of: tap, hold. |
Production
Ship-readiness checklist and credential setup (Meta DAT + any BYOK providers). Narrative guide: /docs/mcp-server/tools/production.
getProductionChecklist
Return a personalized production-readiness checklist based on the SDK capabilities the app uses, the handlers it declares, and the BYOK services it integrates.
| Parameter | Type | Required | Description |
|---|---|---|---|
capabilities | string[] | yes | Capability names from getPlatformInfo.features[].name. |
handlers | string[] | no | Names of the handlers the app declares (labels the API Keys step's affectedHandlers). |
services | string[] | no | BYOK provider names the app actually integrates (e.g. |
platform | string (enum) | yes | One of: android, ios. |
projectPath | string | no | Absolute (or cwd-relative) path to the project root. |
getCredentialGuide
Return step-by-step credential setup for the Meta DAT build identity plus each BYOK AI provider the app integrates.
| Parameter | Type | Required | Description |
|---|---|---|---|
services | string[] (enum) | no | BYOK providers the app integrates — one per integration. One of: anthropic, openai, google_cloud_vision, google_translate, google_gemini, deepl, azure_cognitive, aws_bedrock, huggingface, custom. |
handlers | string[] | no | Optional list of handler names. |
platform | string (enum) | yes | One of: android, ios. |
Documentation
Search the in-session conceptual docs corpus by topic or keyword. Narrative guide: /docs/mcp-server/tools/search.
searchDocs
Search Extentos documentation by topic or keyword.
| Parameter | Type | Required | Description |
|---|---|---|---|
query | string | no | Optional keyword(s). |
topic | string | no | Optional topic id. |
mode | string (enum) | no | Response shape. One of: full, snippets. |
Keeping this page honest
The tool list and parameters above are generated from toolDefinitions in mcp-server/src/tools/definitions.ts. The generator (scripts/generate-docs-reference.mjs) asserts that every shipped tool is categorized, so a new tool can't be added without appearing here. Counts are derived, never hand-typed.
Related
MCP server
The Extentos MCP server (`@extentos/mcp-server`) is an npm package an AI agent (Claude Code, Cursor, Windsurf, Cline) installs once and then uses to add Meta Ray-Ban smart-glasses capabilities to a native iOS or Android app. It exposes a tight set of deterministic tools across 10 categories — discovery, generation, agent configuration, credentials, analytics, guidance, validation, simulation, production-readiness, and documentation — plus a CLI for account linking, telemetry consent, and update checks. This is the agent's operating manual.
Tools reference
The Extentos MCP server exposes a tight set of deterministic tools across deterministic categories — discovery + SDK reference, generation, guidance, validation, simulation, production, agent configuration, and search. Per-tool reference covering input parameters, response shapes, when to call each tool, and worked examples. Verified from the actual tool definitions and handler implementations in @extentos/mcp-server.
Quickstart with an AI agent
Install the Extentos MCP server and let your AI agent scaffold Meta Ray-Ban smart-glasses capabilities into a native iOS or Android app. Free to start.
Reference
Generated API reference — the Android SDK (Dokka), iOS SDK (DocC), MCP server tools (JSON Schema), and a flat error-code index for troubleshooting.
Reference
Generated API reference — the Android SDK (Dokka), iOS SDK (DocC), MCP server tools (JSON Schema), and a flat error-code index for troubleshooting.
Error reference
Every typed error the Extentos SDK can return — ConnectError, CaptureError, AudioError, TransportError, the ExtentosError umbrella, and the Meta-DAT DeviceSessionError — with their payload fields and meaning. Lifecycle operations return ExtentosResult<T, E> with these concrete failure variants rather than throwing; pattern-match them. Generated from the Rust core.