Simulation tools

These tools orchestrate the browser-simulator dev loop — plus the agent-driven testing tools (below) that close the end-to-end loop without a human. The simulator hosts your customer-built app running against a Meta-DAT-shaped transport stub — glasses.audio.transcriptions(), glasses.camera.capturePhoto(), glasses.audio.speak() and every other capability primitive flow through it exactly as on real hardware. The full parameter set for every tool is the generated MCP tools reference.

createSimulatorSession

Get-or-create a browser-based simulator session for the current project. Persistent-simulations model: each (account, project, platform) has at most one saved sim. Calling this returns that sim's existing URL (status: "resumed") instead of minting a new one.

Parameters: glasses: "meta_rayban" (required), platform: "android" | "ios", projectPath, recordBinary, autoOpenBrowser, autoLink (default true; auto-poll the device-code flow if auth_required returns), autoLinkSeconds (default 30).

Response: { status, sessionId, sessionUrl, transportEndpoint, expiresAt, autoBind: "attached" | "no_device" | "error", capabilities, developerInstructions, summary }. Auto-bind tries to attach the running app via the local MCP bridge; falls back to a URL-bake snippet (Android buildConfigField patch / iOS extentos.session.plist write) when the bridge can't reach.

Iteration model — do NOT call this per change: app-code edits rebuild + reinstall and reattach automatically. The simulator URL is stable. Calling this again on the same project just returns the same sim — cheap, idempotent.

Sims do not expire. There is no 24-hour cap, no idle timeout and no cleanup sweep — yesterday's sim is still there this morning, and a brand-new agent session in the same project resumes it. The expiresAt field in the response is a far-future sentinel kept for wire compatibility; don't branch on it.

deleteSimulatorSession

Retire a session on purpose. This is the first half of a rotation: delete, then mint. There is deliberately no one-call force-fresh, because a silent replace is what produced sim churn — discarded sessions leave device registrations that outlive them, so the next mint could auto-bind to an app that was already gone and report it as attached.

Parameters: sessionId (required).

Use it only when you genuinely need a different session identity — the URL leaked, or you want a deliberate clean-slate id. Not to "start over" when something looks wrong: deleting disconnects any attached app and browser, and a URL-baked app needs a rebuild against the new URL. A clean event log doesn't need a new sim either — getEventLog is filterable.

Idempotent: deleting a session that doesn't exist returns alreadyGone: true rather than an error.

ensureSimulatorBrowser

Guarantee a connected simulator browser tab — open one if needed and confirm its WebSocket actually attached before returning. The browser tab is the hardware surrogate's viewport: setSimVideo returns browser_not_attached without it, and the camera flows the assistant runtime drives fail. Idempotent — a no-op if a browser is already connected, so it's safe to call before every camera flow.

When to call: right after createSimulatorSession before a camera-driven flow; after a backend deploy (which severs every sim WebSocket — the tab does not auto-reconnect); whenever the tab was closed; or any time the browser role shows as detached.

Parameters: sessionId (required), timeoutMs (default 20000, clamped [5000, 60000]), autoOpen (default true).

Response: { alreadyOpen, opened, browserConnected, browserClientId, appConnected, sessionUrl, waitedMs, summary }. On an auto-open that doesn't attach in time, errors with browser_not_connected (carrying sessionUrl).

It reads the authoritative in-memory hub liveness — the same signal setSimVideo/inject enforce, so it can't disagree with them (unlike getSimulatorStatus.connectedRoles, which is persisted state that can lag a dead socket by ~25–50s). When a tab is needed it opens the session URL in your default browser cross-platform, from the MCP server itself (no shell command for you to run), then polls until the WebSocket attaches. Headless / remote agents: pass autoOpen: false to get the sessionUrl back with no spawn and no wait — surface it to the developer to open on a machine with a display, then re-run with autoOpen: true to confirm.

completeAuthLink

Poll the backend until the developer finishes signup at the verification URL, then persist the bearer token to ~/.extentos/auth.json. Use only when createSimulatorSession returns status: "auth_required" AND you opted out of autoLink (or the inline poll timed out).

Parameters: deviceCode (from the prior createSimulatorSession response), maxWaitSeconds, pollIntervalSeconds.

getEventLog

Fetch structured event trace inside a simulator session. The primary debugging tool — answers "why didn't my handler see what I expected?"

Parameters: sessionId (required), filter (all / errors / voice / camera / display / ai / lifecycle / custom), limit, cursor (opaque keyset cursor — pass it back to fetch only newer events), follow (block until new events land — live-tail), timeoutMs, responseFormat (concise / detailed), redactBinary (default true — large data: URIs and >2KB strings collapse to a [redacted: N chars] marker so a single capture_photo response doesn't blow the agent's context).

Response: { events: EventRecord[], sessionStatus, totalEvents, hasMore, summary }. Each event carries layer + severity + type + message + optional details. Consecutive payload-identical events collapse into one entry with repeat: N + lastTimestamp (timestamps excluded from the identity); totalEvents counts the raw events, and collapseRepeats: false returns the uncollapsed rows.

Post-2026-05 redesign uses a 7-chip taxonomy. One label per event: severity≥warn absorbs into errors regardless of modality (so camera_error lives in errors, not camera); then prefix-match into voice / camera / display / ai / lifecycle; then custom for app-emitted opens. Pre-pivot filter values (triggers / blocks / callbacks / spec) AND the intermediate layer values (transport / audio / speak / toggles / streams / system) are retired. See searchDocs("event_log_schema") for the complete rules + event-type catalog per chip.

getSimulatorStatus

Read a live simulator session's current state — phase (active / paused / closed), hardware-ready, attached roles (app + browser), active capability streams (which subscriptions are open right now), the testVideos available to drive setSimVideo, and a per-role freshness advisory that flags a connection gone stale after a rebuild or backend redeploy.

Parameters: sessionId (required).

Response: { sessionId, sessionState, hardwareReady, connectionState, connectedRoles, activeStreams[], expiresAt, recording, summary }.

Use during testing to confirm a stream is open, a toggle is set, the session is healthy, or the app role has actually attached. Distinct from getEventLog (event trace) — this is the snapshot view.

Agent-driven testing tools

Eight more tools close the end-to-end test loop without a human or hardware — they drive input and read back what happened. Full parameters are in the MCP tools reference; in brief:

• injectTranscript — inject a synthetic STT transcript to drive a wake phrase or a glasses.voice.onPhrase matcher.
• injectAssistantUtterance — drive a Phase-4 assistant turn (Mock or real OpenAI); returns a watchCursor.
• assertToolCalled — wait for an assistant.tool_called event (pass the watchCursor from the inject so the wait anchors before it).
• setSimVideo — pipe a test video into the simulated camera so capturePhoto / videoFrames run against a known scene.
• setSimDevice — switch the simulated device model (e.g. rayban_display to exercise glasses.display.isAvailable).
• getDisplayState — read the display tree currently rendered plus its selectable node ids.
• injectInput — drive display input (select / navigate / back) to fire a display node's onClick.
• injectHardwareButton — press the hardware capture button (the right temple): tap = pause/resume the live camera stream, hold = stop it, so paused/stopped-camera handling (CaptureError.StreamPaused, re-arm after a stop) is testable end-to-end. Mirrors the sim page's Hardware-buttons panel and its capture LED.

Together with getEventLog(follow: true) these are the agent-driven E2E loop — see the voice-assistant guide and getCodeExample('agent_driven_e2e_full_loop').

Related

• Transport vs app simulation — what the simulator actually simulates and why
• searchDocs topic: simulator_browser_mode — deep dive on the iteration model + persistent-sim semantics
• searchDocs topic: simulator_session_lifecycle — mint once, reuse, delete deliberately
• searchDocs topic: event_log_schema — every event layer + type the log carries
• Tools overview — back to the full tool catalog

Related