The capture button
The touch surface on the Meta Ray-Ban's right temple can pause or stop your camera stream at any moment — a firmware-enforced privacy gesture your app must respect. What your app observes, how to handle it, and how to test it in the simulator.
The right temple of Meta's smart glasses carries a capacitive touch surface — the capture button. It isn't a button your app owns: it's the wearer's control over the camera, enforced by firmware below anything the SDK (or Meta's DAT itself) can override. Every camera-using app inherits its behavior, so this page is worth ten minutes before you ship one.
What the gestures do
Two gestures, hardware-verified:
| Gesture | Effect on your camera stream |
|---|---|
| Tap (single touch) | Pause ⇄ resume. Non-destructive — the connection holds. |
| Tap-and-hold | Stop. The whole device session ends; the connection drops and the SDK auto-recovers over a few seconds. |
The privacy LED on the glasses tracks the same state: lit while the camera is live, dark the moment it's paused or stopped — so bystanders always see the truth, whatever the app wants.
Why it works this way
This is a privacy mechanism, not an API gap. Meta's model is that the person wearing the camera — not the app — has the final say over whether it's capturing. That's why:
- There is no app-callable pause or resume. DAT exposes the state (
StreamState.PAUSED) but no way to change it. We probed the internal resume path on real hardware: the glasses refuse an app-initiated resume of a wearer's pause — the firmware ends the session rather than let software un-pause the camera. - Fighting it makes things worse. Tearing down and reopening the stream to force a resume drops the whole connection, and doing so shortly after a pause can wedge the glasses for ~20 seconds. Don't.
- The gesture always wins. Your app's job is to observe and respond gracefully, and the SDK is built around exactly that.
What your app observes
The SDK turns the gesture into typed, actionable signals — the same ones in the simulator and on hardware, because it's the same shared gate:
While paused (tap):
capturePhoto()/captureVideo()returnCaptureError.StreamPaused, whose message already tells the user what to do: "The camera is paused. Tap the right temple of your glasses to resume the camera, then try again."- Starting
videoFramesthrowsCameraStreamPausedException. A pause mid-stream is not an error — frame delivery halts (your last frame stays on screen) and resumes on the next tap. - An in-flight recording stays alive but captures no footage — end it and the clip contains everything except the paused window, spliced seamlessly.
duration_msreflects footage, not wall clock. - Assistant tools using
orToolErrorautomatically speak the tap-the-temple message to the user.
After a stop (hold):
- The stream is gone and no wearer gesture brings it back for your app — a tap with no live stream goes to Meta's own capture and your app never sees it.
- Your app re-arms the stream with its next camera use: the next frame-grab photo, recording, or
videoFramessubscription starts a fresh stream (on hardware this rides the SDK's automatic session recovery, ~3–5 s after the hold). - An in-flight recording ends with the footage captured so far.
The canonical handling pattern is two lines of intent: surface the message, let the wearer tap, retry. Snippets live in photo capture and video capture.
Test it without glasses
The browser simulator reproduces the whole model, because the paused gate is the same code on both substrates:
- The Capture button panel (right rail) shows the glasses with a marker on the right temple — tap it to pause/resume, press-and-hold to stop. The capture LED in the Glasses View header mirrors the privacy light.
- Every transition lands in the event log under the camera chip (
camera_stream_opened/_paused/_resumed/_closed), and a capture denied while paused logscapture_deniedunder errors with the same actionable message. - Agents drive it headless with the
injectHardwareButtonMCP tool — pause the stream mid-test and assert your app's handling end-to-end.
One surfaced substrate delta: on real glasses a hold also drops the connection for a few seconds while the SDK recovers; the simulator closes the stream without the blip.
Related
- Capture a photo —
CaptureError.StreamPausedhandling with the fullwhenarm. - Capture video — mid-recording pause semantics and the footage-splice behavior.
- Transport vs app simulation — why the simulator's error is byte-for-byte the hardware error.
- Error reference — every
CaptureErrorvariant.
Related
Capture a photo
Take a still photo on the Meta Ray-Ban camera from your handler code, handle CaptureError cleanly, and pipe the image into a vision LLM. Uses glasses.camera.capturePhoto() and the Photos URI helpers.
Capture video
Record a bounded video clip from the Meta Ray-Ban camera with glasses.camera.captureVideo(), stop it cleanly with stopVideo(), or subscribe to the continuous videoFrames stream for on-device ML and live preview.
Transport vs app simulation
Meta's Mock Device Kit simulates the transport layer; Extentos simulates the app layer — voice, photo capture, and the wearing experience. Both matter.
Error reference
Every typed error the Extentos SDK can return — ConnectError, CaptureError, AudioError, TransportError, the ExtentosError umbrella, and the Meta-DAT DeviceSessionError — with their payload fields and meaning. Lifecycle operations return ExtentosResult<T, E> with these concrete failure variants rather than throwing; pattern-match them. Generated from the Rust core.
Capture video
Record a bounded video clip from the Meta Ray-Ban camera with glasses.camera.captureVideo(), stop it cleanly with stopVideo(), or subscribe to the continuous videoFrames stream for on-device ML and live preview.
Audio streaming
Stream raw mic audio off Meta Ray-Ban with audioChunks, get continuous STT with transcriptions(), and play TTS with speak(). Covers A2DP/HFP coexistence.