The capture button
The touch surface on the Meta Ray-Ban's right temple can pause or stop your camera stream at any moment — a firmware-enforced privacy gesture your app must respect. What your app observes, how to handle it, and how to test it in the simulator.
Camera needs the Meta vendor module. com.extentos:glasses carries no
vendor SDK, so add implementation("com.extentos:glasses-meta") alongside it —
see install. Without it your build still succeeds
and voice still works, but capabilities.camera is false and every capture
returns an error. The SDK logs a warning at startup when it spots that
combination.
The right temple of Meta's smart glasses carries a capacitive touch surface — the capture button. It isn't a button your app owns: it's the wearer's control over the camera, enforced by firmware below anything the SDK (or Meta's DAT itself) can override. Every camera-using app inherits its behavior, so this page is worth ten minutes before you ship one.
What the gestures do
Two gestures, hardware-verified:
| Gesture | Effect on your camera stream |
|---|---|
| Tap (single touch) | Pause ⇄ resume. Non-destructive — the connection holds. |
| Tap-and-hold | Stop. The whole device session ends; the connection drops and the SDK auto-recovers over a few seconds. |
The privacy LED on the glasses tracks the same state: lit while the camera is live, dark the moment it's paused or stopped — so bystanders always see the truth, whatever the app wants.
Why it works this way
This is a privacy mechanism, not an API gap. Meta's model is that the person wearing the camera — not the app — has the final say over whether it's capturing. That's why:
- There is no app-callable pause or resume. DAT exposes the state (
StreamState.PAUSED) but no way to change it. We probed the internal resume path on real hardware: the glasses refuse an app-initiated resume of a wearer's pause — the firmware ends the session rather than let software un-pause the camera. - Fighting it makes things worse. Tearing down and reopening the stream to force a resume drops the whole connection, and doing so shortly after a pause can wedge the glasses for ~20 seconds. Don't.
- The gesture always wins. Your app's job is to observe and respond gracefully, and the SDK is built around exactly that.
What your app observes
The SDK turns the gesture into typed, actionable signals — the same ones in the simulator and on hardware, because it's the same shared gate:
While paused (tap):
capturePhoto()/captureVideo()returnCaptureError.StreamPaused, whose message already tells the user what to do: "The camera is paused. Tap the right temple of your glasses to resume the camera, then try again."- Starting
videoFramesthrowsCameraStreamPausedException. A pause mid-stream is not an error — frame delivery halts (your last frame stays on screen) and resumes on the next tap. - An in-flight recording stays alive but captures no footage — end it and the clip contains everything except the paused window, spliced seamlessly.
duration_msreflects footage, not wall clock. - Assistant tools using
orToolErrorautomatically speak the tap-the-temple message to the user.
After a stop (hold):
- The stream is gone and no wearer gesture brings it back for your app — a tap with no live stream goes to Meta's own capture and your app never sees it.
- Your app re-arms the stream with its next camera use: the next frame-grab photo, recording, or
videoFramessubscription starts a fresh stream (on hardware this rides the SDK's automatic session recovery, ~3–5 s after the hold). - An in-flight recording ends with the footage captured so far.
The canonical handling pattern is two lines of intent: surface the message, let the wearer tap, retry. Snippets live in photo capture and video capture.
Watching the stream itself
The signals above all arrive at a call you make. If your app forwards frames somewhere else, a live relay, a recorder, an upload pipeline, there may be no call in flight when the wearer taps: frames simply stop arriving and nothing tells you why.
Subscribe to glasses.camera.streamState() for that. It reports what the stream is doing, independently of any call:
glasses.camera.streamState().collect { phase ->
when (phase) {
CameraStreamState.STREAMING -> relay.resume()
CameraStreamState.PAUSED -> relay.hold("Tap the right temple to resume")
CameraStreamState.IDLE -> relay.stop()
else -> Unit
}
}for await phase in glasses.camera.streamState() {
switch phase {
case .streaming: relay.resume()
case .paused: relay.hold("Tap the right temple to resume")
case .idle: relay.stop()
default: break
}
}Five phases: IDLE, ARMING, STREAMING, PAUSED, STOPPING. You get the current one immediately on subscribing, then only genuine transitions, so every value is a change worth acting on.
Do not watch the connection state for this. It does not move when the wearer taps, and that is deliberate: Active is a supertype that internal camera work stays inside, so tearing your pipeline down on any non-Active state would kill it during normal arming. A paused stream is a perfectly healthy connection that has stopped producing frames, and streamState() is the only place that distinction is visible.
The same subscription covers the stop gesture and the five-minute retirement below: both land on STOPPING then IDLE, and your next camera use re-arms through ARMING back to STREAMING.
Streams end on their own, too
The gestures aren't the only firmware authority over your stream — the glasses manage stream lifetime themselves. Two behaviors, hardware-measured:
- A long-lived stream is retired at roughly five minutes. A continuously-armed stream ends the whole device session ~304 seconds after frames start flowing (measured three times: 303.9 / 304.1 / 304.2 s). Your app sees the same session end as a hold (
SESSION_ENDED_BY_DEVICE), and the handling is identical: the next camera use re-arms everything — a couple of seconds end-to-end, verified without an app restart. - Thermals ride the same window. Streaming warms the glasses on a repeatable curve; the wearer's phone shows Meta's own overheating notification around the two-minute mark. The hotter the glasses when the session ends, the rougher the exit — a pre-heated end can also drop the Bluetooth connection for a couple of minutes before the SDK's recovery brings it back.
There is no reliable app-side lever to defer or avoid this — software-initiated stream teardown is the destabilizing path described above, and the SDK deliberately leaves stream lifetime to the platform. Treat a stream that ends as a normal lifecycle event, not an error: the photo and video handling patterns already cover it.
Verify it in the simulator
The browser simulator reproduces the whole model, because the paused gate is the same code on both substrates:
- The Capture button panel (right rail) shows the glasses with a marker on the right temple — tap it to pause/resume, press-and-hold to stop. The capture LED in the Glasses View header mirrors the privacy light.
- Every transition lands in the event log under the camera chip (
camera_stream_opened/_paused/_resumed/_closed), and a capture denied while paused logscapture_deniedunder errors with the same actionable message. - Agents drive it headless with the
injectHardwareButtonMCP tool — pause the stream mid-test and assert your app's handling end-to-end.
One surfaced substrate delta: on real glasses a hold also drops the connection for a few seconds while the SDK recovers; the simulator closes the stream without the blip.
Related
- Capture a photo —
CaptureError.StreamPausedhandling with the fullwhenarm. - Capture video — mid-recording pause semantics and the footage-splice behavior.
- Transport vs app simulation — why the simulator's error is byte-for-byte the hardware error.
- Error reference — every
CaptureErrorvariant.
Capture video
Record a bounded video clip from the Meta Ray-Ban camera with glasses.camera.captureVideo(), stop it cleanly with stopVideo(), or subscribe to the continuous videoFrames stream for on-device ML and live preview.
Audio streaming
Stream raw mic audio off Meta Ray-Ban with audioChunks, get continuous STT with transcriptions(), and play TTS with speak(). Covers A2DP/HFP coexistence.