Render on the display

Display support is in beta. The full display surface — the builder DSL, select/back input, per-device gating — ships in both SDKs and is fully tested in the simulator on both platforms. On-glasses rendering on Meta Ray-Ban Display hardware is in its verification phase; APIs are stable, with refinements expected as hardware verification completes.

This guide builds a small media gallery on the Ray-Ban Display with no LLM at all: capture a photo, save it to your own store, render a browsable list of cards, and show one full-surface when the user pinches it. Then it points at the assistant-driven variant, where the same display DSL is driven by voice.

Read the display capability for the full DSL, the additive-light rule, and the input model. This page is the task walkthrough.

glasses.display.* ships in both SDKs and is fully sim-testable on both. On-glasses rendering via Meta DAT (0.8.0) is live on Android; iOS on-glasses delivery is in progress. The local-media roots this guide uses — image(photo) and forgetHostedImage — are Android-only (on iOS, use URL-based media). On a display-less device every call is a safe no-op.

Gate on isAvailable

Display is per-device, not per-vendor: Meta Ray-Ban Display has a screen and Ray-Ban Meta doesn't, and the same split exists inside Android XR. Branch on the runtime signal, never on a model name or a vendor, and fall back to voice when there's no screen:

if (!glasses.display.isAvailable) {
    glasses.audio.speak("Saved — you've got ${store.all().size} photos.")
    return
}
glasses.display.show { /* ... */ }

show() on a display-less device is a silent no-op, so the guard is about UX, not safety — give the user a voice path. The gate is live in the simulator: switching the device model flips isAvailable mid-session with no reconnect.

Capture → store → browse (no LLM)

A self-contained gallery driven by direct display calls plus a button or voice trigger. show {} replaces the entire surface each call, so own your nav state app-side (list vs detail) and re-render whole views on every transition. Each show registers its own onBack because back is view-contextual.

import com.extentos.glasses.core.Background
import com.extentos.glasses.core.ExtentosGlasses
import com.extentos.glasses.core.TextStyle
import com.extentos.glasses.core.valueOrNull
import kotlinx.coroutines.*

// 'store' is YOUR persisted gallery (Room / DataStore / files). SavedPhoto.toPhoto()
// rebuilds a Photo the SAME way for show AND forget, so both hit the same hosted copy.
class GalleryDisplay(
    private val glasses: ExtentosGlasses,
    private val store: PhotoStore,
    private val scope: CoroutineScope = CoroutineScope(SupervisorJob() + Dispatchers.Main),
) {
    // Wire this to a button onClick or a voice trigger.
    fun capture() = scope.launch {
        val photo = glasses.camera.capturePhoto().valueOrNull() ?: return@launch
        store.add(photo)
        showList()
    }

    // Root view: one card per saved photo.
    fun showList() = scope.launch {
        if (!glasses.display.isAvailable) return@launch
        val items = store.all().take(5)
        glasses.display.show(onBack = { scope.launch { glasses.display.clear() } }) {
            column(gap = 8, padding = 12) {
                text("Photos (${store.all().size})", style = TextStyle.HEADING)
                for (p in items) {
                    column(
                        onClick = { showDetail(p) },
                        id = "photo-${p.id}",        // stable id — gestures/agents target it
                        background = Background.CARD,
                    ) {
                        text(p.label, style = TextStyle.BODY)
                    }
                }
            }
        }
    }

    // Detail view: the photo full-surface; back returns to the list.
    private fun showDetail(p: SavedPhoto) = scope.launch {
        glasses.display.show(onBack = { showList() }) {
            image(p.toPhoto())   // auto-hosts the local capture, then renders the hosted URL
        }
    }

    // On delete, release the hosted display copy (built the SAME way as the show).
    fun delete(p: SavedPhoto) = scope.launch {
        store.remove(p)
        glasses.display.forgetHostedImage(p.toPhoto())
        showList()
    }
}

What's happening:

• Cards are clickable column containers with a stable id ("photo-" + p.id). Without an explicit id, a clickable container falls back to a positional box-0 / box-1 that shifts on reorder — coupling input routing to render order.
• image(photo) is root-only and full-surface. The glasses fetch only http(s) URLs, so the SDK hosts your local capture at show() time and renders the hosted URL — you never hand-roll hosting. (A photo hosts in well under a second; the video equivalent video(clip) takes a few seconds.) The local-media roots image(photo) / video(clip) are Android-only — on iOS, pass an http(s) URL instead.
• onBack is registered per show: the detail view's onBack re-renders the list; the root's clears. The back gesture is the Neural Band mid-pinch on hardware. onClick / onBack are plain () -> Unit, but show() / clear() are suspend — launch a coroutine inside the handler.
• forgetHostedImage tears down the hosted copy when the user deletes the photo. Rebuild the Photo the same way for show and forget so both resolve to the same copy. It's idempotent and doesn't touch the local file.

Add a Back button to the detail view for discoverability alongside the gesture:

button("Back", style = ButtonStyle.OUTLINE, id = "back") { showList() }

Test it in the simulator

A fresh sim session defaults to rayban_meta (no display) — set the device first, or every show is a silent no-op:

await setSimDevice({ sessionId, device: "rayban_display" });

// Read EXACTLY what's on the glasses — structured tree, no pixels, no browser needed:
const d1 = await getDisplayState({ sessionId });
//   d1.tree = the rendered DisplayNode tree; d1.interactiveIds = the pinchable card ids

// Pinch a card the way the Neural Band would (needs the browser tab — ensureSimulatorBrowser):
await injectInput({ sessionId, action: "select", targetId: d1.interactiveIds[0] });
const d2 = await getDisplayState({ sessionId });   // the detail view

// Back via the gesture — mid-pinch on hardware, injected back in the sim:
await injectInput({ sessionId, action: "back" });  // onBack fires, re-renders the list

getDisplayState reads the live hub snapshot and works headless; injectInput needs the browser tab attached (it owns the rendered tree + focus). The durable trace is getEventLog(filter: "display") — display_show, display_select, display_navigate, display_back — where event order proves causality. Test both branches: rayban_display renders; setSimDevice({ device: "rayban_meta" }) flips isAvailable to false and your voice fallback runs. See the MCP tools reference.

Dead URLs show nothing. Additive light can't render a failed media fetch — the panel stays dark while the tree checks look green. The simulator shows a "video failed to load" placeholder and a display_error on the errors chip; on hardware a display.video_error runtime event fires so you can fall back to a text view. Check the errors chip first when a display flow looks green but shows nothing. (Errors — there is no DisplayError return type.)

The same tree on Android XR

As of 2026-07-27 the display surface has a second vendor: Android XR Display Glasses. The tree you just built renders there too, translated to Google's Glimmer toolkit instead of Meta's view builder. Your handler code does not change, and neither does this guide's code — switch the simulator's device and watch the same tree appear.

await setSimDevice({ sessionId, device: "android_xr_display_glasses" });

Four things are worth designing for, because they differ between the two screens:

The panel is smaller, and trees do not scroll. Meta Ray-Ban Display is 600x600; Android XR display glasses are 450x394. An over-tall tree is clipped, not scrolled, on both. Design for the smaller surface and you fit both; design for Meta's and your bottom row can fall off on XR. The simulator draws each device's real panel, so this is visible rather than theoretical.

Icon names resolve per vendor. Meta matches its own 116-glyph set; Android XR matches a set of shared concepts (check, close, place, search, settings, person, phone, star, play, arrow_back, and about twenty more). A name neither vendor knows renders nothing rather than a placeholder, identically on both. If you want the same icon on both screens, stay inside the shared set.

isAvailable is answered by the device, after connect. On Android XR the SDK asks the connected glasses whether they have a screen and reports false until the answer arrives — it will never claim a screen it has not confirmed. Read glasses.display.isAvailable at the point you need it rather than caching it during startup.

Input is the same actions with a different gesture. Google's touchpad API is three callbacks — swipe forward, swipe backward, click — which are focus-next, focus-previous and select, the same three Meta's Neural Band delivers as thumb-slide either way and index-pinch. Your onClick handlers are unchanged. The one real difference is back: Meta has a back gesture (mid-pinch) that reaches onBack, and Google's touchpad API has no back callback at all, so on that vendor back can only come through the system back path. Give every screen a way out the wearer can select.

One limit stated plainly: no Android XR glasses of either class are on sale, so nothing on that vendor is hardware-verified, and a video node inside a tree plays through the Android framework there but has not been proven on a device. See the Android XR vendor page for the full status.

The same tree on Brilliant Labs

A third vendor renders the same tree: Brilliant Halo and Frame. This one is worth designing against even before you can ship for it, because it breaks assumptions the other two let you keep — and all of them are visible in the simulator today.

await setSimDevice({ sessionId, device: "brilliant_halo" });

The panel is round. Halo's canvas is 256×256 and circular. The SDK lays your content out inside the inscribed square automatically, so a full-width row does not lose its ends — you write the same tree. What it cannot do is make your tree shorter, and at 256×256 this is the smallest surface in the catalog: a layout that fits Meta's 600×600 shows about a third of its content. The simulator draws the real circle and marks what got clipped.

There is one font size. Neither Brilliant device's text primitive takes a size argument, so HEADING, BODY and CAPTION render identically. Colour still differentiates — secondary text carries its own colour — but a hierarchy built on type size flattens here.

The wearer cannot move focus. Halo has a button and an IMU tap and no swipe; Frame has one tap gesture and nothing else. This is the first vendor where the four display actions do not all have a gesture, so build single-action screens rather than browsable lists. The simulator disables the controls the hardware lacks, so a flow you can drive there is one the device could perform.

Frame has a display and no speaker. If your display path falls back to audio.speak, that fallback is silence on Frame. Gate on the speaker capability, not on the display's absence.

Brilliant's transport ships inside the SDK on both platforms (nothing to add on Android; TransportChoice.brilliant on iOS), but no hardware has run it — see the Brilliant vendor page for the full status. The simulator is the part you can rely on today.

The assistant-driven variant

The same display DSL can be driven by voice instead of buttons: an assistant tool calls glasses.display.show(...), and "show my notes" / "open the groceries note" / "go back" drive the same app-side state machine the Neural Band does. One state machine, two input rails, no conflict — gate the tool on isAvailable and return the reason in ToolResult.Err so the model explains it ("These glasses have no display — want me to read it aloud?").

tool(
    "show_notes",
    "Show the browsable list of the user's notes on the glasses display. " +
        "Also call when the user asks to go back to the list from an open note.",
) {
    if (!glasses.display.isAvailable) {
        return@tool ToolResult.Err("These glasses have no display. Offer to read the notes aloud instead.")
    }
    showBrowse()
    ToolResult.Ok("Notes are on the display — slide between them, pinch one to open it.")
}

For the full two-view browse ⇄ detail pattern with both rails wired, see the assistant runtime and getCodeExample(pattern: "display_browse_detail") via the MCP server.

Related

• The display capability — the full DSL, additive-light rule, hosting, and input model
• The assistant runtime — drive the display by voice from assistant tools
• Capabilities — the per-device capability model and the full SDK vocabulary
• Error reference — the no-DisplayError model and the display runtime events
• Vendors: Meta Ray-Ban — the device family and the Ray-Ban Display
• Vendors: Brilliant Labs — the round panel, the single font size, and the display-without-a-speaker case

Related