---
title: Audio streaming
description: Stream raw mic audio off Meta Ray-Ban with audioChunks, get continuous STT with transcriptions(), and play TTS with speak(). Covers A2DP/HFP coexistence.
type: guide
platform: all
vendor: meta
related:
  - /docs/guides/voice-triggers
  - /docs/guides/photo-capture
  - /docs/concepts/capabilities
  - /docs/reference/errors
---

The audio sub-client has three streaming surfaces and one output surface. **In:** `glasses.audio.audioChunks` (raw PCM), `glasses.audio.transcriptions()` (continuous speech-to-text), and `glasses.audio.recordDiscrete(...)` (a silence-bounded clip). **Out:** `glasses.audio.speak(text)` (text-to-speech), with `cancelSpeak()` for barge-in. This page covers each and the one Bluetooth-audio gotcha that bites everyone.

> Kotlin leads the snippets and is the source of truth. iOS ships every surface on this page (`for await chunk in ...`, `case .success`) — and additionally exposes `playSound` / `registerSound` / `soundNames` on its audio client.

## Speak (text-to-speech)

```kotlin
glasses.audio.speak("Got it.")  // suspends until the TTS engine finishes by default
```

`speak()` synthesizes with the phone's platform TTS engine (Android `TextToSpeech` / iOS `AVSpeechSynthesizer`) and routes the audio to the glasses speaker. It returns `ExtentosResult<Unit, AudioError>` and **blocks the calling coroutine until the engine finishes** by default — to listen and speak at the same time, launch it in its own scope.

### A better voice, with no assistant involved

The platform engine is fine for short confirmations and robotic on longer text. If you've added the on-device voice module, name a voice in `SpeakConfig` and the same call speaks in a natural one:

```kotlin
glasses.audio.speak(articleSummary, SpeakConfig(voice = "kokoro-emma"))
```

```swift
_ = await glasses.audio.speak(articleSummary, config: SpeakConfig(voice: "kokoro-emma"))
```

No assistant session is required, and the text can come from anywhere — a cloud model's reply, your own content, a notification. It works on **every transport including the vendorless audio baseline**, so a voice-only app with no glasses paired gets the same voice through the phone or earbuds. Setup and the eleven speakers: [on-device models](/docs/concepts/local-models#on-device-voices).

Routing is per call and never errors: an unknown voice id, or a model that hasn't finished downloading, serves the platform voice for that utterance instead. `cancelSpeak()` interrupts a local synthesis exactly as it does platform TTS. `rate` and `pitch` apply to the platform engine only — the local voice speaks at its model's own prosody.

To interrupt mid-utterance (barge-in), call `cancelSpeak()` from another coroutine:

```kotlin
val job = launch { glasses.audio.speak(longText) }
// ...on detected user speech:
glasses.audio.cancelSpeak()  // fire-and-forget; the speaker falls silent within ~tens of ms
job.cancel()
```

The full race-and-cancel barge-in composition is `getCodeExample(pattern: "barge_in_speak")`.

> **`speak()` and `earcon()` are never gated by a toggle** — audio *output* always plays. No toggle, not even `privacy_mode`, silences output (the toggles gate audio/camera *input* + STT). A `DisabledByUser` result is therefore impossible for `speak`/`earcon`; build your own mute switch if you need one.

> **Speak quality is bound by the phone's TTS engine, not the glasses.** For a premium voice, either use the Phase-4 [assistant runtime](/docs/concepts/assistant) (cloud TTS over the [managed gateway](/docs/concepts/ai-gateway)) or wait for the `outgoing_audio_stream` playback primitive (future).

### A2DP vs HFP coexistence

`speak()` routes audio over the **Bluetooth HFP profile** (voice-band — the link negotiates wideband speech, mSBC at 16 kHz mono), the same profile the glasses use as a microphone — **not** A2DP. The consequence: while `speak()` is active, high-quality A2DP music playback drops to voice-band HFP mono for the duration of the speech. The same is true for `transcriptions()`, `recordDiscrete()`, `audioChunks`, and `captureVideo(VideoConfig(includeAudio = true))` — any time the glasses act as a mic, you are on HFP and A2DP is suspended.

This is a hardware property of Meta DAT, not an Extentos choice. Surface a "listening…" / "speaking…" indicator so the user understands why their music quality dipped, and avoid overlapping a music-playback feature with continuous capture. Coexistence is a real-hardware behavior — the browser simulator has no A2DP music path, so verify this on actual glasses rather than expecting the sim to reproduce it.

## Continuous transcription

`transcriptions()` is a `Flow<Transcript>` of partial + final speech-to-text — the primitive behind live captions and string-matched wake phrases:

```kotlin
import com.extentos.glasses.core.Transcript
import com.extentos.glasses.core.TranscriptionConfig

glasses.audio.transcriptions(TranscriptionConfig(language = "en-US")).collect { t ->
    // Transcript is a sealed class: Partial | Final. Partial fires 5–20x/sec.
    if (t !is Transcript.Final) return@collect
    handleFinalText(t.text)
}
```

Three things that trip people up:

- **Always pattern-match `is Transcript.Final` before reading `t.text`.** `Partial` fires 5–20 times a second and partial text can match a wake phrase mid-utterance, causing double-fires.
- **The library does not normalize.** `t.text` is exactly what the recognizer returned, in the casing the user spoke. Match against `t.text.lowercase()` (and strip punctuation/apostrophes) yourself.
- **Subscribe once.** Each subscription is a separate recognizer instance — subscribing per-loop or per-trigger means duplicate transcripts and battery drain.

Transcription is gated by `listening_mode` (default treated as on; `"off"` is the user's hard kill-switch), `audio_capture_enabled`, and `privacy_mode`. (`transcription_enabled` is declared in the catalog but not consumed by the library yet — don't chase it.) If transcripts stop arriving, check those first — see [Wake phrase not matching](/docs/troubleshooting/voice-trigger-not-firing). For matching a wake phrase you usually want [voice triggers](/docs/guides/voice-triggers) (`glasses.voice.onPhrase`) rather than a raw collector; drop to `transcriptions()` directly only for live-caption feeds or custom (regex / stateful) matching. The live-captions-into-Compose shape is `getCodeExample(pattern: "live_transcription_ui")`.

### Language, and where it is and isn't honored

`TranscriptionConfig(language = …)` is the only language control in the SDK, and **it does not take effect on every transport.**

**It is honored everywhere on iOS, and only on the simulator transports on Android.**

| Transport | Android | iOS |
|---|---|---|
| Real glasses | ❌ **ignored** | ✅ |
| System-audio baseline (no vendor) | ❌ **ignored** | ✅ |
| Browser simulator | ✅ | ✅ |
| Local in-process sim | ✅ | ✅ |

iOS runs Apple's `SFSpeechRecognizer` and builds it from `Locale(identifier: language)`, so every locale Apple supports works on every transport. Android runs a **bundled Vosk** recognizer on real glasses and on the audio baseline — pinned to an en-US acoustic model that never reads the field — and the platform `SpeechRecognizer` (which does read it) only on the simulator transports.

<Callout type="warn">
  **On Android hardware, a non-English `language` is accepted and then discarded.**
  Passing `language = "de-DE"` on real glasses or the audio baseline gives you
  English recognition of German speech — not an error. And because the Android
  *simulator* honors the setting, a multilingual flow can pass in sim and fail on
  hardware. iOS is unaffected. If your product depends on a non-English language,
  test it on an Android device before building on it.
</Callout>

## Raw microphone chunks

When you need raw PCM — your own on-device STT model, audio analysis, custom keyword spotting — subscribe to `audioChunks`:

```kotlin
import com.extentos.glasses.core.AudioChunkConfig

glasses.audio.audioChunks(
    AudioChunkConfig(
        chunkMillis = 20,    // 20ms is the standard opus/CELT frame — don't change unless your model needs it
        sampleRate = 16000,  // REQUESTED rate; see the caveat below
    ),
).collect { chunk ->
    // feed chunk.data (ByteArray) to your STT / VAD / classifier. Also chunk.sampleRate, chunk.timestampMs.
}
```

Use `transcriptions()` instead if you just want speech-to-text — the SDK's recognizer is cheaper and pre-routed (on Android hardware, on-device Vosk; on iOS hardware, Apple's `SFSpeechRecognizer`; the platform recognizer on the dev/sim path). `audioChunks` is for cases that genuinely need PCM.

> **`sampleRate` is a request, not a guarantee.** Meta DAT delivers mic audio over the Bluetooth HFP link, which negotiates **wideband speech (mSBC, 16 kHz mono)** — on real hardware the effective rate is the HFP rate, not necessarily your config value. For any rate-sensitive DSP (FFT bins, VAD windows), read the per-chunk `chunk.sampleRate` rather than assuming the config value. The stream buffers with drop-oldest overflow — consumers that can't keep up lose frames rather than back-pressuring the mic.

`audioChunks` is gated by `audio_capture_enabled` and `privacy_mode` (but **not** `listening_mode` — it's a raw-audio path, not STT).

## Bounded capture (record a turn)

For a one-shot "ask a question" capture, `recordDiscrete(...)` records until the user goes silent and returns the clip (`rawAudioUri`); on real hardware the `transcript` field is empty today — transcription there is your app's job (the browser sim populates it). It is covered with the voice flows: see `getCapabilityGuide(feature: "record_audio")` for the silence-VAD knobs and `getCodeExample(pattern: "voice_notes")` for a wake → record → persist composition.

## Verify it in the simulator

The browser simulator drives transcription from your real laptop microphone and plays `speak()` output through your speakers, so you can rehearse the whole audio loop before hardware. `recordDiscrete` emits a `record_audio_returned` event (`getEventLog(filter: "voice")`). The audio chips in the event log are `voice` (transcripts, recordings, speech) — note there is no `audio` filter. The simulator is designed to behave like the glasses, but the audio substrate differs (24 kHz PCM in sim vs the glasses' wideband 16 kHz HFP) and that fidelity is under active validation — confirm audio behavior on real glasses.

## Related

- [Voice triggers](/docs/guides/voice-triggers) — `glasses.voice.onPhrase`, the structured wrapper over `transcriptions()`.
- [Capture a photo](/docs/guides/photo-capture) — the vision-LLM flow that pairs capture with `speak()`.
- [Capabilities](/docs/concepts/capabilities) — the full audio capability catalog.
- [Error reference](/docs/reference/errors#audio-errors) — every `AudioError` variant.
- `getCapabilityGuide(feature: "speak")` / `getCapabilityGuide(feature: "transcription_incremental")` / `getCapabilityGuide(feature: "audio_chunks")` — call shapes + full gotchas.