PyModel/pythinker-code / .agents/skills/opentui
opentui
VerifiedBuild terminal UIs with OpenTUI. Covers core, components, audio, keymaps, React, Solid, plugins, testing, standalone executables, QR encoding, SSH, and Three.js WebGPU.
docs/core-concepts/audio.mdx
View on GitHub---
title: Native audio
description: Capture, load, stream, play, mix, and inspect audio with the native miniaudio backend
order: 11
skill:
entry: true
intents: [audio, native-audio, sound, playback, capture, microphone, streaming, radio, mp3, flac, pcm, fft]
---
# Native audio
OpenTUI exposes a native miniaudio-backed `Audio` engine from `@opentui/core`. It can capture input, decode audio bytes or files, play sounds through named groups, choose devices, mix into PCM buffers, and expose recent mixed frames for visualization.
## Basic usage
```typescript
import { Audio } from "@opentui/core"
const audio = Audio.create({ autoStart: false })
audio.on("error", (error, context) => {
console.error(`${context.action}: ${error.message}`)
})
const sound = await audio.loadSoundFile("click.wav")
if (sound != null && audio.start()) {
audio.play(sound, { volume: 0.8, pan: 0, loop: false })
}
```
Keep the engine alive while playback is needed, then call `audio.dispose()` from your application's shutdown lifecycle.
Use `setupAudio(options?)` when you prefer a function wrapper around `Audio.create(options?)`.
`Audio.create()` and `setupAudio()` initialize synchronously and throw `AudioInitializationError` if native library resolution, engine creation, or `autoStart` fails. Construction never emits events; successful `autoStart` also does not emit `started` before listeners can be attached.
## Engine options
| Option | Default | Description |
| ------------------ | -------- | ---------------------------------------------- |
| `autoStart` | `false` | Start playback during synchronous construction |
| `sampleRate` | `48000` | Engine sample rate |
| `playbackChannels` | `2` | Requested playback channel count |
| `startOptions` | defaults | Options passed to `start()` when auto-starting |
Create a new `Audio` instance to change `sampleRate` or `playbackChannels`.
## Sounds and voices
| Method | Description |
| ------------------------------- | ------------------------------------------------------------- |
| `loadSound(data)` | Decode `Uint8Array` or `ArrayBuffer`; returns `AudioSound` |
| `loadSoundFile(path)` | Read and decode a file; returns `Promise<AudioSound \| null>` |
| `unloadSound(sound)` | Free a loaded sound and stop voices using it |
| `play(sound, options?)` | Start a voice; returns `AudioVoice \| null` |
| `stopVoice(voice)` | Stop one active voice |
| `group(name)` | Create or reuse a named group; group `0` is the default group |
| `setVoiceGroup(voice, group)` | Move a voice to another group |
| `setGroupVolume(group, volume)` | Set group volume |
| `setMasterVolume(volume)` | Set master volume |
`play()` accepts `{ volume, pan, loop, groupId }`. Native playback clamps `volume` to `0..4` and `pan` to `-1..1`. The engine supports 32 active voices and streams in total.
## Encoded streams
Streaming supports `format: "mp3"` (the default) and `format: "flac"`. The selected format is exposed as `AudioStream.format`; unsupported formats reject before opening a source. Choose the entry point by source ownership:
| Method | Use |
| ------------------------------------ | -------------------------------------------------------------------- |
| `playStreamUrl(url, options?)` | Fetch, ICY negotiation, HTTP retry, reconnect, and response demuxing |
| `playStream(bytes, options?)` | One-shot encoded `ReadableStream` or `AsyncIterable` |
| `playStreamSource(connector, opts?)` | Reconnectable custom transport with per-connection demuxing |
All three resolve after the native decoder is ready, normally while buffering. Streaming begins before the source ends.
### URL streams
```typescript
import { Audio, AudioStreamError, type AudioStream } from "@opentui/core"
const audio = Audio.create({ autoStart: false })
let stream: AudioStream | null = null
const safeMetadata = (value: string | undefined) => value?.replace(/[\u0000-\u001f\u007f-\u009f]/g, " ")
audio.on("error", (error) => {
console.error(error.message)
})
try {
if (!audio.start()) throw new Error("No playback device available")
stream = await audio.playStreamUrl("https://anomaly.fm/radio", {
format: "mp3",
reconnect: { maxRetries: 5, retryOnEnd: true },
})
const initialMetadata = stream.getMetadata()
console.log(safeMetadata(initialMetadata?.fields.StreamTitle))
stream.on("metadata", (metadata) => {
console.log(safeMetadata(metadata?.headers["icy-name"]), safeMetadata(metadata?.fields.StreamTitle))
})
stream.on("error", (error) => {
console.error(error.message)
})
// Run the application until playback should stop.
} catch (error) {
if (error instanceof AudioStreamError) {
console.error(`${error.context.action}: ${error.message}`)
} else {
throw error
}
} finally {
stream?.dispose()
await stream?.closed
audio.dispose()
}
```
The default `contentTypePolicy: "validate"` accepts `audio/mpeg`, `audio/mp3`, and `application/mp3` for MP3; `audio/flac` and `audio/x-flac` for FLAC; and `application/octet-stream` or a missing type for either format. Set it to `"ignore"` to let the decoder validate mislabeled responses, or pass a callback receiving `{ format, contentType, status, url }`. Every response, including reconnect replacements, is evaluated before its bytes reach the decoder. `request` accepts `RequestInit` except `body` and `signal`; use the stream-level `signal` option for cancellation.
URL streams request ICY metadata with `Icy-MetaData: 1` unless `request.headers` already contains that header. Set it to `0` explicitly to disable negotiation. Any `icy-*` response headers are exposed as an ICY metadata snapshot; a positive `icy-metaint` additionally enables in-band framing. URLs, content types, station headers, and server names are not used to infer framing. Invalid or ambiguous intervals reject the response before a native voice is allocated.
ICY framing is removed in TypeScript before bytes reach the native decoder. `getMetadata()` returns the latest immutable `{ format, headers, fields }` snapshot or `null`. ICY response header names are lowercase in `headers`; in-band field names such as `StreamTitle` and `StreamUrl` retain their original case. Unknown fields are preserved. The `metadata` event emits the latest changed snapshot, and emits `null` when a replacement response no longer advertises ICY metadata. Rapid changes can coalesce into one asynchronous event; `getMetadata()` always returns the current snapshot. Zero-length and repeated equivalent metadata blocks do not emit events.
ICY does not define a text encoding. OpenTUI follows Icecast's documented non-Ogg default of ISO-8859-1, represented by the Encoding Standard's `windows-1252` decoder. Set `metadataEncoding`, for example `metadataEncoding: "utf-8"`, when the station uses another encoding. Metadata is untrusted text and should be sanitized before writing it directly to a terminal or interpreting metadata URLs.
Metadata events follow response ingestion, not audible playback. Native buffering means a title can arrive before its audio is heard. Metadata received during setup is retained and emitted asynchronously after `playStreamUrl()` resolves.
URL reconnect is opt-in. It retries network failures, interrupted or missing response bodies, HTTP 408/425/429, and HTTP 5xx responses. Reconnects restart the decoder in the same native stream session, preserving controls, counters, the voice slot, and any decoded PCM still available after an interruption. `retryOnEnd` restarts the same session after a clean response ends and its buffered PCM drains. Initial retries happen while `playStreamUrl()` is pending and do not emit `reconnecting` because the stream has not been returned yet; `reconnect.retry` still observes and can stop them. `maxRetries` limits consecutive retries for one outage and resets after a replacement decoder becomes ready; omitting it means unlimited retries. An `AbortSignal` or `dispose()` stops them.
Each replacement response creates a new demuxer and can advertise a different ICY interval. Existing metadata remains available while reconnecting, then the replacement clears prior fields. A response without ICY headers clears metadata.
Reconnect options default to `initialDelayMs: 1000`, `maxDelayMs: 15000`, `backoffFactor: 2`, `retryOnEnd: false`, and unlimited `maxRetries`. Valid `Retry-After` values override exponential backoff up to `maxDelayMs`. `reconnect.retry` can override default URL response classification; `{}` keeps the transport delay, while `{ delayMs }` replaces it.
### Byte streams
`playStream()` accepts encoded MP3 or FLAC bytes and is one-shot. Input chunks must be `Uint8Array`; request and reconnect options belong to the other entry points. Backpressure and encoded/decoded buffering remain bounded.
Pass a demuxer factory when framing or metadata is interleaved with the encoded audio. The built-in ICY demuxer accepts semantic options directly and does not require HTTP headers:
```typescript
import { Audio, createIcyStreamDemuxer } from "@opentui/core"
const stream = await audio.playStream(socketBytes, {
format: "mp3",
demuxer: () =>
createIcyStreamDemuxer({
metadataInterval: negotiatedInterval,
metadataEncoding: "utf-8",
}),
})
stream.on("metadata", (metadata) => {
const title = metadata?.fields.StreamTitle?.replace(/[\u0000-\u001f\u007f-\u009f]/g, " ")
console.log(title)
})
```
`createIcyStreamDemuxer()` defaults to the same ISO-8859-1/`windows-1252` decoding as URL streams. Optional `headers` are copied, lowercased, and exposed in metadata; non-HTTP transports can omit them.
### Custom connectors
A connector creates one transport connection at a time. It receives `{ signal, attempt }` and returns `{ body, info, close? }`. The initial connection uses attempt `0`; every reconnect gets a new connection and demuxer. Consecutive attempt numbering resets after a replacement decoder becomes ready, while `getStats().reconnectAttempts` remains cumulative.
```typescript
import { Audio, createIcyStreamDemuxer, type AudioStreamConnector } from "@opentui/core"
interface RadioInfo {
metadataInterval: number
}
const connector: AudioStreamConnector<RadioInfo> = {
async connect({ signal }) {
const connection = await openRadioConnection({ signal })
return {
body: connection.readable,
info: { metadataInterval: connection.metadataInterval },
close: () => connection.close(),
}
},
}
const stream = await audio.playStreamSource(connector, {
format: "mp3",
reconnect: {
maxRetries: 5,
retry() {
return {}
},
},
demuxer: (info) => createIcyStreamDemuxer({ metadataInterval: info.metadataInterval }),
})
```
`reconnect.retry(error, context)` is optional for URL and custom connectors. Custom connector and body-read failures are retryable by default. Return `false` to stop, `{}` to keep the transport hint or normal backoff, or `{ delayMs }` for a bounded finite non-negative integer delay. `context` contains the next `attempt`, `maxRetries`, and `phase`. `close()` is invoked once after EOF, failure, or cancellation; cleanup gets a bounded grace period so an uncooperative transport cannot prevent shutdown.
Custom demuxers expose `initialMetadata`, `push(chunk)`, and `flush()`, with optional `abort(reason)`. `push()` and `flush()` return ordered `{ type: "audio", data }` or `{ type: "metadata", metadata }` outputs. Flush output is consumed before decoder EOF; interruption calls `abort()` instead. A `push()` exception is a terminal demuxer error. A `flush()` exception reports truncated input and follows the reconnect read policy. `AudioStream<M>` exposes the same metadata type through `getMetadata()` and `metadata` events. Custom demuxers should return immutable snapshots and suppress equivalent updates when desired.
### Options and lifecycle
| Shared option | Default | Description |
| ------------------- | ------- | --------------------------------------------------------- |
| `format` | `"mp3"` | Encoded stream format: MP3 or FLAC |
| `volume` | `1` | Stream volume; native playback clamps to `0..4` |
| `pan` | `0` | Stereo pan; native playback clamps to `-1..1` |
| `groupId` | `0` | Mixer group |
| `buffer.capacityMs` | `2000` | Decoded PCM buffer capacity |
| `buffer.startupMs` | `1000` | Initial playback threshold |
| `buffer.resumeMs` | `1000` | Resume threshold after starvation |
| `maxProbeBytes` | `1 MiB` | Maximum encoded bytes inspected during decoder setup |
| `signal` | none | Cancels setup, retries, source reads, and active playback |
Buffer values must be positive integers; startup and resume thresholds cannot exceed capacity.
| `AudioStream` member | Description |
| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `state` | Latest cached lifecycle state |
| `getStats()` | Refreshes state and returns sample rate, channels, buffer capacity/use, encoded bytes, decoded/played frames, underruns, and reconnects |
| `getMetadata()` | Latest metadata snapshot or `null` |
| `setVolume()`, `setPan()`, `setGroup()` | Controls that remain effective across reconnects |
| `dispose()` | Cancels transport, demuxer, retries, and native playback |
| `closed` | Resolves after bounded cleanup and delivery of the terminal `ended`, `error`, or `disposed` event |
Events are `metadata`, `reconnecting`, `ended`, `error`, and `disposed`. Transport, response, demuxer, native creation, and decoder setup failures reject with `AudioStreamError`; validation performed in JavaScript uses `TypeError`/`RangeError`, cancellation uses `AbortError`, and starting a stream after its owning `Audio` was disposed rejects with `Error`. Later terminal failures emit `error` with context. Disposing the owning `Audio` also disposes pending and active streams.
`bytesReceived` counts only encoded audio bytes emitted by the demuxer; framing and metadata are excluded. Final stats and metadata remain readable after natural completion. Native readiness, backpressure, and EOF draining use short-lived polling; there is no persistent idle statistics loop.
In browser-like Fetch environments, cross-origin ICY negotiation requires an appropriate `Access-Control-Allow-Origin`, permission for the `Icy-MetaData` request header, and exposure of `icy-metaint` through CORS. Other `icy-*` response headers must also be exposed for them to appear in `getMetadata().headers`. Legacy servers that respond with the non-HTTP `ICY 200 OK` status line can be rejected by `fetch()` before OpenTUI receives a response.
Streams use the same mixer and master tap as loaded sounds. Once `enableTap()` is active, `readTapFrames()` includes streamed audio automatically. The tap is the combined master mix, not an isolated copy of one stream.
## Starting audio
`start()` starts native playback and returns `false` when no output device is available. Use `isStarted()` to check native playback state.
The playback device remains open until `stop()` or `dispose()` is called, even when no voices are active. `stop()` releases the device while preserving loaded sounds, so call `start()` again before the next playback. Because `play()` returns when a voice starts rather than when it finishes, calling `stop()` immediately afterward can interrupt the sound.
For headless mixing, tests, benchmarks, or manual `mixFrames()` output, use `startMixer()`. It starts the mixer without opening a playback device. Use `isMixerStarted()` to check either mode.
## Bun standalone executables
Bun embeds files in `bun build --compile` executables when they are imported with `with { type: "file" }`. Read the embedded file as bytes and pass it to `loadSound()` so audio decodes from memory instead of a file path.
```typescript
import { file } from "bun"
import { Audio } from "@opentui/core"
import clickPath from "./click.wav" with { type: "file" }
const audio = Audio.create({ autoStart: false })
const clickBytes = await file(clickPath).bytes()
const click = audio.loadSound(clickBytes)
if (click != null && audio.start()) {
audio.play(click)
}
```
Compile the app normally with `bun build --compile ./app.ts --outfile app`.
## Node.js standalone executables
After following the [Node.js SEA asset setup](/docs/reference/standalone-executables#nodejs-sea), add application audio files to the SEA `assets` map alongside the OpenTUI runtime manifest:
```js
assets: {
...Object.fromEntries(openTuiAssets.map(({ key, source }) => [key, source])),
"app/click.wav": resolve("click.wav"),
}
```
Load the embedded bytes through Node's SEA API and pass them directly to `loadSound()`, which accepts `ArrayBuffer` and `Uint8Array`:
```typescript
import { getAsset } from "node:sea"
import { Audio } from "@opentui/core"
const audio = Audio.create({ autoStart: false })
const click = audio.loadSound(getAsset("app/click.wav"))
if (click != null && audio.start()) {
audio.play(click)
}
```
## Devices
Select devices before starting the engine:
```typescript
const audio = Audio.create({ autoStart: false })
const devices = audio.listPlaybackDevices() ?? []
const device = devices.find((item) => item.isDefault) ?? devices[0]
if (device) {
audio.selectPlaybackDevice(device.index)
}
if (!audio.start()) {
console.error("No playback device available")
}
```
`listPlaybackDevices()` returns `{ index, name, isDefault }[] | null`. Use `clearPlaybackDeviceSelection()` to return to the default device selection.
## Capture
Capture records PCM from an input device into a bounded native ring buffer. It is independent from playback: capture does not start output or the mixer, and `start()`, `startMixer()`, and `stop()` do not change capture state.
> **Microphone access**
>
> The terminal app launching OpenTUI must be allowed to use the microphone. On macOS, enable it under **System Settings > Privacy & Security > Microphone**, then fully restart the terminal. Packaged apps need their own microphone permission.
Select an input with `selectCaptureDevice()` before opening capture, or let the backend use its default input.
### Capture streams
`openCapture()` starts input and resolves to an `AudioCaptureStream`. Its `readable` emits exact-length `Float32Array` chunks: normally `chunkFrames * channels` samples, with a shorter final chunk when `stop()` drains the ring.
```typescript
import { Audio } from "@opentui/core"
const audio = Audio.create({ autoStart: false })
const stream = await audio.openCapture({ channels: 1, chunkFrames: 2048 })
stream.on("error", (error, context) => {
console.error(`${context.action}: ${error.message}`)
})
async function consumePcm(pcm: Float32Array) {
console.log(`received ${pcm.length / stream.channels} frames`)
}
const consumption = (async () => {
for await (const pcm of stream.readable) {
await consumePcm(pcm)
}
})()
// Later: stop input, consume the final drain, then await terminal event delivery.
stream.stop()
await consumption
await stream.closed
audio.dispose()
```
| Stream option | Default | Description |
| ---------------- | ------------------ | ------------------------------------------------------- |
| `channels` | `1` | Number of interleaved input channels |
| `capacityFrames` | `audio.sampleRate` | Native ring capacity, equal to one second by default |
| `chunkFrames` | `2048` | Frames per normal stream chunk |
| `startOptions` | defaults | Low-level options passed when starting the input device |
| `signal` | none | Disposes capture when aborted |
All frame counts and channel counts must be positive integers, and `chunkFrames` cannot exceed `capacityFrames`. Capture uses `audio.sampleRate`; create another `Audio` instance to change it. `startOptions` accepts the fields in [Start options](#start-options).
PCM is interleaved 32-bit floating point, nominally in `[-1, 1]`; every emitted array contains only valid samples and a frame contains one sample per channel. The stream does not build a JavaScript chunk queue. Its native ring is the bounded backpressure buffer, and the native callback never waits for JavaScript. If it fills, newest input frames are dropped while older frames remain ordered; `getStats().framesDropped` reports the loss.
| `AudioCaptureStream` member | Description |
| --------------------------- | ----------------------------------------------------------------------------------------------- |
| `readable` | Demand-driven `ReadableStream<Float32Array>` of interleaved PCM |
| `sampleRate`, `channels` | Output format |
| `chunkFrames` | Requested normal chunk size |
| `state` | `initializing`, `capturing`, `stopping`, `stopped`, `errored`, or `disposed` |
| `getStats()` | Returns state, format, ring capacity/use and duration, and received/read/dropped frame counters |
| `stop()` | Stops the device and gracefully drains unread PCM to an active consumer |
| `dispose()` | Terminates capture without a graceful stream drain |
| `closed` | Resolves after cleanup and delivery of the one terminal `stopped`, `error`, or `disposed` event |
After `stop()`, a locked consumer can read all remaining full chunks and one exact-length partial chunk. If no consumer holds the readable lock, unread PCM is discarded in bounded batches so shutdown and `closed` cannot be held open by absent demand. Canceling `readable`, calling `dispose()`, aborting `signal`, or disposing the owning `Audio` terminates the stream as `disposed` instead of performing the graceful drain.
An `AudioCaptureStream` exclusively owns capture until terminal cleanup completes. A second `openCapture()` or `recordToFile()` rejects, while direct destructive methods `startCapture()`, `readCaptureFrames()`, `stopCapture()`, `selectCaptureDevice()`, and `clearCaptureDeviceSelection()` fail and emit `Audio.error`. Device enumeration and capture statistics remain available.
Setup failures reject with `AudioCaptureStreamError`; a pre-start abort rejects with `AbortError`. Later native failures error `readable` and emit `error` with the same `AudioCaptureStreamError` and `{ action, status? }` context. Terminal events are asynchronous and emitted once; `closed` always resolves after the event handler has run.
### Record WAV
`recordToFile()` captures directly to a PCM16 WAV file through an `AudioRecorder`:
```typescript
import { Audio } from "@opentui/core"
const audio = Audio.create({ autoStart: false })
const recorder = await audio.recordToFile("recording.wav", { channels: 1 })
recorder.on("error", (error, context) => {
console.error(`${context.action}: ${error.message}`)
})
// Later: drain capture, finalize the WAV header, and publish the file.
recorder.stop()
await recorder.closed
audio.dispose()
```
The recorder always writes little-endian PCM16 WAV (`format === "wav"`) and accepts only mono or stereo capture. It converts each exact-length float chunk and awaits each file write sequentially, so the native capture ring remains the bounded buffer while storage is slow. The classic RIFF size limit applies.
Recording inherits the capture options and defaults above. Data is first written to a uniquely named temporary file in the destination directory; the destination is atomically replaced only after the temporary file has its final header, is synced, and is closed.
| `AudioRecorder` member | Description |
| ------------------------ | ----------------------------------------------------------------------------------------------- |
| `filePath`, `format` | Destination path and the literal `"wav"` |
| `sampleRate`, `channels` | Recorded format |
| `state` | `initializing`, `recording`, `stopping`, `stopped`, `errored`, or `disposed` |
| `getStats()` | Capture stats plus `framesWritten`, `dataBytesWritten`, and `durationMs` |
| `stop()` | Stops input, drains every buffered frame, finalizes and syncs the WAV, then publishes it |
| `dispose()` | Aborts before publication and removes the temporary file |
| `closed` | Resolves after cleanup and delivery of the one terminal `stopped`, `error`, or `disposed` event |
Aborting `signal` or disposing the owning `Audio` has the same cleanup behavior as `AudioRecorder.dispose()`. Once publication has started it is the point of no return and is allowed to finish. Any dropped capture frame is terminal: the recorder emits `AudioRecorderError`, removes its temporary file, and never publishes incomplete audio. Open/start failures reject `recordToFile()` after cleanup; later failures emit `error` with `{ action, status? }`. `stopped`, `error`, and `disposed` are asynchronous, mutually exclusive terminal events, and `closed` resolves after event delivery.
### Low-level capture
Use `startCapture()` and `readCaptureFrames()` when application-controlled polling is required. They expose the same native ring without stream lifecycle management:
```typescript
import { Audio } from "@opentui/core"
const audio = Audio.create({ autoStart: false })
if (!audio.startCapture({ channels: 1 })) throw new Error("Could not start capture")
const result = audio.readCaptureFrames(1024)
if (result) {
const pcm = result.frames.subarray(0, result.framesRead)
console.log(`read ${pcm.length} mono frames`)
}
audio.stopCapture()
audio.dispose()
```
| Method | Result and behavior |
| ------------------------------- | --------------------------------------------------------------------------------- |
| `listCaptureDevices()` | Returns `AudioCaptureDevice[] \| null`; an empty array means no inputs were found |
| `selectCaptureDevice(index)` | Selects an enumerated input and returns whether selection succeeded |
| `clearCaptureDeviceSelection()` | Clears the explicit selection so the backend chooses the default input |
| `startCapture(options?)` | Starts input, returning `false` on native or permission failure |
| `isCapturing()` | Reports whether input is currently running |
| `readCaptureFrames(frameCount)` | Consumes up to `frameCount` frames and returns `{ frames, framesRead } \| null` |
| `getCaptureStats()` | Returns format, buffer use, and received/read/dropped counters |
| `stopCapture()` | Stops input while preserving unread PCM for direct reads |
Direct reads allocate `frameCount * channels` samples, but only the first `framesRead * channels` are valid. `frameCount` cannot exceed the configured capacity. While running, `startCapture()` is idempotent when no new device options are supplied and any explicit channels/capacity match the active configuration. Stop capture before changing its configuration. A later successful start resets the ring and counters. Successful transitions emit `captureStarted` and `captureStopped`; native failures return `false` or `null` and emit `Audio.error`. Device indices belong to the latest enumeration and can change after hot-plugging. Invalid dimensions and indices throw `TypeError` or `RangeError` before native work.
### Permissions and scope
Windows privacy controls can block desktop microphone access. Linux capture uses PulseAudio or ALSA, including PipeWire through its PulseAudio compatibility layer, and depends on device permissions. Headless sessions, sandboxes, containers, and CI commonly expose no input. Enumeration may return `null` or `[]`; denied or unavailable inputs may make `startCapture()` fail or deliver clocked digital silence, so advancing frame counters alone does not prove a signal is present.
Capture streams expose raw PCM and the recorder encodes PCM16 WAV. Compressed recording formats, system-output loopback capture, JavaScript callbacks on the native audio thread, and echo cancellation remain unsupported. Perform compressed encoding and signal processing outside the capture callback path, and use platform-specific facilities for loopback capture.
## Mixing and analysis
| Method | Description |
| --------------------------------------- | ------------------------------------------------------------------------------------------ |
| `mixFrames(frameCount, channels=2)` | Mix into a `Float32Array`; mono downmixes, channels above stereo keep extras zero |
| `getStats()` | Returns `soundsLoaded`, `voicesActive`, `framesMixed`, `lockMisses`, `lastPeak`, `lastRms` |
| `enableTap(capacityFrames=8192)` | Enable a fixed-size native ring buffer of recent mixed frames |
| `readTapFrames(frameCount, channels=2)` | Copy the latest tapped frames; returns `{ frames, framesRead }` |
| `disableTap()` | Free the tap buffer |
The tap is disabled by default. When enabled, it stores the latest frames and overwrites old data; reads do not drain it. OpenTUI does not compute FFT natively. For spectrum visualization, read tap frames and run FFT in TypeScript.
## Start options
`start(options?)` accepts low-level device options: `periodSizeInFrames`, `periodSizeInMilliseconds`, `periods`, `performanceProfile`, `shareMode`, `noPreSilencedOutputBuffer`, `noClip`, `noDisableDenormals`, `noFixedSizedCallback`, `wasapiNoAutoConvertSrc`, `wasapiNoDefaultQualitySrc`, `alsaNoMMap`, `alsaNoAutoFormat`, `alsaNoAutoChannels`, and `alsaNoAutoResample`.
`performanceProfile` uses `0` for low latency and `1` for conservative. `shareMode` uses `0` for shared and `1` for exclusive.
## Lifecycle and errors
Use `start()`, `startMixer()`, `stop()`, `isStarted()`, `isMixerStarted()`, and `dispose()` for lifecycle. After construction, `Audio` emits `started`, `mixerStarted`, `captureStarted`, `captureStopped`, `stopped`, `disposed`, and `error`. Most non-stream operations report failure by returning `false` or `null` and emitting `Audio.error` with `{ action, status }`; stream setup rejection and `AudioStream` errors are described above.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |