benschac/snap_post / .agents/skills/react-native-vision-camera
react-native-vision-camera
UnverifiedBest-practices guide for React Native VisionCamera v5 setup, migration, capture, controls, outputs, and basic frame processing. Use the separate react-native-vision-camera-realtime skill for production low-latency GPU, ML, CV, Skia or WebGPU pipelines and frame-coupled overlays.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 8.6 KB |
| references/advanced-features.md | markdown | 12.2 KB |
| references/capture-and-controls.md | markdown | 11.4 KB |
| references/frame-processors.md | markdown | 7.4 KB |
| references/migration-templates.md | markdown | 16.3 KB |
| references/migration-v4-to-v5.md | markdown | 22.5 KB |
| references/outputs-and-constraints.md | markdown | 10.1 KB |
| references/quickstart-v5.md | markdown | 6.8 KB |
references/outputs-and-constraints.md
View on GitHub# Outputs, Constraints, Devices, and Session lifecycle
The three ideas that define v5:
1. **Outputs are first-class objects.** A Camera is "idle" until outputs are attached. Each output (Photo, Video, Frame, Depth, Preview, Object) is a `HybridObject` created upfront, passed via `outputs={[...]}`, and owns its own capture methods.
2. **Constraints are negotiated declarations of intent.** You say what you want (fps, HDR, resolution bias, stabilization); the Camera picks the closest supported config. Array order = priority.
3. **There are three usage forms.** Declarative `<Camera />`, hook-driven `useCamera(...)`, and imperative `VisionCamera.createCameraSession(...)`. Pick the least powerful one that does the job.
## Outputs
| Output | Create with | Purpose |
|---|---|---|
| `CameraPhotoOutput` | `usePhotoOutput(options)` | Still capture. `capturePhoto` / `capturePhotoToFile`. |
| `CameraVideoOutput` | `useVideoOutput(options)` | Video recording. `createRecorder`. |
| `CameraFrameOutput` | `useFrameOutput({ onFrame, pixelFormat, targetResolution })` | Real-time frame streaming for ML / CV. |
| `CameraDepthFrameOutput` | `useDepthOutput({ onDepth })` | Depth streams from LiDAR / ToF / disparity virtual cameras. |
| `CameraPreviewOutput` | `usePreviewOutput()` | Manual preview via `<NativePreviewView />` (multi-cam). |
| `CameraObjectOutput` (iOS only) | `useObjectOutput({ types, onObjectsScanned })` | Native QR/face/body detection via `AVCaptureMetadataOutput`, no ML dep. |
| Barcode output (separate package) | `useBarcodeScannerOutput(...)` | MLKit barcode detection on both platforms. |
Key rules:
- Attach all outputs you want to use at once — swapping outputs at runtime causes a brief reconfiguration pause.
- **Fewer outputs = higher negotiated resolution.** 8K photo on iOS is only reachable with photo-only output sets.
- Each output has its own `targetResolution`. Set it to the smallest resolution the consumer actually needs (e.g. your ML model wants 128×128 → don't stream 4K frames to dispose of 99% of pixels).
- Use the `onConfigured` callback (on the Camera or session) to gate work that depends on the output being ready.
### Hook vs imperative creation
```tsx
// Hook (declarative + useCamera)
const photoOutput = usePhotoOutput({ targetResolution: CommonResolutions.UHD_16_9 })
const videoOutput = useVideoOutput({ enableAudio: true })
<Camera device="back" isActive={true} outputs={[photoOutput, videoOutput]} />
// Or:
const camera = useCamera({
device,
outputs: [photoOutput, videoOutput],
constraints: [{ fps: 30 }],
isActive: true,
})
```
```ts
// Imperative — full control, required for multi-cam
const session = await VisionCamera.createCameraSession(/* isMultiCam */ false)
const device = await getDefaultCameraDevice('back')
const photoOutput = VisionCamera.createPhotoOutput({})
const [controller] = await session.configure([{
input: device,
outputs: [{ output: photoOutput, mirrorMode: 'auto' }],
constraints: [{ fps: 30 }],
}], {})
await session.start()
// ... later:
await session.stop()
// No session.dispose() — CameraSession has no dispose(); Nitro releases it once unreferenced.
// source: CameraSession.nitro.ts only exposes isRunning/configure/start/stop/addOn*Listener
```
## Constraints
The Constraints API replaces the entire v4 formats system. You never pick a `CameraDeviceFormat` by hand again.
### Principles
- **Array order = priority, descending.** The first constraint is the one the Camera will bend least to satisfy.
- **Always succeeds.** `{ fps: 99999 }` resolves to the highest supported FPS. Constraints never throw for unreachable combos.
- **Probe with `device.isSessionConfigSupported(config)`** — a synchronous method on `CameraDevice` returning `boolean` — when you want to conditionally show UI (e.g. only render an HDR toggle on devices that can actually do HDR at the requested resolution). <!-- source: CameraDevice.nitro.ts:611 `isSessionConfigSupported(config: CameraSessionConfig): boolean` -->
- **Observe the chosen config** via `onSessionConfigSelected={(config) => ...}` or by reading `config` returned from `resolveConstraints`.
### Constraint types
```ts
// FPS
{ fps: 60 }
// Photo HDR (3-frame fusion at ISP)
{ photoHDR: true }
// Video HDR / Log
{ videoDynamicRange: { bitDepth: 'hdr-10-bit', colorSpace: 'hlg-bt2020', colorRange: 'full' } }
// Shortcut:
{ videoDynamicRange: CommonDynamicRanges.ANY_HDR }
// Apple Log: colorSpace: 'apple-log'
// Stabilization
{ videoStabilizationMode: 'cinematic-extended' }
{ previewStabilizationMode: 'standard' }
// Pixel format (frame output)
{ pixelFormat: 'yuv-420-8-bit-full' }
// Resolution — prefer an output-driven bias over pixel dimensions
{ resolutionBias: photoOutput }
// Binned sensor readout — bigger effective pixels, better low-light, less bandwidth
{ binned: true }
```
### Common recipes
Photo-first tuning (highest-res photos, still a usable preview/video path):
```ts
constraints={[
{ resolutionBias: photoOutput },
{ photoHDR: true },
{ resolutionBias: videoOutput },
]}
```
High-FPS video (60fps wins over resolution):
```ts
constraints={[
{ fps: 60 },
{ resolutionBias: videoOutput },
]}
```
Low-resolution frame stream for ML:
```ts
const frameOutput = useFrameOutput({ targetResolution: CommonResolutions.VGA_16_9, onFrame })
constraints={[{ resolutionBias: frameOutput }]}
```
10-bit HLG video:
```ts
constraints={[{ videoDynamicRange: { bitDepth: 'hdr-10-bit', colorSpace: 'hlg-bt2020', colorRange: 'full' } }]}
```
### Programmatic resolution
```ts
const config = await VisionCamera.resolveConstraints(
device,
[{ output: videoOutput, mirrorMode: 'auto' }],
[{ resolutionBias: videoOutput }, { fps: 60 }],
) // resolveConstraints is a method on VisionCamera — source: CameraFactory.nitro.ts:149
const ok = device.isSessionConfigSupported(config) // sync, single arg, on the device — source: CameraDevice.nitro.ts:611
```
## Devices
- `useCameraDevice('back')` / `useCameraDevice('front')` / `useCameraDevice('external')` — most common.
- `useCameraDevices()` for reactive listings (e.g. UVC plug/unplug).
- Filter by physical types when you need a simpler pipeline (faster startup):
```tsx
const device = useCameraDevice('back', { physicalDevices: ['wide-angle'] })
```
vs. a `'triple'` virtual device that switches across ultra-wide / wide / telephoto.
- Probe capabilities upfront: `device.getSupportedResolutions('photo')`, `device.supportedFPSRanges`, `device.supportedPixelFormats`, `device.supportsPhotoHDR`, `device.supportsVideoStabilizationMode('cinematic')`, `device.supportsExposureBias`, `device.supportsFocusMetering`, `device.supportsFocusLocking`, `device.zoomLensSwitchFactors`. Multi-cam is **platform-level**, not a device flag: use `VisionCamera.supportsMultiCamSessions` and `deviceFactory.supportedMultiCamDeviceCombinations`. <!-- source: device props in CameraDevice.nitro.ts; multi-cam in CameraFactory.nitro.ts:71 + CameraSession.nitro.ts:70,182 (no supportsMultiCamSessions on CameraDevice) -->
- External cameras (iPad/Mac/UVC on Android) use `'external'` and emit change notifications: `addOnCameraDevicesChangedListener`.
## Session lifecycle
Three states:
1. **Idle** — no connections, not running.
2. **Ready** — configured with outputs but not streaming. Memory + permissions still held.
3. **Active** — streaming.
Toggle between Ready and Active with `isActive`. Do **not** toggle between Idle and Ready by mounting/unmounting — that tears down and rebuilds the whole session.
```tsx
import { useIsFocused } from '@react-navigation/native'
function Screen() {
const isFocused = useIsFocused()
return <Camera isActive={isFocused} /* ... */ />
}
```
Interruptions (phone calls, thermal throttle, another app grabbing the camera):
```tsx
<Camera
onInterruptionStarted={(reason) => {}}
onInterruptionEnded={() => {}}
onError={(error) => {}}
/>
```
Reconfiguration cost: changing `outputs`, `device`, or `constraints` pauses briefly while the session rebuilds. Batch changes and avoid changing them in render loops. The docs guide explicitly says: *"ensure that most configuration is handled before you start the session, as any configuration while the session is running can be expensive and cause stutters."*
## Performance cheatsheet
- Prefer single-physical-device cameras (`'wide-angle'` only) over virtual multi-device cameras when you don't need seamless zoom switching — faster startup.
- Disable features you don't need: video HDR, stabilization, unneeded outputs.
- For frame output, the default `pixelFormat` is `'native'` (zero-copy, format negotiated — check `frame.pixelFormat`). Among CPU-accessible formats, `'yuv'` beats `'rgb'` (~2.6× less bandwidth); `'rgb'` forces a conversion per frame. <!-- source: useFrameOutput.ts:123 (default 'native'); CameraFrameOutput.nitro.ts:71-95 -->
- Match FPS and resolution to the consumer. 30fps is sufficient for 99% of recording; 60/120/240 only when the UX demands it.
- Binned formats (`{ binned: true }`) for better low-light and lower bandwidth when fine detail is not needed.
- For rapid photo bursts: `qualityPrioritization: 'speed'` on the photo output options or per capture. For instant capture with zero shutter lag, use `takeSnapshot()` via the Camera ref.
- Don't rotate buffers in the pipeline — handle orientation downstream with flags.
## Pointers
- Docs — Camera Outputs: https://visioncamera.margelo.com/docs/camera-outputs
- Docs — Constraints: https://visioncamera.margelo.com/docs/constraints
- Docs — Photo Output: https://visioncamera.margelo.com/docs/photo-output
- Docs — Depth Output: https://visioncamera.margelo.com/docs/depth-output
- Docs — Object Output: https://visioncamera.margelo.com/docs/object-output
- Docs — Async Frame Processing: https://visioncamera.margelo.com/docs/async-frame-processing
- API — `Constraint` type: https://visioncamera.margelo.com/api/react-native-vision-camera/type-aliases/Constraint
- API — `CameraPhotoOutput`: https://visioncamera.margelo.com/api/react-native-vision-camera/hybrid-objects/CameraPhotoOutput
- Related: [capture-and-controls.md](./capture-and-controls.md), [frame-processors.md](./frame-processors.md), [advanced-features.md](./advanced-features.md), [quickstart-v5.md](./quickstart-v5.md)
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |