jonluca/palate / .agents/skills/react-native-vision-camera
react-native-vision-camera
UnverifiedBest-practices guide for react-native-vision-camera v5 (Nitro rewrite, April 2026) and migrating from v4. Use when installing, configuring, or writing code with the Camera, outputs, frame processors, recording, or barcode/depth/RAW features. Also use when converting v4 code (photo={true}, takePhoto, useCameraFormat, useFrameProcessor) to the new v5 API.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 8.2 KB |
| references/advanced-features.md | markdown | 12.2 KB |
| references/capture-and-controls.md | markdown | 11.4 KB |
| references/frame-processors.md | markdown | 6.8 KB |
| references/migration-templates.md | markdown | 16.3 KB |
| references/migration-v4-to-v5.md | markdown | 22.5 KB |
| references/outputs-and-constraints.md | markdown | 10.1 KB |
| references/quickstart-v5.md | markdown | 6.8 KB |
references/outputs-and-constraints.md
View on GitHub# Outputs, Constraints, Devices, and Session lifecycle
The three ideas that define v5:
1. **Outputs are first-class objects.** A Camera is "idle" until outputs are attached. Each output (Photo, Video, Frame, Depth, Preview, Object) is a `HybridObject` created upfront, passed via `outputs={[...]}`, and owns its own capture methods.
2. **Constraints are negotiated declarations of intent.** You say what you want (fps, HDR, resolution bias, stabilization); the Camera picks the closest supported config. Array order = priority.
3. **There are three usage forms.** Declarative `<Camera />`, hook-driven `useCamera(...)`, and imperative `VisionCamera.createCameraSession(...)`. Pick the least powerful one that does the job.
## Outputs
| Output | Create with | Purpose |
|---|---|---|
| `CameraPhotoOutput` | `usePhotoOutput(options)` | Still capture. `capturePhoto` / `capturePhotoToFile`. |
| `CameraVideoOutput` | `useVideoOutput(options)` | Video recording. `createRecorder`. |
| `CameraFrameOutput` | `useFrameOutput({ onFrame, pixelFormat, targetResolution })` | Real-time frame streaming for ML / CV. |
| `CameraDepthFrameOutput` | `useDepthOutput({ onDepth })` | Depth streams from LiDAR / ToF / disparity virtual cameras. |
| `CameraPreviewOutput` | `usePreviewOutput()` | Manual preview via `<NativePreviewView />` (multi-cam). |
| `CameraObjectOutput` (iOS only) | `useObjectOutput({ types, onObjectsScanned })` | Native QR/face/body detection via `AVCaptureMetadataOutput`, no ML dep. |
| Barcode output (separate package) | `useBarcodeScannerOutput(...)` | MLKit barcode detection on both platforms. |
Key rules:
- Attach all outputs you want to use at once — swapping outputs at runtime causes a brief reconfiguration pause.
- **Fewer outputs = higher negotiated resolution.** 8K photo on iOS is only reachable with photo-only output sets.
- Each output has its own `targetResolution`. Set it to the smallest resolution the consumer actually needs (e.g. your ML model wants 128×128 → don't stream 4K frames to dispose of 99% of pixels).
- Use the `onConfigured` callback (on the Camera or session) to gate work that depends on the output being ready.
### Hook vs imperative creation
```tsx
// Hook (declarative + useCamera)
const photoOutput = usePhotoOutput({ targetResolution: CommonResolutions.UHD_16_9 })
const videoOutput = useVideoOutput({ enableAudio: true })
<Camera device="back" isActive={true} outputs={[photoOutput, videoOutput]} />
// Or:
const camera = useCamera({
device,
outputs: [photoOutput, videoOutput],
constraints: [{ fps: 30 }],
isActive: true,
})
```
```ts
// Imperative — full control, required for multi-cam
const session = await VisionCamera.createCameraSession(/* isMultiCam */ false)
const device = await getDefaultCameraDevice('back')
const photoOutput = VisionCamera.createPhotoOutput({})
const [controller] = await session.configure([{
input: device,
outputs: [{ output: photoOutput, mirrorMode: 'auto' }],
constraints: [{ fps: 30 }],
}], {})
await session.start()
// ... later:
await session.stop()
// No session.dispose() — CameraSession has no dispose(); Nitro releases it once unreferenced.
// source: CameraSession.nitro.ts only exposes isRunning/configure/start/stop/addOn*Listener
```
## Constraints
The Constraints API replaces the entire v4 formats system. You never pick a `CameraDeviceFormat` by hand again.
### Principles
- **Array order = priority, descending.** The first constraint is the one the Camera will bend least to satisfy.
- **Always succeeds.** `{ fps: 99999 }` resolves to the highest supported FPS. Constraints never throw for unreachable combos.
- **Probe with `device.isSessionConfigSupported(config)`** — a synchronous method on `CameraDevice` returning `boolean` — when you want to conditionally show UI (e.g. only render an HDR toggle on devices that can actually do HDR at the requested resolution). <!-- source: CameraDevice.nitro.ts:611 `isSessionConfigSupported(config: CameraSessionConfig): boolean` -->
- **Observe the chosen config** via `onSessionConfigSelected={(config) => ...}` or by reading `config` returned from `resolveConstraints`.
### Constraint types
```ts
// FPS
{ fps: 60 }
// Photo HDR (3-frame fusion at ISP)
{ photoHDR: true }
// Video HDR / Log
{ videoDynamicRange: { bitDepth: 'hdr-10-bit', colorSpace: 'hlg-bt2020', colorRange: 'full' } }
// Shortcut:
{ videoDynamicRange: CommonDynamicRanges.ANY_HDR }
// Apple Log: colorSpace: 'apple-log'
// Stabilization
{ videoStabilizationMode: 'cinematic-extended' }
{ previewStabilizationMode: 'standard' }
// Pixel format (frame output)
{ pixelFormat: 'yuv-420-8-bit-full' }
// Resolution — prefer an output-driven bias over pixel dimensions
{ resolutionBias: photoOutput }
// Binned sensor readout — bigger effective pixels, better low-light, less bandwidth
{ binned: true }
```
### Common recipes
Photo-first tuning (highest-res photos, still a usable preview/video path):
```ts
constraints={[
{ resolutionBias: photoOutput },
{ photoHDR: true },
{ resolutionBias: videoOutput },
]}
```
High-FPS video (60fps wins over resolution):
```ts
constraints={[
{ fps: 60 },
{ resolutionBias: videoOutput },
]}
```
Low-resolution frame stream for ML:
```ts
const frameOutput = useFrameOutput({ targetResolution: CommonResolutions.VGA_16_9, onFrame })
constraints={[{ resolutionBias: frameOutput }]}
```
10-bit HLG video:
```ts
constraints={[{ videoDynamicRange: { bitDepth: 'hdr-10-bit', colorSpace: 'hlg-bt2020', colorRange: 'full' } }]}
```
### Programmatic resolution
```ts
const config = await VisionCamera.resolveConstraints(
device,
[{ output: videoOutput, mirrorMode: 'auto' }],
[{ resolutionBias: videoOutput }, { fps: 60 }],
) // resolveConstraints is a method on VisionCamera — source: CameraFactory.nitro.ts:149
const ok = device.isSessionConfigSupported(config) // sync, single arg, on the device — source: CameraDevice.nitro.ts:611
```
## Devices
- `useCameraDevice('back')` / `useCameraDevice('front')` / `useCameraDevice('external')` — most common.
- `useCameraDevices()` for reactive listings (e.g. UVC plug/unplug).
- Filter by physical types when you need a simpler pipeline (faster startup):
```tsx
const device = useCameraDevice('back', { physicalDevices: ['wide-angle'] })
```
vs. a `'triple'` virtual device that switches across ultra-wide / wide / telephoto.
- Probe capabilities upfront: `device.getSupportedResolutions('photo')`, `device.supportedFPSRanges`, `device.supportedPixelFormats`, `device.supportsPhotoHDR`, `device.supportsVideoStabilizationMode('cinematic')`, `device.supportsExposureBias`, `device.supportsFocusMetering`, `device.supportsFocusLocking`, `device.zoomLensSwitchFactors`. Multi-cam is **platform-level**, not a device flag: use `VisionCamera.supportsMultiCamSessions` and `deviceFactory.supportedMultiCamDeviceCombinations`. <!-- source: device props in CameraDevice.nitro.ts; multi-cam in CameraFactory.nitro.ts:71 + CameraSession.nitro.ts:70,182 (no supportsMultiCamSessions on CameraDevice) -->
- External cameras (iPad/Mac/UVC on Android) use `'external'` and emit change notifications: `addOnCameraDevicesChangedListener`.
## Session lifecycle
Three states:
1. **Idle** — no connections, not running.
2. **Ready** — configured with outputs but not streaming. Memory + permissions still held.
3. **Active** — streaming.
Toggle between Ready and Active with `isActive`. Do **not** toggle between Idle and Ready by mounting/unmounting — that tears down and rebuilds the whole session.
```tsx
import { useIsFocused } from '@react-navigation/native'
function Screen() {
const isFocused = useIsFocused()
return <Camera isActive={isFocused} /* ... */ />
}
```
Interruptions (phone calls, thermal throttle, another app grabbing the camera):
```tsx
<Camera
onInterruptionStarted={(reason) => {}}
onInterruptionEnded={() => {}}
onError={(error) => {}}
/>
```
Reconfiguration cost: changing `outputs`, `device`, or `constraints` pauses briefly while the session rebuilds. Batch changes and avoid changing them in render loops. The docs guide explicitly says: *"ensure that most configuration is handled before you start the session, as any configuration while the session is running can be expensive and cause stutters."*
## Performance cheatsheet
- Prefer single-physical-device cameras (`'wide-angle'` only) over virtual multi-device cameras when you don't need seamless zoom switching — faster startup.
- Disable features you don't need: video HDR, stabilization, unneeded outputs.
- For frame output, the default `pixelFormat` is `'native'` (zero-copy, format negotiated — check `frame.pixelFormat`). Among CPU-accessible formats, `'yuv'` beats `'rgb'` (~2.6× less bandwidth); `'rgb'` forces a conversion per frame. <!-- source: useFrameOutput.ts:123 (default 'native'); CameraFrameOutput.nitro.ts:71-95 -->
- Match FPS and resolution to the consumer. 30fps is sufficient for 99% of recording; 60/120/240 only when the UX demands it.
- Binned formats (`{ binned: true }`) for better low-light and lower bandwidth when fine detail is not needed.
- For rapid photo bursts: `qualityPrioritization: 'speed'` on the photo output options or per capture. For instant capture with zero shutter lag, use `takeSnapshot()` via the Camera ref.
- Don't rotate buffers in the pipeline — handle orientation downstream with flags.
## Pointers
- Docs — Camera Outputs: https://visioncamera.margelo.com/docs/camera-outputs
- Docs — Constraints: https://visioncamera.margelo.com/docs/constraints
- Docs — Photo Output: https://visioncamera.margelo.com/docs/photo-output
- Docs — Depth Output: https://visioncamera.margelo.com/docs/depth-output
- Docs — Object Output: https://visioncamera.margelo.com/docs/object-output
- Docs — Async Frame Processing: https://visioncamera.margelo.com/docs/async-frame-processing
- API — `Constraint` type: https://visioncamera.margelo.com/api/react-native-vision-camera/type-aliases/Constraint
- API — `CameraPhotoOutput`: https://visioncamera.margelo.com/api/react-native-vision-camera/hybrid-objects/CameraPhotoOutput
- Related: [capture-and-controls.md](./capture-and-controls.md), [frame-processors.md](./frame-processors.md), [advanced-features.md](./advanced-features.md), [quickstart-v5.md](./quickstart-v5.md)
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |