Skip to content

LeoYeAI/agent-master-skills / skills/bengii-gemini-fix

openclaw

Flagged

Comprehensive guide for installing, configuring, operating, and troubleshooting OpenClaw — a self-hosted, multi-channel AI agent gateway. Use when the user asks about OpenClaw setup, configuration, channel management (WhatsApp/Telegram/Discord/Slack/iMessage/etc.), model provider setup, Gateway operations, multi-agent routing, security hardening, troubleshooting, or any maintenance task related to their local OpenClaw installation. Also use when encountering errors from `openclaw` CLI commands or the Gateway daemon.

FileKindSize
README.mdmarkdown7.5 KB
README_CN.mdmarkdown7.3 KB
SKILL.mdskill md18.7 KB
_meta.jsonconfig286 B
references/acp_agents.mdmarkdown12.6 KB
references/agent_runtime.mdmarkdown5.6 KB
references/architecture.mdmarkdown3.8 KB
references/automation.mdmarkdown5.8 KB
references/bonjour.mdmarkdown3.3 KB
references/browser.mdmarkdown8.5 KB
references/channel_routing.mdmarkdown4.0 KB
references/channel_troubleshooting.mdmarkdown11.6 KB
references/channels.mdmarkdown6.2 KB
references/clawhub.mdmarkdown3.6 KB
references/config_reference.mdmarkdown11.2 KB
references/diffs_firecrawl.mdmarkdown3.3 KB
references/elevated.mdmarkdown2.8 KB
references/exec.mdmarkdown6.5 KB
references/exec_approvals.mdmarkdown3.1 KB
references/gateway_internals.mdmarkdown7.7 KB
references/gateway_ops.mdmarkdown9.6 KB
references/heartbeat.mdmarkdown4.9 KB
references/hooks.mdmarkdown14.6 KB
references/install.mdmarkdown4.1 KB
references/llm_task.mdmarkdown2.6 KB
references/lobster.mdmarkdown7.8 KB
references/media.mdmarkdown4.2 KB
references/memory.mdmarkdown11.9 KB
references/model_failover.mdmarkdown3.1 KB
references/multi_agent.mdmarkdown4.4 KB
references/nodes.mdmarkdown8.8 KB
references/openprose.mdmarkdown2.6 KB
references/pairing.mdmarkdown3.1 KB
references/pdf_tool.mdmarkdown3.5 KB
references/platforms.mdmarkdown2.7 KB
references/plugins.mdmarkdown6.9 KB
references/polls.mdmarkdown2.2 KB
references/presence_discovery.mdmarkdown2.2 KB
references/providers.mdmarkdown6.0 KB
references/queue.mdmarkdown2.1 KB
references/remote_access.mdmarkdown4.7 KB
references/sandboxing.mdmarkdown5.6 KB
references/secrets.mdmarkdown5.4 KB
references/security.mdmarkdown24.5 KB
references/sessions.mdmarkdown8.9 KB
references/skills.mdmarkdown4.4 KB
references/slash_commands.mdmarkdown5.1 KB
references/streaming.mdmarkdown1.8 KB
references/subagents.mdmarkdown12.3 KB
references/thinking.mdmarkdown2.8 KB
references/tools.mdmarkdown13.3 KB
references/tui.mdmarkdown4.0 KB
references/voice.mdmarkdown2.7 KB
references/web_tools.mdmarkdown2.7 KB
references/web_ui.mdmarkdown4.5 KB

references/media.md

View on GitHub
# Media (Camera, Images, Audio) Reference

> Sources: https://docs.openclaw.ai/nodes/camera, /nodes/images, /nodes/audio

## Camera Capture

The agent can request camera photos from connected nodes (iOS, Android, macOS).

### Platform Support

| Platform | Default | Notes |
|---|---|---|
| iOS | On | Requires foreground; uses `node.invoke` via Gateway |
| Android | On | Requires camera + storage permissions; foreground required |
| macOS | Off | Uses `node invoke` CLI helper |

### Agent-Initiated Capture
- Agent calls `node.invoke` with camera command via Gateway protocol.
- iOS/Android require the app to be in foreground.
- Captured image is saved to a temp file and injected as media attachment.

### CLI Helper
```bash
openclaw node invoke <nodeId> camera capture
```

### Safety + Practical Limits
- Camera capture requires user opt-in per-node.
- Image resolution and size limits apply per-platform.
- macOS screen video is OS-level, not agent-controlled.

## Image & Media Support

### CLI Surface
```bash
openclaw message send --media <path-or-url> [--message <caption>]
```
- `--media` optional; caption can be empty for media-only sends
- `--dry-run` prints resolved payload
- `--json` emits `{ channel, to, messageId, mediaUrl, caption }`

### WhatsApp Web Behavior

| Media Type | Processing | Max Size |
|---|---|---|
| **Images** | Resize + recompress to JPEG (max side 2048px) targeting `agents.defaults.mediaMaxMb` (default 5 MB) | 6 MB |
| **Audio/Voice/Video** | Pass-through; audio sent as voice note (`ptt: true`) | 16 MB |
| **Documents** | Pass-through, filename preserved | 100 MB |

- **WhatsApp GIF playback**: send MP4 with `gifPlayback: true` (CLI: `--gif-playback`)
- MIME detection: magic bytes → headers → file extension
- Caption from `--message` or `reply.text`; empty caption allowed

### Auto-Reply Pipeline
- `getReplyFromConfig` returns `{ text?, mediaUrl?, mediaUrls? }`
- Multiple media entries sent sequentially

### Inbound Media
- Incoming images/documents are downloaded and attached to the agent prompt.
- Size limits enforced before sending to the model.

## Audio / Voice Notes

### What Works
1. Auto-detect enabled audio transcription providers.
2. Locate first audio attachment (local path or URL).
3. Enforce `maxBytes` per provider.
4. Run first eligible model entry in order (provider or CLI).
5. On success: replace Body with `[Audio]` block and set `{{Transcript}}`.
6. Command parsing: slash commands work from transcript.

### Auto-Detection Priority (Default)
1. **Local CLIs** (if installed):
   - `sherpa-onnx-offline` (requires `SHERPA_ONNX_MODEL_DIR`)
   - `whisper-cli` (whisper-cpp; uses `WHISPER_CPP_MODEL` or bundled tiny)
   - `whisper` (Python CLI; auto-downloads models)
2. **Gemini CLI** (`gemini`) using `read_many_files`
3. **Provider keys**: OpenAI → Groq → Deepgram → Google

### Config Examples

```json5
// Provider + CLI fallback (OpenAI + Whisper CLI)
{
  tools: {
    media: {
      audio: {
        enabled: true,
        models: [
          { provider: "openai", maxBytes: 26214400 },
          { cli: "whisper-cli", maxBytes: 26214400 },
        ],
      },
    },
  },
}
```

```json5
// Provider-only (Deepgram)
{
  tools: {
    media: {
      audio: {
        enabled: true,
        models: [{ provider: "deepgram" }],
      },
    },
  },
}
```

```json5
// Echo transcript to chat (opt-in)
{
  tools: {
    media: {
      audio: {
        echoTranscript: true,
      },
    },
  },
}
```

### Mention Detection in Groups
- Audio transcription enables mention detection in group voice notes.
- If transcript contains agent mention pattern, the message triggers agent response.

### Gotchas
- `tools.media.audio.enabled: false` explicitly disables all transcription.
- Auto-detection probes `PATH` for CLI tools.
- Proxy environment variables respected for provider API calls.

### Environment Variables

| Variable | Purpose |
|---|---|
| `SHERPA_ONNX_MODEL_DIR` | Model dir for sherpa-onnx-offline |
| `WHISPER_CPP_MODEL` | Model path for whisper-cli |
| `DEEPGRAM_API_KEY` | Deepgram API key |
| `OPENAI_API_KEY` | OpenAI Whisper API key |

## See Also

- [voice.md](voice.md) — Talk Mode and Voice Wake
- [nodes.md](nodes.md) — iOS, Android, macOS companion apps
- [channels.md](channels.md) — Channel-specific media limits
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall