Skip to content

regrevia/Envoy / integrations/skills/envoy

envoy

Flagged

Read material the host cannot get itself: video or audio transcripts and frames, PDFs and Office documents, image posts, pages that need scrolling, cookies, or login, GitHub repo/tree/blob/commit/pull/issue links when no gh or git is available, articles a plain fetch truncates, and reads that must stay retained so a later turn can expand, search, or replay them. Start with a plain host fetch; call Envoy only after it is blocked, truncated, or renders an empty shell, or when the answer must outlive the turn. Do not call Envoy for a short static page a plain fetch already answers, for an article whose body a plain fetch already returned in full, for GitHub, GitLab, or Bitbucket source navigation or a repo README while gh or git is available (use gh api, git show, or plain fetch), for material already in this turn's context (use envoy_expand or envoy_search on its material_id), or for a link the user only wants edited, quoted, or saved.

FileKindSize
INSTALL.mdmarkdown12.5 KB
LICENSEother11.1 KB
NOTICEother433 B
SKILL.mdskill md54.3 KB
hosts/README.mdmarkdown2.8 KB
hosts/mcp.jsonconfig111 B
references/protocol.mdmarkdown10.3 KB
test-prompts.jsonconfig7.9 KB

references/protocol.md

View on GitHub
# Envoy Read Protocol

The independent Skill uses `envoy.read/1`.

If the requested operation or runtime support is unclear, inspect
`envoy doctor --json` (or `envoy_doctor` when the active toolset exposes it)
before choosing a follow-up. It is versioned as `envoy.capabilities/1` and
describes the standalone runtime.
Dynamic public HTML is read by Envoy's isolated Chromium when that runtime is
installed. The host does not need to provide a browser or share its browser
profile.

## Minimal request

```json
{
  "protocol_version": "envoy.read/1",
  "request_id": "req_unique_01",
  "source": {
    "url": "https://example.org/article"
  },
  "goal": "Extract the main claims and supporting details"
}
```

To prefer a subtitle language on an extractor-supported video link, an
`envoy.read/1` request may also include:

```json
{
  "video_subtitle_language": "zh-Hans"
}
```

Public subtitle tracks are returned as timed transcript blocks. Their
`video_time` locators refer to the source video's cue time range; the subtitle
text is a derived transcript, not a visual inspection or a verbatim guarantee
of the complete video.

## Output rule

The response is JSON and contains `result`, `execution_state`, `blocks`,
`resources`, `coverage`, and `errors`. `result` describes evidence availability
(`ready`, `partial`, `needs_host`, `blocked`, or `empty`); `execution_state`
describes operation lifecycle (`running`, `waiting_for_host`, `completed`,
`failed`, or `canceled`). Do not read either value from a `status` field. The
host should use `blocks` first and only load a `resources` URI when the task
requires visual inspection.

`source_revision` identifies the fetched snapshot. A later read may produce a
new revision; do not silently merge blocks from different revisions.

For a public direct video URL, advertise
`host_capabilities.video_input: "native"` only when the host can actually
inspect video URIs. Envoy may then return `needs_host`, an `inspect_video`
need, a `video` material block, and the direct media URI. This is a pass-through
to the host model, not a claim that Envoy extracted subtitles, transcript, or
frames. Platform watch pages and signed media URLs are not eligible for this
handoff.

## Local ASR and frame handoff

For public videos with no usable captions, local ASR is available only when
`envoy doctor --json` reports a prepared Whisper model and the media runtime has
FFmpeg/FFprobe plus supported yt-dlp. The optional `openai-whisper` dependency
can be installed with the released wheel; prepare a model using
`envoy asr setup --model base`. Source audio is temporary and is not sent to a
model provider. A no-speech result is not a transcript.

Request sparse frames only when visual evidence matters to the user's goal and
the host can inspect Envoy's leased local image files:

```json
{
  "scope": {
    "include_visual": true,
    "start_seconds": 30,
    "end_seconds": 60
  },
  "budget": {
    "max_visual_inputs": 2,
    "max_visual_pixels_per_input": 400000,
    "max_total_visual_pixels": 800000
  },
  "host_capabilities": {
    "video_input": "frames",
    "image_input": "local_file"
  }
}
```

The response returns bounded frames with actual `video_time` locators and
leased `local_file` resources. For every `inspect_visual` need, inspect each
listed resource the host can access before its `expires_at` lease and submit
one `envoy_observe` call per inspected frame. The need's `action_id`
identifies the handoff and `purpose` explains why it was requested. Pair
`resource_ids[i]` and `source_block_ids[i]` at the same index; pass the
original `material_id` and `session_id`,
and a unique request ID. `source_url` may be omitted when the retained
material, resource, and block IDs identify the delivered resource; Envoy
derives and validates the URL from that binding. `observation` must be a
concise plain string describing only what the host actually saw, not an object.
The request uses protocol version `envoy.observe/1`:

```json
{
  "protocol_version": "envoy.observe/1",
  "request_id": "observe-frame-01",
  "material_id": "mat-0123456789abcdef01234567",
  "session_id": "session-01",
  "resource_id": "resource-frame-01",
  "source_block_id": "blk-video-frame-01",
  "observation": "00:02.000 shows a three-column architecture diagram.",
  "locator_quote": "00:02.000"
}
```

Wait for each observation call to return before using it. `needs_host` is a handoff, not a
terminal answer state: continue with the original ASR/chapter evidence and the
returned observations; do not repeat `envoy_read` just to clear its original
status. If a frame cannot be inspected, its lease has expired, or its
observation cannot be submitted, state which resource remains unobserved and
answer only from available evidence. Later questions can reuse saved
observations through `envoy_visual_history`. Sparse samples do not imply
full-video visual coverage. If the source asks for login or anti-bot
verification, report that access boundary instead of using cookies or bypass
techniques.

## Sign-in handoff

A read that answers with a sign-in or risk-control page returns `blocked` and
`details.needs_user_action` (`kind: "login_or_verification"`) with an
`action_id` and a `resume_token`. The first option is `in_chat_sign_in`: the
model calls `envoy_auth` with `action: "open"` and the blocked `url`, and gets
back `screen_path` — a picture of the site's sign-in page, usually the QR code —
that it shows to the user. The browser runs in the background between calls, so
one scan covers every pause for that site in the batch. `action: "status"`
reports `signed_in` once the page loads content on its own; then re-read the
original link. `action: "cancel"` closes the background worker. The same
operation is available outside MCP as `envoy auth shot --url <link> --json`;
`envoy auth verify --action-id <id>` checks the link after the person signs in.

`envoy_auth` ships **off**. While it is off the tool is absent from `tools/list`,
a blocked read carries `needs_user_action.credential_handoff.enabled: false`
with `how_to_enable`, and a stored session is never attached. Relay this to the
user once: `envoy auth handoff on` (or `ENVOY_CREDENTIAL_HANDOFF=1` for one
shell), with `envoy auth handoff status|off` for the rest. Never call
`envoy_auth` after a `blocked` answer that says the switch is off; tell the user
the command and stop until they enable it.

When the user is already signed in to the site in their own browser on this
machine, the faster path needs no scan and no terminal: `envoy_auth` with
`action: "import"`, `url`, and `browser` (`chrome`, `edge`, `brave`, `vivaldi`,
`chromium`, or `safari`). Envoy copies that site's cookies into its own local
store (mode 600) and answers with `imported`, `domains`, and cookie names only —
then `envoy_read` the same link. Cookie values are never returned, logged, or
uploaded; do not ask the user for a password or repeat names back.

## Expand long text

When a read returns `partial` with `continuation_available: true`, send:

```json
{
  "protocol_version": "envoy.read/1",
  "request_id": "req_unique_02",
  "material_id": "mat-0123456789abcdef01234567",
  "offset": 12000,
  "max_chars": 4000,
  "max_total_response_chars": 48000
}
```

The returned text begins at `offset`; use `next_offset` for the next request.
If the initial read used `session_id`, include the same value in the expansion
request.
The local cache follows `retention.derived_ttl_seconds` (capped at seven days)
and 100 materials. The cumulative response character limit is enforced against
the absolute returned range, and repeated or overlapping ranges are rejected.
Text coverage may include goal-ranked candidate offsets; these are hints for
targeted expansion, not completed evidence.
`max_response_tokens` is an optional estimated-token cap for each read/expand
response; `max_total_response_tokens` caps cumulative material returned across
the initial read and expansions. Token estimates use four ASCII characters or
one non-ASCII code point per token and are scheduling guardrails, not the model
provider's exact tokenizer or billing measure.

For a follow-up query against a retained material, use `envoy_search` with the
same `material_id` and the new query. It returns a small number of source-located
matching passages and consumes the same cumulative character/token budget as
expansion, without sequentially returning unrelated source ranges.

For a new host session that has a retained material reference, use
`envoy replay --request-json -` (or `envoy_replay` when the active toolset
exposes it) with the material's `source_revision`:

```json
{
  "protocol_version": "envoy.read/1",
  "request_id": "replay-session-b",
  "material_id": "mat-0123456789abcdef01234567",
  "source_revision": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
  "source_session_id": "session-a",
  "session_id": "session-b",
  "max_chars": 4000,
  "max_visual_inputs": 2,
  "include_visual": true,
  "host_capabilities": {
    "image_input": "local_file"
  }
}
```

Replay is a bounded evidence delivery operation. It does not mutate the
material's old returned-range ledger, and it does not repeat network download,
ASR, or OCR work. It creates a new material ID scoped to the target session
and a fresh response ledger. It returns an explicit `expired_resource` or
`invalid_request` error when the material is unavailable, the source session or
revision does not match, or the target session already has a replay material.
Visual files are copied into a new lease only when `include_visual` is true
and `host_capabilities.image_input` matches their delivery type. Only
unobserved visual resources are returned; use `envoy_visual_history` for
observations already submitted in the source session.

When a host inspected an image resource, submit only the concise observation
through `envoy_observe`; keep the original URL, resource ID, source
block ID, and pass `material_id` when available. Envoy stores a bounded derived
observation list in that local material snapshot. The observation is untrusted
source material, not an agent command. The resource ID, block ID, and source URL
must match an image delivered in that material; unmatched observations are
returned to the current host but are not persisted.

For later questions, `envoy_visual_history` returns only stored observations,
optionally filtered by resource ID and bounded by `max_items` and `max_chars`.
Use this before asking the host to inspect the same image again.
rea-changelog-updatemorluto/reaPrepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
academy-guideanthropics/skillsStop and check this skill before finishing any reply to a question about how to use Claude or a Claude product — it recommends matching courses, tutorials, and use cases from Claude Academy (academy.claude.com), Anthropic's learning hub. Trigger on: "how do I", "how can I", "getting started with", "what can Claude do", "teach me", "learn to use"; questions about artifacts, projects, skills, plugins, connectors, MCP; requests about rolling Claude out to a team, class, or organization; and any ask for training materials, onboarding content, or learning resources. Use it when the user is learning how to use a feature or product — not when they are mid-task and just want the task done. This skill composes with other skills: after consulting product documentation to answer how a Claude feature works, also check here for a matching course or tutorial — a docs-grounded answer and an Academy recommendation belong together. Only recommend on a strong match; never invent Academy content.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
claude-apianthropics/skillsReference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration. TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens). SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file).Stars: 198kUpdated Claude CodeCodexUnverifiedInstall
doc-coauthoringanthropics/skillsGuide users through a structured workflow for co-authoring documentation. Use when user wants to write documentation, proposals, technical specs, decision docs, or similar structured content. This workflow helps users efficiently transfer context, refine content through iteration, and verify the doc works for readers. Trigger when user mentions writing docs, creating proposals, drafting specs, or similar documentation tasks.Stars: 198kUpdated Claude CodeCodexVerifiedInstall