hashgraph-online/awesome-codex-plugins / plugins/beastlabai/multi-llm-plugin/skills/multi-llm
multi-llm
FlaggedOrchestrate plan reviews, task generation, implementation and code reviews across several LLM CLIs in parallel, or ask every model one free-text question about a plan. Use when the user says: review my plan, multi-llm, run a code review, generate tasks from a plan, implement this plan, ask the models about my plan.
hosts/render_capabilities.py
View on GitHub#!/usr/bin/env python3
"""Render ``references/host-capabilities.md`` from ``hosts/capabilities.yaml`` (D5b).
The capability matrix is **data with a schema, and the markdown is a build
artifact**. Keeping a prose table by hand would leave the shims,
``install-host --check`` and the tests each re-encoding the same facts, and would
put a table an agent might read back on the read path.
The generated document has a **fixed shape, not an authored one**: one row per host
with the columns the design names, followed by one short per-host section carrying
only the free-form ``notes`` strings and that host's ``env_guards`` names — the one
place a reader can see which variables get stripped from a dispatched runner.
Nothing hand-written survives a render, which is what makes the byte-compare
meaningful rather than a formatting lottery.
The **"Hosting multi-llm in other harnesses" section of ``docs/hosts.md`` is
generated by this same script** (D7), between two markers, for exactly the reason
the reference document is: a hand-maintained second copy of the matrix is a copy
that goes stale, and ``--check`` fails CI when either file differs from a fresh
render. That section used to live in the repo ``README.md``; it moved out when the
README was trimmed to a front page, and only the target path changed.
**A missing host doc is stale, not absent**: ``--check`` fails when
``docs/hosts.md`` does not exist or has lost its markers, and so does a write —
this script fills in the block between the two markers and never creates the
page, because the header above and the footer below them are hand-written. The
bootstrap is therefore: create ``docs/hosts.md`` with both markers plus that
header and footer, then run this script to fill the block.
Usage::
python hosts/render_capabilities.py # write both generated files
python hosts/render_capabilities.py --check # exit 1 if either copy is stale
python hosts/render_capabilities.py --stdout # print the reference doc, write nothing
python hosts/render_capabilities.py --stdout --host-doc # print the docs/hosts.md block
"""
from __future__ import annotations
import argparse
import os
import sys
from pathlib import Path
from typing import Any, Dict, List, Optional, Sequence, Tuple
SKILL_ROOT = Path(os.path.abspath(__file__)).parent.parent
sys.path.insert(0, str(SKILL_ROOT))
from utils import host_capabilities as caps # noqa: E402
from utils.stream_bootstrap import bootstrap_streams # noqa: E402
from utils.unattended import uv_run_args # noqa: E402
#: Where the rendered document lands. A ``references/*.md`` file, and therefore
#: exempt from the host-neutrality lint: naming host tools is its whole job.
OUTPUT = os.path.join("references", "host-capabilities.md")
_HEADER = """<!-- GENERATED FILE — DO NOT EDIT.
Rendered from hosts/capabilities.yaml by hosts/render_capabilities.py.
Edit the YAML and re-render; CI fails if this file differs from a fresh
render, so a hand-edit cannot ship. -->
# Host capabilities
Human-readable view of `hosts/capabilities.yaml`. **Nothing reads this file at run
time** — not an orchestrator, not a shim, and above all not a driving agent. The
YAML is consumed by `multi_llm.py`, by `install-host` and by the tests; this
document exists for people reading the repo.
Every value carries a **provenance**: `verified` means a recorded live smoke run
observed it, `documented` means host docs only, `unknown` is the shipping default.
A `documented`-only value never selects the non-conservative path — which is why
hosts whose native subagent dispatch is merely documented ship as
`subagents: executor`, and why a documented command-timeout ceiling is worked
around with `--detach` rather than trusted.
The three **capability verbs** are the spine of each entry. The host-neutral
instruction core says *spawn subagents* / *ask the user* / *run detached*; the
columns below say what each resolves to, and a shim bakes that resolution in as a
sentence when it is installed.
"""
_MATRIX_HEAD = [
"| Host | Shim (repo → installed) | `arguments` | *spawn subagents* | "
"*ask the user* | *run detached* | Default runner | `nested_exec` "
"(spawn/network/creds) | Max subagents | Timeout ceiling | Verified |",
"|---|---|---|---|---|---|---|---|---|---|---|",
]
def _verb_cell(host: Dict[str, Any], verb: str) -> str:
value = str(host[verb])
tool = (host.get("native_tool") or {}).get(verb)
provenance = host["provenance"].get(verb, "unknown")
cell = f"`{value}`"
if tool:
cell += f" (`{tool}`)"
return f"{cell} — {provenance}"
def _shim_cell(host: Dict[str, Any]) -> str:
shim = host["shim"]
template = shim.get("template")
installed = shim["installed"]
if template is None:
return f"*not rendered* → `{installed}`"
return f"`{template}` → `{installed}`"
def _timeout_cell(host: Dict[str, Any]) -> str:
ceiling = host["timeout_ceiling_s"]
if not ceiling:
return "none documented"
override = host["timeout_override"]
channels = [
f"{name}: `{value}`"
for name, value in override.items()
if value != "none"
]
if not channels:
return f"{ceiling} s (not raisable)"
return f"{ceiling} s (raise via {', '.join(channels)})"
def _provenance_rows(provenance: Dict[str, Any]) -> List[Tuple[str, Any]]:
"""Each provenance row as ``(name, value)``, a map-valued row flattened.
A row whose value is a map (``next_command``: one provenance value per
route) yields one dotted name per key (``next_command.package``). Without
this a promoted route would be a dict, never equal to ``"verified"``, and
the generated page would silently drop it.
"""
rows: List[Tuple[str, Any]] = []
for field, value in provenance.items():
if isinstance(value, dict):
rows.extend((f"{field}.{key}", inner) for key, inner in value.items())
else:
rows.append((field, value))
return rows
def _verified_cell(host: Dict[str, Any]) -> str:
verified = sorted(
field for field, value in _provenance_rows(host["provenance"])
if value == "verified"
)
return ", ".join(f"`{f}`" for f in verified) if verified else "*none yet*"
def _matrix(hosts: Dict[str, Any], base: Optional[str] = None) -> List[str]:
rows = list(_MATRIX_HEAD)
for host_id in sorted(hosts):
host = hosts[host_id]
rows.append(
"| `{id}` | {shim} | `{args}` | {sub} | {ask} | {det} | `{runner}` | "
"{nested} | {maxc} | {timeout} | {verified} |".format(
id=host_id,
shim=_shim_cell(host),
args=host["arguments"],
sub=_verb_cell(host, "subagents"),
ask=_verb_cell(host, "ask"),
det=_verb_cell(host, "detach"),
runner=host["default_subagent_runner"],
nested="/".join(
str(host["nested_exec"][key])
for key in ("spawn", "network", "credentials")
),
maxc=host["max_concurrent_subagents"],
timeout=_timeout_cell(host),
verified=_verified_cell(host),
)
)
return rows
def _sections(hosts: Dict[str, Any], base: Optional[str] = None) -> List[str]:
lines: List[str] = ["", "## Per-host notes and stripped variables", ""]
for host_id in sorted(hosts):
host = hosts[host_id]
lines.append(f"### `{host_id}`")
lines.append("")
lines.append(
f"- Installable: `{str(host['installable']).lower()}`; "
f"reload after install: `{host['reload']}`"
)
lines.append(
f"- Invocation ({host['provenance']['invocation']}): invoke it "
f"{host['invocation']}"
)
lines.append(
"- Driving agent gets a turn after a command returns "
f"(`multi_turn`): `{str(host['multi_turn']).lower()}` — "
f"{host['provenance']['multi_turn']}. A `false` here is what "
"addresses a failed batch's retry instruction to the human rather "
"than to an agent that will never read it."
)
tiers = host["subagent_tiers"]
if tiers is None:
lines.append(
"- Subagent tiers: resolved from `providers.yaml` "
"`driver.subagent_runner{,_cheap}` (this host does not dispatch "
"subagents natively)"
)
else:
cheap = tiers.get("cheap", tiers["full"])
lines.append(
f"- Subagent tiers: full `{tiers['full']}`, cheap `{cheap}`"
)
lines.append("- Verb sentences a rendered shim carries:")
for sentence in caps.verb_sentences(host_id, base):
lines.append(f" - {sentence}")
for scope in ("project", "global", "legacy"):
paths = host["install_paths"][scope]
rendered = ", ".join(f"`{p}`" for p in paths) if paths else "*none*"
lines.append(f"- Install paths ({scope}): {rendered}")
runners = host["runners"]
lines.append("- Runners:")
for name in sorted(runners):
spec = runners[name]
lines.append(
f" - `{name}`: bundled=`{str(spec['bundled']).lower()}`, "
f"needs_auth=`{str(spec['needs_auth']).lower()}`"
+ (f" (`{spec['login_cmd']}`)" if spec["login_cmd"] else "")
+ f", profiles=[{', '.join(spec['profiles'])}]"
+ f", usage=`{spec['usage']}`"
)
guards = host["env_guards"]
for key, label in (
("guards", "Refuse-to-start guards stripped from a dispatched runner"),
("session_scoped", "Session-scoping variables stripped"),
("passthrough", "Sanctioned exceptions that survive stripping"),
):
names = guards[key]
rendered = ", ".join(f"`{n}`" for n in names) if names else (
"*none recorded* — "
+ (
"audited, found nothing"
if host["provenance"]["env_guards"] != "unknown"
else "the env diff has not run yet"
)
)
lines.append(f"- {label}: {rendered}")
notes = host["nested_exec"]["notes"].strip()
if notes:
lines.append(f"- Nested-exec notes: {' '.join(notes.split())}")
lines.append("")
return lines
def render(base: Optional[str] = None) -> str:
"""The whole document, deterministically ordered."""
hosts = caps.load(base).get("hosts", {})
body = [_HEADER, ""]
body += _matrix(hosts, base)
body += _sections(hosts, base)
return "\n".join(body).rstrip("\n") + "\n"
# ---------------------------------------------------------------------------
# The host-install document (D7) — generated, never hand-maintained
# ---------------------------------------------------------------------------
#: Where the generated host-install section lands, relative to the repo (and
#: published-plugin) root — the directory holding `.claude-plugin/`, two levels up
#: from `skills/multi-llm/`. The markers, not the filename, are what this script
#: keys on, so moving the section again is a one-line change here.
HOST_DOC = os.path.join("docs", "hosts.md")
#: The block this script owns. Everything between the markers is regenerated; the
#: markers themselves are the only hand-written part, and the ``--check`` guard is
#: what keeps a hand-edit inside them from shipping.
HOST_DOC_BEGIN = "<!-- BEGIN GENERATED: host-install (hosts/render_capabilities.py) -->"
HOST_DOC_END = "<!-- END GENERATED: host-install -->"
def host_doc_path() -> Path:
return SKILL_ROOT.parent.parent / HOST_DOC
def _install_path_cell(paths: List[str]) -> str:
return ", ".join(f"`{p}`" for p in paths) if paths else "—"
def _shell_dispatcher() -> str:
"""The dispatcher command **as this page teaches it**, flags included.
``$SKILL`` stays a shell variable — the reader sets it themselves a block
up, which is why :func:`utils.cli_common.dispatcher_prefix` (it resolves and
quotes a real skill dir) cannot be called here. The *flags* still come from
the one owner, :func:`utils.unattended.uv_run_args`, rather than being
hand-typed a second time: spelled by hand this page shipped a bare
``uv run --project``, so the first command a new user ran was the one
command in the project that omitted ``--no-dev`` and performed the whole
dev-group sync.
"""
flags = "".join(f" {flag}" for flag in uv_run_args())
return f'uv run{flags} --project "$SKILL" -- python "$SKILL/multi_llm.py"'
#: How each registry's install is spelled, and what "upgrading" means on it.
#: Keyed by the `package.registry` discriminator rather than by host id, so a
#: second host on the same registry inherits the wording and a second registry
#: is one row rather than a branch.
#:
#: **Every registry-specific sentence of the paragraph lives here**, never in
#: :func:`_package_section`: the install intro (``config``), the install
#: commands (``fence``, one template per command, rendered in order into one
#: fence), the provenance blurbs (``verified`` / ``documented``), and — where the
#: upgrade is a different command sequence from the install — ``upgrade`` plus
#: ``upgrade_fence``. Each is ``.format()``ed from :func:`_format_keywords`, so a
#: slug, a ref or a sparse path is a ``package:`` cell rather than a literal a
#: later package on the same registry would inherit.
_REGISTRY_FORMS: Dict[str, Dict[str, Any]] = {
"npm": {
"config": (
'Add it to the host config (`~/.config/{host}/{host}.json`) and '
"restart:"
),
"fence": ['{{ "{config_key}": ["{spec}"] }}'],
"language": "json",
"verified": (
"The recommended route on `{host}`: one config edit, no shell "
"bootstrap, and no baked path that goes stale on an update — the "
"plugin resolves its own location at every startup."
),
"documented": (
"A second route on `{host}`, and the one this documentation "
"will recommend once it is published. **No published install has "
"been verified yet**, so a config entry naming this spec resolves "
"to nothing today — and on this host that failure is silent: the "
"plugin stays listed, the exit code is 0, and the debug log is "
"empty. Until then, use the `install-host` route above, or build "
"the package and name its **directory** — `npm pack`'s `.tgz` is "
"not an installable spec on this host: MEASURED 2026-09-19 against "
"opencode 1.18.31, a config entry naming a tarball resolves to the "
"archive's *containing directory* and never opens the archive, so "
"it loads nothing unless that directory happens to be an unpacked "
"package itself."
),
# The package's own copy of the provider config lives in the
# spec-keyed npm cache, which a pin bump replaces, so the pointer is
# `--init`, never the installed file.
"configure": (
"Then configure providers once per repository: `/multi-llm --init` "
"writes `<git-root>/.multi-llm/providers.yaml`, outside the "
"package cache, where no upgrade touches it."
),
},
"codex-marketplace": {
"config": (
"Install it from the marketplace — two commands, in this order, "
"then start a **new** {host} session:"
),
# R30's measured order: the marketplace first, then the plugin from it.
"fence": [
"{host} plugin marketplace add {repo} --ref {ref} {sparse_flags}",
"{host} plugin add {name}@{marketplace}",
],
"language": "bash",
# The namespaced name the plugin's skill is advertised under — a worked
# example of its own, because it is NOT the `install-host` paragraph's
# bare name, and a reader copying that one would name the file shim.
# A `$` mention, which codex's own skill instructions name as the way a
# user invokes a skill ("with $SkillName or plain text"); observed live
# on 0.156.1, the arguments reaching the `multi_llm` tool intact.
"examples": [
"${name}:{name} plans/my-feature.md",
"${name}:{name} --review-code plans/my-feature.md",
],
# Once per repository, in the new session. The plugin's own copy of the
# provider config lives in the version-keyed install, which the next
# `plugin add` replaces wholesale — so the pointer is `--init`, never
# the installed file.
"configure": (
"Then configure providers once per repository, in the new session: "
"`${name}:{name} --init` "
"writes `<git-root>/.multi-llm/providers.yaml`, outside "
"`$CODEX_HOME`, where no reinstall or upgrade touches it."
),
# Not the install re-run: adding a marketplace that is already added is
# a no-op (R37), so a reader told to repeat the install would re-install
# the version they already had and be told nothing.
"upgrade": (
"Upgrading is a different sequence from the install, because "
"repeating the install is a no-op on a marketplace that is already "
"registered: refresh the marketplace, re-add the plugin from it, then "
"start a **new** {host} session:"
),
"upgrade_fence": [
"{host} plugin marketplace upgrade",
"{host} plugin add {name}@{marketplace}",
],
"verified": (
"The recommended route on `{host}`: two commands in this order, no "
"config file edited by hand, and no baked path that goes stale on an "
"update — the plugin is a copy of the `{ref}` ref, installed where "
"{host} resolves it."
),
"documented": (
"A second route on `{host}`, and the one this documentation will "
"recommend once it is published. **It is not yet published**: the "
"`{ref}` ref the first command names does not exist until the "
"publish workflow first moves it, so today `{host} plugin "
"marketplace add … --ref {ref}` fails **loudly**, with a git error "
"naming the missing ref — nothing is half-installed. Until then, the "
"`install-host {host}` route above is what works."
),
},
}
#: What ``update`` means, in words, per enum value. Registry-neutral words only:
#: a registry whose row carries its own ``upgrade`` text renders none of these,
#: so the neutral sentence can neither contradict that row's fence nor be
#: deleted out from under a later host that selects it.
_UPDATE_FORMS: Dict[str, str] = {
"pinned-spec": (
"Upgrading is bumping the pin and restarting: the cache key is the "
"literal spec string and the installer early-returns when the package "
"directory already exists, so an **unpinned** entry resolves once and "
"never again"
),
"reinstall": (
"Upgrading re-installs the package through the host's own upgrade "
"command — not by repeating the install, which a host may treat as "
"already done"
),
"range": "The host re-resolves the declared range on its own",
}
#: What ``write_posture`` means, in words. Rendered *above* the install fence,
#: so a reader meets it before running the install rather than after.
_WRITE_POSTURE_FORMS: Dict[str, str] = {
"refuses-under-readonly": (
"**Read this before installing.** The plugin's tool runs the "
"orchestrator from a server {host} starts *outside* its own exec "
"sandbox, so the session's `--sandbox` posture does not govern it: "
"installing the plugin gives a {host} session a write path it did not "
"have. So under a read-only posture, or one it cannot read, the tool "
"**refuses write-capable modes** before it spawns anything, unless "
"`MULTI_LLM_{host_upper}_UNSANDBOXED_WRITES=1` is in the environment "
"that starts {host}. Despite its name, "
"`MULTI_LLM_{host_upper}_UNSANDBOXED_WRITES=1` lifts only this tool's "
"own refusal of write-capable modes under a read-only or unreadable "
"session posture. It never lets the {host} processes the orchestrator "
"starts write without a sandbox, and it does not change which sandbox "
"they run under: a {host} editing job refused because {host}'s own "
"sandbox cannot start on this host stays refused with it set. Provider "
"CLIs the orchestrator dispatches stay under their own sandboxes."
),
}
def _format_keywords(host_id: str, host: Dict[str, Any]) -> Dict[str, str]:
"""Every keyword a registry row's templates may name, from the block's cells.
The spec's version is :func:`caps.package_spec_version`'s answer, never a
cell read here: the candidate ``version`` before the route is verified, the
trailing ``public_version`` once it is, so a bump never puts an unpublished
version into a recommendation. A pin that disagrees with its provenance row
refuses the render rather than rendering a plausible wrong spec.
"""
package = host["package"]
try:
pinned = caps.package_spec_version(host)
except caps.PackagePinError as exc:
raise SystemExit(f"host '{host_id}': {exc}") from exc
spec = package["name"]
if pinned:
spec = f"{spec}@{pinned}"
return {
"config_key": package["config_key"],
"spec": spec,
"host": host_id,
"host_upper": host_id.upper().replace("-", "_"),
"name": package["name"],
"marketplace": package.get("marketplace", ""),
"repo": package.get("repo", ""),
"ref": package.get("ref", ""),
# One `--sparse <path>` per element, so the list stays data.
"sparse_flags": " ".join(
f"--sparse {path}" for path in package.get("sparse", [])
),
}
def _package_section(host_id: str, host: Dict[str, Any]) -> List[str]:
"""The "Install via <registry>" paragraph, or nothing.
**Its wording is a function of the provenance row**, which is what keeps the
pre- and post-publish documentation one generated paragraph rather than two
hand-written ones. At ``documented`` it names the route and says plainly
that no published install has been verified; at ``verified`` the same
paragraph reads as the recommended install. Flipping it is a provenance edit
(with ``public_version`` set in the same edit, which the schema requires)
and a re-render, never a prose rewrite — and the two generated pages are
byte-compared, so a hand edit is a CI failure. The pinned spec follows the
same row: the candidate ``version`` under the unpublished notice, the last
registry-checked ``public_version`` under the recommendation.
**This function holds no registry-specific sentence.** Each one is a
template in the registry's :data:`_REGISTRY_FORMS` row, so the npm prose
can never leak into a marketplace paragraph (or the reverse).
"""
package = host.get("package")
if not package:
return []
registry = package["registry"]
form = _REGISTRY_FORMS.get(registry)
if form is None:
raise SystemExit(
f"host '{host_id}' declares registry '{registry}', which this "
"renderer has no form for. Add a row to _REGISTRY_FORMS rather than "
"letting the paragraph render empty."
)
provenance = host["provenance"].get("package", "unknown")
keywords = _format_keywords(host_id, host)
verified = provenance == "verified"
lines = [
f"**`{host_id}` — install via {registry}"
+ ("" if verified else " (not yet published)")
+ f"** — {provenance}",
"",
form["verified" if verified else "documented"].format(**keywords),
"",
]
# Ahead of the install fence, so a reader meets it before running it.
posture = package.get("write_posture")
if posture:
lines += [_WRITE_POSTURE_FORMS[posture].format(**keywords), ""]
lines += [
form["config"].format(**keywords),
"",
f"```{form['language']}",
]
lines += [command.format(**keywords) for command in form["fence"]]
lines += ["```", ""]
# A registry row that spells its own upgrade renders no neutral sentence.
update = "" if "upgrade" in form else _UPDATE_FORMS[package["update"]] + ". "
lines += [
f"Needs {host_id} {package['min_host_version']} or newer. "
+ update
+ f"It provides {_provides_sentence(package['provides'])}, and you "
f"invoke it {package['invocation']}",
"",
]
if "examples" in form:
lines += ["```text"]
lines += [example.format(**keywords) for example in form["examples"]]
lines += ["```", ""]
if "configure" in form:
lines += [form["configure"].format(**keywords), ""]
if "upgrade" in form:
lines += [
form["upgrade"].format(**keywords),
"",
f"```{form['language']}",
]
lines += [command.format(**keywords) for command in form["upgrade_fence"]]
lines += ["```", ""]
# No worked-example fence of its own *unless the registry row carries one*.
# On npm the per-host block this paragraph is appended to has already
# rendered one from this same `provides`, and a second byte-identical copy
# one paragraph later reads as a *different* example that happens to
# coincide. The codex marketplace row does carry one, because its skill is
# advertised under a namespaced name the file shim's fence does not spell.
# The half of `provides` no fence can show — a model-callable tool has no
# typed spelling — is named in the invocation sentence above instead.
return lines
def _provides_sentence(provides: Sequence[str]) -> str:
"""``provides`` as prose, in the declared order."""
spelled = {
"command": "a slash command",
"skill": "a skill",
"tool": "a model-callable tool",
}
items = [spelled[item] for item in provides]
if len(items) == 1:
return items[0]
return ", ".join(items[:-1]) + " and " + items[-1]
def _worked_examples(provides: Sequence[str]) -> List[str]:
"""The fence style, **selected from ``provides``** rather than sniffed.
This used to be a substring test over the invocation prose — `startswith("as
`/")` for the slash form, `"skill" in ...` otherwise — which made the fence a
function of how a sentence happened to be phrased. ``provides`` is data that
says which it is.
"""
if "command" in provides:
return [
"```text",
"/multi-llm plans/my-feature.md",
"/multi-llm --review-code plans/my-feature.md",
"```",
"",
]
if "skill" in provides:
return [
"```text",
"Use the multi-llm skill with exactly these arguments: "
"plans/my-feature.md",
"Use the multi-llm skill with exactly these arguments: "
"--review-code plans/my-feature.md",
"```",
"",
]
return []
def _reload_sentence(host_id: str, host: Dict[str, Any]) -> str:
value = str(host["reload"])
if value == "none":
return (
"no reload step — this shim ships with the plugin, so a plugin update "
"is the refresh"
)
if value == "restart":
return f"restart {host_id} afterwards (it reads its config once, at startup)"
return (
f"start a new {host_id} session afterwards (it enumerates its "
"commands/skills at session start)"
)
def _installable_prose(host_ids: List[str]) -> str:
"""The installable hosts in prose: `a`, then `a` and `b`, then `a`, `b` and `c`.
Read off the same list the quickstarts iterate, so a host added to the data
joins the opening sentence on the day it is added rather than leaving a
hand-written "Codex and OpenCode" behind to go stale.
Its contract is the *list of host names*, never the sentence around them —
which is why the empty case returns a bare noun phrase and why
:func:`_opening_sentence` does not use it there: interpolating "no other
host yet" into "runs unchanged inside … as well" states something false,
and the repair belongs to the sentence, not to this helper.
"""
marks = [f"`{host_id}`" for host_id in host_ids]
if len(marks) <= 1:
return marks[0] if marks else "no other host yet"
return f"{', '.join(marks[:-1])} and {marks[-1]}"
def _opening_sentence(host_ids: List[str]) -> str:
"""The host doc's first sentence — one shape per arity, all four true.
With at least one installable host it is the D4 sentence, with the host
list rendered by :func:`_installable_prose` so a fourth host joins it from
the data alone. With **none** it is a different sentence rather than the
same one with an empty slot: flipping every ``installable:`` in
``hosts/capabilities.yaml`` to ``false`` is a data-only change, and it must
not be able to ship "it runs unchanged inside no other host yet as well".
"""
if not host_ids:
return (
"multi-llm ships as a Claude Code plugin; no other host is "
"installable yet."
)
return (
"multi-llm ships as a Claude Code plugin, and it runs unchanged inside "
f"{_installable_prose(host_ids)} as well."
)
def render_host_doc_section(base: Optional[str] = None) -> str:
"""The `docs/hosts.md` block, generated from the same data as everything else."""
hosts = caps.load(base).get("hosts", {})
installable = [h for h in sorted(hosts) if hosts[h]["installable"]]
# Every `uv run …` line on this page is this one string plus its arguments.
dispatch = _shell_dispatcher()
lines: List[str] = [
"# Hosting multi-llm in other harnesses",
"",
f"{_opening_sentence(installable)} The workflow itself is "
"host-neutral: the instruction core names three capability *verbs* "
"(*spawn subagents*, *ask the user*, *run detached*) and no tool, and one "
"dispatcher command runs every mode. What a host needs is therefore just "
"an **entry shim** — a small file, in that host's own command/skill "
"format, with this skill's absolute path and its `--host` id baked in. "
"`install-host` renders and writes it:",
"",
"```bash",
"# from anywhere; SKILL is this skill tree on disk",
f"{dispatch} install-host <host> [--global|--project]",
"```",
"",
"`SKILL` is either a git checkout of this repository or the plugin "
"install `claude plugin list --json` reports. Set it once — the first of "
"these three that applies to you, not all three. On Windows the shell "
"must be **Git Bash** (see [Prerequisites](../README.md#15-prerequisites)):",
"",
"```bash",
"# Set SKILL once - the first of these three that applies to you, not all three.",
"",
"# 1. You cloned this repository:",
"SKILL=<checkout>/skills/multi-llm",
"",
"# 2. You installed the plugin: ask Claude Code where it put it (needs `jq`). Installs of the",
"# same plugin coexist per project and per version, and versions older than the dispatcher",
"# are still in the cache - so prefer enabled and most-recently-updated, then take the first",
"# that actually carries multi_llm.py:",
"SKILL=$(claude plugin list --json \\",
" | jq -r 'map(select(.id == \"multi-llm@beastlabai\")) | "
"sort_by([.enabled, .lastUpdated]) | reverse | .[].installPath' \\",
" | while IFS= read -r p; do [ -f \"$p/skills/multi-llm/multi_llm.py\" ] "
"&& { echo \"$p/skills/multi-llm\"; break; }; done)",
"",
"# 3. Same, but no `jq` on this machine: glob the cache, whose path changes on every update.",
"# The marketplace segment is pinned to the one `/plugin install` used in the README - change it if",
"# you installed from a fork:",
"SKILL=$(dirname \"$(ls -t ~/.claude/plugins/cache/beastlabai/multi-llm"
"/*/skills/multi-llm/multi_llm.py | head -1)\")",
"",
"# Then install, gated on SKILL: an unresolved SKILL is empty (2) or the current directory (3),",
"# and either would run install-host against the wrong tree.",
"if [ -f \"$SKILL/multi_llm.py\" ]; then",
]
for host_id in installable:
extra = (
" # add --project to scope it to one repo"
if host_id == installable[0]
else " # add --check to verify an install"
)
lines.append(f" {dispatch} install-host {host_id}{extra}")
lines += [
"else",
" echo \"no multi-llm skill tree at '$SKILL' - install the plugin, or "
"set SKILL=<checkout>/skills/multi-llm\" >&2",
"fi",
"```",
"",
"It writes a **rendered copy** — never a symlink or junction, so nothing "
"here needs symlink privileges on Windows — atomically, then verifies what "
"it wrote and rolls back if the verification fails. `--check` re-runs that "
"verification on its own: shim present (every copy found, in the host's "
"discovery order, naming the one that loads), baked skill-dir prefix still "
"current, rendered shape valid, shim generation current, and `uv` plus the "
"host's runner CLI resolvable.",
"",
"## Per-host quickstart",
"",
]
for host_id in installable:
host = hosts[host_id]
lines += [
f"**`{host_id}`**",
"",
"```bash",
f"{dispatch} install-host {host_id} # machine-wide",
f"{dispatch} install-host {host_id} --project # just this repository",
f"{dispatch} install-host {host_id} --check # verify an install",
"```",
"",
f"Writes `{host['install_paths']['global'][0]}/"
f"{os.path.basename(host['shim']['installed'])}`"
+ (
" (whose `$XDG_CONFIG_HOME` falls back to `~/.config` when unset)"
if "XDG_CONFIG_HOME" in host["install_paths"]["global"][0]
else ""
)
+ f" (`--project`: `{host['install_paths']['project'][0]}/"
f"{os.path.basename(host['shim']['installed'])}`), then "
f"{_reload_sentence(host_id, host)}. Invoke it "
# Per host, from the data: the two installable hosts do NOT take the
# same spelling. A constant here published `/multi-llm …` for codex,
# whose composer answers it with "Unrecognized command" — measured
# 2026-08-24, and the reason this reads `invocation` instead.
f"{host['invocation']} Unattended is the default on every host but "
"Claude Code, so `--yes` is baked in.",
"",
]
# Only where the obvious headless form is wrong in a way that fails
# confusingly rather than loudly — see the field's schema description.
if host.get("headless_invocation"):
lines += [
f"Headless, the argument list needs the host's own separator: "
f"`{host['headless_invocation']}`.",
"",
]
# Worked examples moved from the README: the same arguments the Claude
# Code skill takes, in the spelling this host actually accepts. The
# fence style comes from `provides` on the package block where one
# exists, and otherwise from the invocation prose, which is what the
# file-only hosts still have to go on.
package = host.get("package")
if package:
lines += _worked_examples(package["provides"])
elif host["invocation"].lstrip().startswith("as `/"):
lines += _worked_examples(["command"])
elif "skill" in host["invocation"].lower():
lines += _worked_examples(["skill"])
# And, where a package route exists, how to install it that way.
lines += _package_section(host_id, host)
runner_clause = " and ".join(
f"`{hosts[h]['default_subagent_runner']}` on `{h}`" for h in installable
) or "the host's default runner"
# `--interactive` is a *file-route* affordance, and saying so is the whole
# point of this clause: on a package route the plugin refuses `--interactive`
# and `-i` before it spawns anything (nothing is on the dispatcher's stdin
# there, so the picker they force could not prompt), and a reader who
# arrived through that route would otherwise be handed the one instruction
# that cannot work for them. Derived from the data, so it disappears on a
# tree whose hosts declare no package rather than becoming a stale caveat.
packaged = [h for h in installable if hosts[h].get("package")]
opt_back = (
" Pass `--interactive` to opt back into a rendered gate."
if not packaged
else (
" On the `install-host` file route, `--interactive` opts back into "
"a rendered gate. The package route on "
f"{_installable_prose(packaged)} refuses `--interactive` and `-i` "
"instead — it spawns the dispatcher with nothing on its stdin, so "
"the model picker they force could not prompt — and asks you to "
"choose non-interactively: `--models \"provider:model,…\"` or "
"`--quick`, or `defaults.models` in `providers.yaml`."
)
)
differ: List[str] = [
"## What differs from Claude Code",
"",
"Three differences worth knowing on these hosts:",
"",
"- **Unattended is the default.** Claude Code is the only host with a "
"structured question tool, so the shim bakes in `--yes`: every gate "
"resolves from config and flags instead of prompting you." + opt_back,
"- **Subagents are dispatched through a provider CLI** rather than a "
"native task tool, so the default runner is "
f"{runner_clause} — not a Claude subagent.",
]
for host_id in installable:
headless = hosts[host_id].get("headless_invocation")
if headless:
differ.append(
f"- **Headless `{host_id}` needs its own separator:** "
f"`{headless}`."
)
lines += differ + [
"",
"## Capability matrix",
"",
"Generated from `skills/multi-llm/hosts/capabilities.yaml` — the same data "
"`install-host` renders a shim from. Every value carries a **provenance**: "
"`verified` means a recorded live run observed it, `documented` means host "
"docs only, `unknown` is the shipping default, and a `documented`-only "
"value never selects the non-conservative path (which is why hosts whose "
"native subagent dispatch is merely documented ship as `executor`).",
"",
"| Host | Installable | Command format | Args | *spawn subagents* | "
"*ask the user* | *run detached* | Default runner | Reload |",
"|---|---|---|---|---|---|---|---|---|",
]
for host_id in sorted(hosts):
host = hosts[host_id]
lines.append(
"| `{id}` | {inst} | `{fmt}` | `{args}` | {sub} | {ask} | {det} | "
"`{runner}` | `{reload}` |".format(
id=host_id,
inst="yes" if host["installable"] else "no (ships with the plugin)",
fmt=host["shim"]["format"],
args=host["arguments"],
sub=_verb_cell(host, "subagents"),
ask=_verb_cell(host, "ask"),
det=_verb_cell(host, "detach"),
runner=host["default_subagent_runner"],
reload=host["reload"],
)
)
lines += [
"",
"## Where the shim goes",
"",
"One row per host **and scope**, not one path per host. `install-host` "
"writes only the preferred path of the scope you asked for; it *detects and "
"reports* the rest, so an existing install is upgraded rather than silently "
"shadowed. Compatibility locations stay valid inputs and are never written: "
"migrating is re-run `install-host`, then delete the file it names. `~` is "
"`%USERPROFILE%` on Windows and the XDG roots honour `$XDG_CONFIG_HOME`; "
"the tokens below are what the data stores.",
"",
"| Host | Global (preferred) | Project (preferred) | "
"Compatibility (detected, never written) |",
"|---|---|---|---|",
]
for host_id in sorted(hosts):
host = hosts[host_id]
paths = host["install_paths"]
if not host["installable"]:
lines.append(
f"| `{host_id}` | *n/a* — the plugin's own "
f"`{host['shim']['installed']}` | *n/a* | "
f"{_install_path_cell(paths['project'] + paths['global'])} "
"(non-plugin install) |"
)
continue
lines.append(
f"| `{host_id}` | {_install_path_cell(paths['global'])} | "
f"{_install_path_cell(paths['project'])} | "
f"{_install_path_cell(paths['legacy'])} |"
)
lines += [
"",
"## After a plugin update",
"",
"A plugin update **changes the skill directory path** — the Claude plugin "
"cache is keyed by commit hash — so every installed shim's baked prefix "
"goes stale. That is the common case, not an edge case, and it is one "
"command:",
"",
"```bash",
f"{dispatch} install-host --check --all # what moved",
f"{dispatch} install-host --all # re-render every installed "
"shim",
"```",
"",
"Repair is always re-render, never in-place patching, and a re-install with "
"nothing changed is byte-identical and does nothing. A shim you edited by "
"hand is **never** overwritten silently: the install stops and names the "
"file, and `--force` keeps your version as `<shim>.local.bak`. The inverse "
"is `uninstall-host <host> [--global|--project|--all] [--dry-run]`, which "
"removes only files carrying our `multi_llm_shim` block — identified by "
"that block, never by filename.",
]
return "\n".join(lines) + "\n"
def _bootstrap_hint() -> str:
"""How to create the page, since this script never creates it itself.
The renderer owns the block *between* the markers and nothing else: the
comment header above ``HOST_DOC_BEGIN`` and the footer link row below
``HOST_DOC_END`` are hand-written and have no source to be rendered from.
So both failure messages below end in the same instruction rather than in a
silent skip or a conjured file.
"""
return (
f"Create {HOST_DOC} carrying both markers —\n"
f" {HOST_DOC_BEGIN}\n {HOST_DOC_END}\n"
"— with the hand-written header above the first and the footer link "
"row below the second, then re-run `python "
"hosts/render_capabilities.py` to fill the block."
)
def _missing_markers_message() -> str:
return (
f"ERROR: {HOST_DOC} carries no generated-host-install markers. Add\n"
f" {HOST_DOC_BEGIN}\n {HOST_DOC_END}\n"
"where the section belongs, then re-run this script."
)
def host_doc_problem(host_doc: Optional[Path] = None) -> Optional[str]:
"""Why the host doc cannot be rendered into, or ``None`` when it can.
Absence and a marker-less file are **one** class of failure — there is no
block for this script to own — and both are reported, never repaired.
Skipping either was safe only while the target was the always-present
``README.md``; against ``docs/hosts.md`` a skip lets ``--check`` print
"up to date" for a tree carrying no host-install page at all, which is the
one thing that guard exists to catch.
"""
host_doc = host_doc_path() if host_doc is None else host_doc
if not host_doc.is_file():
return (
f"ERROR: {HOST_DOC} does not exist ({host_doc}). This script fills "
"in the block between two markers; it never creates the page. "
+ _bootstrap_hint()
)
body = host_doc.read_text(encoding="utf-8")
if HOST_DOC_BEGIN not in body or HOST_DOC_END not in body:
return _missing_markers_message()
return None
def host_doc_with_section(current: str, base: Optional[str] = None) -> str:
"""*current* host doc with the generated block replaced (markers preserved)."""
section = render_host_doc_section(base)
block = f"{HOST_DOC_BEGIN}\n\n{section}\n{HOST_DOC_END}"
start = current.find(HOST_DOC_BEGIN)
end = current.find(HOST_DOC_END)
if start < 0 or end < 0:
# Reachable only for a caller that reads the file itself; `main()` asks
# `host_doc_problem()` first, so a CLI run reports this as a failure
# rather than unwinding a SystemExit through it.
raise SystemExit(_missing_markers_message())
return current[:start] + block + current[end + len(HOST_DOC_END):]
def main(argv: Optional[Sequence[str]] = None) -> int:
bootstrap_streams()
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--check", action="store_true",
help="Exit 1 if either generated copy differs from a fresh render "
"(the CI staleness guard).",
)
parser.add_argument(
"--stdout", action="store_true", help="Print the render; write nothing."
)
parser.add_argument(
"--host-doc", "--readme", dest="host_doc", action="store_true",
help=f"Operate on {HOST_DOC}'s generated host-install section instead of "
"references/host-capabilities.md. (`--readme` is the former spelling, "
"kept working for anything that still types it.)",
)
args = parser.parse_args(list(argv) if argv is not None else None)
if args.stdout:
sys.stdout.write(
render_host_doc_section() if args.host_doc else render()
)
return 0
text = render()
target = SKILL_ROOT / OUTPUT
host_doc = host_doc_path()
if args.check:
failures = []
current = target.read_text(encoding="utf-8") if target.exists() else ""
if current != text:
failures.append(OUTPUT)
# A missing or marker-less host doc is a failure, not a skipped step:
# "the install page for two of the three hosts does not exist" must
# never render as "up to date". Reported with its own message, because
# "re-run the renderer and commit" is not the fix for either.
problem = host_doc_problem(host_doc)
if problem is None:
body = host_doc.read_text(encoding="utf-8")
if host_doc_with_section(body) != body:
failures.append(HOST_DOC)
if not failures and problem is None:
print(f"{OUTPUT} and {HOST_DOC} are up to date.")
return 0
if failures:
print(
f"ERROR: {', '.join(failures)} stale. Generated from "
"hosts/capabilities.yaml — re-run `python "
"hosts/render_capabilities.py` and commit the result.",
file=sys.stderr,
)
if problem is not None:
print(problem, file=sys.stderr)
return 1
if not args.host_doc:
target.write_text(text, encoding="utf-8")
print(f"wrote {OUTPUT}")
# The write path **refuses** rather than creating the page or skipping it.
# This script keys on the markers, not on the filename (see HOST_DOC), so a
# write into a file that has neither would have nowhere to land — and a
# write into a file that does not exist would have to invent the
# hand-written header and footer around them. Both end at the same
# bootstrap instruction, and both are exit 1: a first render that silently
# did nothing is exactly the failure `--check` above is being taught to
# catch, and it must not be reachable from the write path either.
problem = host_doc_problem(host_doc)
if problem is not None:
print(problem, file=sys.stderr)
return 1
body = host_doc.read_text(encoding="utf-8")
host_doc.write_text(host_doc_with_section(body), encoding="utf-8")
print(f"wrote {HOST_DOC} (generated host-install section)")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |