Skip to content

hashgraph-online/awesome-codex-plugins / plugins/beastlabai/multi-llm-plugin/skills/multi-llm

multi-llm

Flagged

Orchestrate plan reviews, task generation, implementation and code reviews across several LLM CLIs in parallel, or ask every model one free-text question about a plan. Use when the user says: review my plan, multi-llm, run a code review, generate tasks from a plan, implement this plan, ask the models about my plan.

FileKindSize
SKILL.mdskill md4.0 KB
apply_code_fixes_orchestrator.pyscript56.6 KB
apply_commit_fixes_orchestrator.pyscript2.3 KB
apply_suggestions_orchestrator.pyscript4.5 KB
apply_task_suggestions_orchestrator.pyscript19.1 KB
ask_orchestrator.pyscript162.5 KB
assets/icons/Anthropic.svgother584 B
assets/icons/Codex.svgother1.3 KB
assets/icons/Copilot.svgother1.5 KB
assets/icons/Cursor.svgother793 B
assets/icons/GoogleGemini.svgother623 B
assets/icons/Goose.svgother3.2 KB
assets/icons/Kilocode.svgother531 B
assets/icons/MiniMax.pngbinary1.5 KB
assets/icons/Mistral.svgother717 B
assets/icons/MoonshotAI.svgother957 B
assets/icons/Nvidia.svgother2.4 KB
assets/icons/OhMyPi.svgother434 B
assets/icons/OpenAI.svgother1.7 KB
assets/icons/Opencode.svgother325 B
assets/icons/Pi.svgother440 B
assets/icons/Qwen.svgother1.7 KB
assets/icons/Zhipu.pngbinary442 B
assets/icons/xAI.svgother343 B
assets/omp/overlay.ymlconfig2.4 KB
check_workflow_prerequisites.pyscript74.5 KB
code_review_orchestrator.pyscript324.0 KB
display_decisions.pyscript9.8 KB
finalize_tracking.pyscript1.3 KB
full_workflow_orchestrator.pyscript58.8 KB
generate_tasks_orchestrator.pyscript44.2 KB
hosts/capabilities.yamlconfig49.9 KB
hosts/codex/plugin-skill.md.tmplother2.8 KB
hosts/codex/shim.md.tmplother7.3 KB
hosts/opencode/plugin-command.md.tmplother2.2 KB
hosts/opencode/shim.md.tmplother6.8 KB
hosts/render_capabilities.pyscript48.4 KB
hosts/snippets/poll-loop.mdmarkdown4.4 KB
hosts/snippets/uv-preflight.mdmarkdown2.2 KB
hosts/uv-missing.txtmarkdown1.5 KB
implement_orchestrator.pyscript156.0 KB
improve_code_orchestrator.pyscript1.6 KB
improve_commits_orchestrator.pyscript2.5 KB
improve_plan_orchestrator.pyscript1.6 KB
init_config.pyscript57.6 KB
instructions/apply-code-fixes.mdmarkdown52.0 KB
instructions/apply-commit-fixes.mdmarkdown10.3 KB
instructions/apply-suggestions.mdmarkdown48.2 KB
instructions/apply-task-suggestions.mdmarkdown53.9 KB
instructions/ask.mdmarkdown27.6 KB
instructions/full-workflow.mdmarkdown24.6 KB
instructions/generate-tasks.mdmarkdown18.1 KB
instructions/implement.mdmarkdown54.0 KB
instructions/improve-code.mdmarkdown25.0 KB
instructions/improve-commits.mdmarkdown24.8 KB
instructions/improve-plan.mdmarkdown26.2 KB
instructions/init-config.mdmarkdown6.6 KB
instructions/review-code.mdmarkdown43.5 KB
instructions/review-commits.mdmarkdown11.4 KB
instructions/review-plan.mdmarkdown38.4 KB
instructions/review-tasks.mdmarkdown37.8 KB
modes.pyscript58.8 KB
multi_llm.pyscript101.1 KB
prompts/ask.txtmarkdown948 B
prompts/code_review.txtmarkdown4.5 KB
prompts/commit_review.txtmarkdown6.2 KB
prompts/consolidate_suggestions.txtmarkdown3.1 KB
prompts/implementation_task.txtmarkdown1.4 KB
prompts/plan_review.txtmarkdown8.0 KB
prompts/task_review.txtmarkdown8.3 KB
providers.yamlconfig19.6 KB
pyproject.tomlmanifest6.8 KB
references/apply-write-executor.mdmarkdown3.0 KB
references/consolidation.mdmarkdown5.6 KB
references/host-capabilities.mdmarkdown13.5 KB
references/human-decision-batch.mdmarkdown17.2 KB
references/implement-prerequisites.mdmarkdown6.0 KB
references/implement-write-executor.mdmarkdown4.9 KB
references/interrupted-run-recovery.mdmarkdown3.1 KB
references/own-check.mdmarkdown4.3 KB
references/resume-detection.mdmarkdown8.5 KB
references/revalidation.mdmarkdown6.9 KB
references/review-code-apply-fixes.mdmarkdown4.3 KB
references/salvage-handling.mdmarkdown12.5 KB
references/summary-format.mdmarkdown10.0 KB
references/uv-check.mdmarkdown5.7 KB
references/wave-batching.mdmarkdown5.2 KB
review_commits_orchestrator.pyscript2.3 KB
review_plan_orchestrator.pyexecutablescript22.3 KB
review_tasks_orchestrator.pyscript10.7 KB
schemas/cascade_record.schema.jsonconfig6.6 KB
schemas/code_review_issues.schema.jsonconfig2.0 KB
schemas/consolidated.schema.jsonconfig6.8 KB
schemas/consolidation_batch.schema.jsonconfig3.7 KB
schemas/generated_tasks.schema.jsonconfig3.1 KB
schemas/host_capabilities.schema.jsonconfig38.3 KB
schemas/human_decision_verdicts.schema.jsonconfig2.6 KB
schemas/result_line.schema.jsonconfig10.1 KB
schemas/review_suggestions.schema.jsonconfig2.4 KB
schemas/state_file.schema.jsonconfig21.5 KB
schemas/status_file.schema.jsonconfig24.9 KB
schemas/status_json.schema.jsonconfig10.8 KB
schemas/subagent_tasks.schema.jsonconfig20.5 KB
schemas/task_decomposition.schema.jsonconfig1.2 KB
schemas/write_batch_result.schema.jsonconfig8.8 KB
templates/config/providers.override.yamlconfig17.2 KB
templates/consolidated_report_template.htmlother44.1 KB
templates/pr_report_template.htmlother181.2 KB
templates/report_template.htmlother67.5 KB
templates/shared/report_core.cssother4.1 KB
templates/shared/report_core.jsscript25.5 KB
update_plan_tasks.pyscript20.9 KB
utils/__init__.pyscript284 B
utils/advisory.pyscript4.8 KB
utils/apply_orchestrator_base.pyscript220.6 KB
utils/apply_output_helpers.pyscript10.9 KB
utils/apply_path_helpers.pyscript683 B
utils/apply_report.pyscript51.2 KB
utils/apply_selection_helpers.pyscript6.8 KB
utils/apply_suggestion_phase.pyscript11.7 KB
utils/atomic_io.pyscript12.9 KB
utils/attempt_classifier.pyscript31.4 KB
utils/backup.pyscript2.0 KB
utils/batch_done.pyscript79.2 KB
utils/batch_ledger.pyscript27.4 KB
utils/batch_prompt.pyscript29.3 KB
utils/batch_prompt_builder.pyscript12.2 KB
utils/batch_reply.pyscript32.9 KB
utils/cascade_archive.pyscript50.6 KB
utils/cascade_ledger.pyscript39.3 KB
utils/cascade_provenance.pyscript22.0 KB
utils/check_records.pyscript59.3 KB
utils/check_reply.pyscript9.5 KB
utils/check_tree.pyscript1.2 KB
utils/cli_common.pyscript91.2 KB
utils/cli_version_cache.pyscript14.0 KB
utils/clock.pyscript3.5 KB
utils/code_fix_batcher.pyscript30.6 KB
utils/codex_trust.pyscript12.9 KB
utils/commit_apply.pyscript73.3 KB
utils/commit_attribution.pyscript21.2 KB
utils/commit_brief.pyscript32.2 KB
utils/commit_front_door.pyscript81.6 KB
utils/commit_paths.pyscript18.0 KB
utils/commit_range.pyscript61.2 KB
utils/commit_report.pyscript19.9 KB
utils/commit_review.pyscript413 B
utils/commit_spelling.pyscript7.9 KB
utils/commit_write_guard.pyscript9.9 KB
utils/concurrency.pyscript11.6 KB
utils/consolidation.pyscript87.3 KB
utils/detach.pyscript186.8 KB
utils/dispatch_spawn.pyscript56.3 KB
utils/doctor.pyscript76.7 KB
utils/env_vars.pyscript20.5 KB
utils/failure_capture.pyscript10.1 KB
utils/file_discovery.pyscript11.0 KB
utils/filtering.pyscript19.9 KB
utils/git_diff.pyscript23.6 KB
utils/git_utils.pyscript86.6 KB
utils/held_folder.pyscript21.3 KB
utils/host_capabilities.pyscript48.0 KB
utils/host_install.pyscript77.6 KB
utils/host_paths.pyscript12.0 KB
utils/host_routes/__init__.pyscript5.7 KB
utils/host_routes/codex.pyscript18.3 KB
utils/host_routes/npm.pyscript12.2 KB
utils/host_shim.pyscript14.7 KB
utils/html_report_generator.pyscript118.9 KB
utils/human_decisions.pyscript38.3 KB
utils/human_task_plan.pyscript30.2 KB
utils/importance.pyscript6.2 KB
utils/improve_cascade.pyscript63.7 KB
utils/improve_cascade_next.pyscript22.2 KB
utils/improve_steps.pyscript32.4 KB
utils/improve_workflow.pyscript114.4 KB
utils/interactive.pyscript31.6 KB
utils/json_extractor.pyscript61.5 KB
utils/judge_vocab.pyscript5.3 KB
utils/known_checks.pyscript37.5 KB
utils/landlock_dispatch.pyscript103.8 KB
utils/landlock_exec.pyscript21.4 KB
utils/launch_limiter.pyscript13.5 KB
utils/llm_client.pyscript126.9 KB
utils/manual_tasks.pyscript14.9 KB
utils/metrics.pyscript68.1 KB
utils/next_phase.pyscript96.1 KB
utils/next_phase_words.pyscript6.4 KB
utils/output_handler.pyscript23.2 KB
utils/phase_walk.pyscript6.4 KB
utils/plan_updater.pyscript17.9 KB
utils/positionals.pyscript5.8 KB
utils/prior_changes.pyscript28.7 KB
utils/probe_state.pyscript17.9 KB
utils/process_capture.pyscript37.6 KB
utils/process_control.pyscript228.7 KB
utils/process_state.pyscript6.6 KB
utils/progress_tracker.pyscript92.2 KB
utils/prompt_channels.pyscript34.1 KB
utils/prompt_files.pyscript24.5 KB

pyproject.toml

View on GitHub
[project]
name = "multi-llm"
version = "0.1.9"
description = "Multi-LLM automation for plan review, task generation, implementation, and code review"
requires-python = ">=3.10"
dependencies = ["pyyaml>=6.0"]

[dependency-groups]
dev = ["pytest>=7.0", "pytest-cov>=4.0", "pytest-xdist>=3.0", "ruff>=0.1.0", "jsonschema>=4.0"]

[tool.pytest.ini_options]
markers = [
    "robustness: marks tests for LLM error handling robustness scenarios",
    "live: marks tests that require Claude Code CLI and API (skipped in CI)",
    "timeout: marks tests with a timeout limit (requires pytest-timeout)",
    "config_override: opts out of the autouse per-project config isolation fixture (tests that exercise override discovery/env themselves)",
    "model_catalog: opts out of the autouse stub that pins OpenCodeProvider.describe_unlisted_model to known=None (tests that drive the real `opencode models` read, or patch it themselves)",
    "debug_config: opts out of the autouse stub that answers `opencode debug config` with a failed read (tests that drive the reader themselves, with a fake binary or a patched spawn)",
    "sandbox_probe: opts out of the autouse guard that answers a real provider CLI's sandbox probe with `unknown` (tests that deliberately drive the developer's own `codex sandbox`)",
    "shell_probe: lets a test's own sandbox probes be shell text or a prefixed command line (`sh -c …`, `python -c`, `env …`, `timeout …`), which the autouse probe guard otherwise refuses because it cannot see which files that text runs (the gate's tests whose fake probes are `sh -c` scripts)",
    "slow: marks tests that drive several real orchestrator invocations end to end (the instruction-refactor golden guard); runs in CI, not deselected by default",
    "container: marks tests that build and drive the E2E container image (needs a Docker daemon; opt in with -m container). Deselected by default via addopts below",
    "package: marks rows that need a packaging artifact plain CI does not build (an `npm pack` tarball, a staged tree). Named for what it *needs*, not for npm specifically, so a second host's packaging plan can reuse it. Deselected by default via addopts below and run by the Linux-only PR package job with `-m \"package and not container\"`; note `pytest -m package` with zero matches exits 5, so a per-host leg must not run it for a host that declares no such rows",
    "live_provider: marks the credentialed provider lane (harness-expansion.md §7.4) — a real CLI against https://openrouter.ai on the user's own key, so it SPENDS CREDITS. Carried in addition to `container`, so the addopts default already deselects it and CI's `not container` does too; `make e2e-live-provider` is the only thing that sources the key",
    "live_native_phase: marks the host lane's claude-code per-phase rows (chain-lane-write-runner-matrix.md §3.4) — each one a native Task dispatch the host performs itself, anchored on `claude-code-apply-native`'s measured $0.586302 pass rather than on a read row. Carried IN ADDITION TO `container` and `live`, so the addopts default and CI's `not container` keep excluding it with no further edit, and `make e2e-container-host` excludes it explicitly (`-m \"container and live and not live_native_phase\"`) so the cheap tier's cost does not move. `make e2e-container-host-phases` is the only target that selects it, with its own MLLM_E2E_BUDGET_USD: the $2.00 default is a BETWEEN-ROWS pytest.exit, and a ~$3-5 tier crosses it around the fourth row, stranding the rest",
    "live_next_command: marks the host lane's T10 next-command rows (next-phase-command.md T10) — one `generate-tasks` row per host and route plus a short typed follow-up session, graded by `TestTheNextCommandLiveCheck`. Carried IN ADDITION TO `container` and `live`, so the addopts default and CI's `not container` keep excluding it, and `make e2e-container-host` excludes it explicitly so the ordinary tier's cost does not move. `make e2e-container-host-next-command` is the only target that selects it",
    "landlock: marks tier 2/3 rows of the plugin-landlock sandbox rung (codex-sandbox-unavailable-findings-and-fixes §8.1): the real `utils/landlock_exec.py` against the real kernel. They need a kernel whose Landlock ABI query returns >= 3 and SKIP otherwise, naming the errno or ABI found; `MULTI_LLM_REQUIRE_LANDLOCK=1` turns that skip into a failure, so a host that must run them cannot pass by skipping. Not deselected by default: zero-spend and fast where the kernel supports it",
    "codex_jail: marks tier 3 of the plugin-landlock rung — the installed `codex` inside the real jail, against the localhost stub (throwaway CODEX_HOME, zero spend). Opt in with -m codex_jail; deselected by default via addopts below, like `container`, because it drives the developer's own codex binary. Carried IN ADDITION TO `landlock`, so it also skips without a usable kernel",
]
# A marker deselects nothing on its own — it only gives a -m expression something
# to exclude. This default is what keeps a bare `pytest` green on a machine with no
# container runtime. It is a *default*: an explicit -m on the command line replaces
# it, which is why .github/workflows/test.yml has to say `not container` (and
# `not codex_jail`) itself.
addopts = "-m 'not container and not package and not codex_jail'"
# Filter out warnings about dataclass fixtures being collected by pytest
filterwarnings = [
    "ignore:cannot collect test class.*FixturePlan.*:pytest.PytestCollectionWarning",
]

# Lint gate (stage 1). The command is `make lint`: it passes this file with
# --config because packaging/ sits outside the project directory, so ruff would
# not find it there. With --config the globs below resolve from the repo root.
# See docs/development/ for the stages that follow.
[tool.ruff]
extend-exclude = [
    "packaging/claude/multi-llm",
    "packaging/codex/multi-llm",
    "packaging/opencode/dist",
    "packaging/opencode/skills",
]

[tool.ruff.lint]
select = ["E9", "F", "B006", "B008", "B023", "B904", "PLE"]
ignore = ["PLE2502"]

[tool.ruff.lint.per-file-ignores]
# The tests keep only the rules that find real defects: E9, F82x (undefined
# names), F63x and F70x. Everything else the selection brings in is dropped for
# them: unused imports and variables (F40x, F84x), format-string checks (F5),
# dict and assert checks (F60x, F62x), annotations (F72x), `__all__` (F90x),
# fixture-name shadowing (F811), the B and PLE families, late imports (E402),
# one-line statements (E702) and short names (E741) are how the suite is written.
"skills/multi-llm/tests/**" = [
    "B", "PLE", "E402", "E702", "E741",
    "F40", "F5", "F60", "F62", "F72", "F811", "F84", "F90",
]
# These two modules export their moved names lazily through a module-level
# `__getattr__`, so `__all__` lists names the module does not bind.
"skills/multi-llm/utils/subagent_executor.py" = ["F822"]
"skills/multi-llm/utils/write_batch_scheduler.py" = ["F822"]
receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementationStars: 321kUpdated Claude CodeCodexVerifiedInstall
requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirementsStars: 321kUpdated Claude CodeCodexVerifiedInstall
using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallbackStars: 321kUpdated Claude CodeCodexVerifiedInstall
verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions alwaysStars: 321kUpdated Claude CodeCodexVerifiedInstall