Skip to content

hashgraph-online/awesome-codex-plugins / plugins/beastlabai/multi-llm-plugin/skills/multi-llm

multi-llm

Flagged

Orchestrate plan reviews, task generation, implementation and code reviews across several LLM CLIs in parallel, or ask every model one free-text question about a plan. Use when the user says: review my plan, multi-llm, run a code review, generate tasks from a plan, implement this plan, ask the models about my plan.

FileKindSize
SKILL.mdskill md4.0 KB
apply_code_fixes_orchestrator.pyscript57.0 KB
apply_commit_fixes_orchestrator.pyscript2.3 KB
apply_suggestions_orchestrator.pyscript4.6 KB
apply_task_suggestions_orchestrator.pyscript19.2 KB
ask_orchestrator.pyscript162.5 KB
assets/icons/Anthropic.svgother584 B
assets/icons/Codex.svgother1.3 KB
assets/icons/Copilot.svgother1.5 KB
assets/icons/Cursor.svgother793 B
assets/icons/GoogleGemini.svgother623 B
assets/icons/Goose.svgother3.2 KB
assets/icons/Kilocode.svgother531 B
assets/icons/MiniMax.pngbinary1.5 KB
assets/icons/Mistral.svgother717 B
assets/icons/MoonshotAI.svgother957 B
assets/icons/Nvidia.svgother2.4 KB
assets/icons/OhMyPi.svgother434 B
assets/icons/OpenAI.svgother1.7 KB
assets/icons/Opencode.svgother325 B
assets/icons/Pi.svgother440 B
assets/icons/Qwen.svgother1.7 KB
assets/icons/Zhipu.pngbinary442 B
assets/icons/xAI.svgother343 B
assets/omp/overlay.ymlconfig2.4 KB
check_workflow_prerequisites.pyscript77.8 KB
code_review_orchestrator.pyscript326.3 KB
display_decisions.pyscript9.8 KB
finalize_tracking.pyscript1.3 KB
full_workflow_orchestrator.pyscript69.7 KB
generate_tasks_orchestrator.pyscript44.2 KB
hosts/capabilities.yamlconfig50.2 KB
hosts/codex/plugin-skill.md.tmplother2.8 KB
hosts/codex/shim.md.tmplother7.3 KB
hosts/opencode/plugin-command.md.tmplother2.2 KB
hosts/opencode/shim.md.tmplother6.8 KB
hosts/render_capabilities.pyscript48.4 KB
hosts/snippets/poll-loop.mdmarkdown4.4 KB
hosts/snippets/uv-preflight.mdmarkdown2.2 KB
hosts/uv-missing.txtmarkdown1.5 KB
implement_orchestrator.pyscript156.4 KB
improve_code_orchestrator.pyscript1.6 KB
improve_commits_orchestrator.pyscript2.5 KB
improve_plan_orchestrator.pyscript1.6 KB
init_config.pyscript58.0 KB
instructions/apply-code-fixes.mdmarkdown56.2 KB
instructions/apply-commit-fixes.mdmarkdown10.3 KB
instructions/apply-suggestions.mdmarkdown48.9 KB
instructions/apply-task-suggestions.mdmarkdown54.5 KB
instructions/ask.mdmarkdown27.6 KB
instructions/full-workflow.mdmarkdown25.3 KB
instructions/generate-tasks.mdmarkdown18.1 KB
instructions/implement.mdmarkdown55.8 KB
instructions/improve-code.mdmarkdown25.7 KB
instructions/improve-commits.mdmarkdown25.5 KB
instructions/improve-plan.mdmarkdown26.7 KB
instructions/init-config.mdmarkdown6.6 KB
instructions/review-code.mdmarkdown44.0 KB
instructions/review-commits.mdmarkdown11.4 KB
instructions/review-plan.mdmarkdown38.8 KB
instructions/review-tasks.mdmarkdown38.2 KB
modes.pyscript58.8 KB
multi_llm.pyscript101.1 KB
prompts/ask.txtmarkdown948 B
prompts/code_review.txtmarkdown4.5 KB
prompts/commit_review.txtmarkdown6.2 KB
prompts/consolidate_suggestions.txtmarkdown3.1 KB
prompts/implementation_task.txtmarkdown1.4 KB
prompts/plan_review.txtmarkdown8.0 KB
prompts/task_review.txtmarkdown8.3 KB
providers.yamlconfig20.7 KB
pyproject.tomlmanifest7.0 KB
references/apply-write-executor.mdmarkdown3.5 KB
references/consolidation.mdmarkdown5.6 KB
references/host-capabilities.mdmarkdown13.5 KB
references/human-decision-batch.mdmarkdown17.2 KB
references/implement-prerequisites.mdmarkdown6.0 KB
references/implement-write-executor.mdmarkdown5.4 KB
references/interrupted-run-recovery.mdmarkdown3.1 KB
references/own-check.mdmarkdown4.3 KB
references/resume-detection.mdmarkdown8.5 KB
references/revalidation.mdmarkdown6.9 KB
references/review-code-apply-fixes.mdmarkdown4.3 KB
references/salvage-handling.mdmarkdown12.5 KB
references/summary-format.mdmarkdown10.0 KB
references/uv-check.mdmarkdown5.7 KB
references/wave-batching.mdmarkdown5.2 KB
review_commits_orchestrator.pyscript2.3 KB
review_plan_orchestrator.pyexecutablescript24.6 KB
review_tasks_orchestrator.pyscript10.7 KB
schemas/cascade_record.schema.jsonconfig6.6 KB
schemas/code_review_issues.schema.jsonconfig2.0 KB
schemas/consolidated.schema.jsonconfig6.8 KB
schemas/consolidation_batch.schema.jsonconfig3.7 KB
schemas/generated_tasks.schema.jsonconfig3.1 KB
schemas/host_capabilities.schema.jsonconfig38.3 KB
schemas/human_decision_verdicts.schema.jsonconfig2.6 KB
schemas/result_line.schema.jsonconfig10.1 KB
schemas/review_suggestions.schema.jsonconfig2.4 KB
schemas/state_file.schema.jsonconfig23.3 KB
schemas/status_file.schema.jsonconfig24.9 KB
schemas/status_json.schema.jsonconfig10.8 KB
schemas/subagent_tasks.schema.jsonconfig20.5 KB
schemas/task_decomposition.schema.jsonconfig1.2 KB
schemas/write_batch_result.schema.jsonconfig8.8 KB
templates/config/providers.override.yamlconfig17.8 KB
templates/consolidated_report_template.htmlother44.1 KB
templates/pr_report_template.htmlother181.2 KB
templates/report_template.htmlother67.5 KB
templates/shared/report_core.cssother4.1 KB
templates/shared/report_core.jsscript25.5 KB
update_plan_tasks.pyscript20.9 KB
utils/__init__.pyscript284 B
utils/advisory.pyscript4.8 KB
utils/apply_orchestrator_base.pyscript229.0 KB
utils/apply_output_helpers.pyscript10.9 KB
utils/apply_path_helpers.pyscript683 B
utils/apply_report.pyscript51.6 KB
utils/apply_selection_helpers.pyscript6.8 KB
utils/apply_suggestion_phase.pyscript11.7 KB
utils/atomic_io.pyscript12.9 KB
utils/attempt_classifier.pyscript33.2 KB
utils/backup.pyscript2.0 KB
utils/batch_done.pyscript81.3 KB
utils/batch_ledger.pyscript27.7 KB
utils/batch_prompt.pyscript30.1 KB
utils/batch_prompt_builder.pyscript13.2 KB
utils/batch_reply.pyscript32.9 KB
utils/batch_suppressions.pyscript15.5 KB
utils/cascade_archive.pyscript50.6 KB
utils/cascade_ledger.pyscript39.3 KB
utils/cascade_provenance.pyscript22.0 KB
utils/check_records.pyscript59.7 KB
utils/check_reply.pyscript9.5 KB
utils/check_tree.pyscript1.2 KB
utils/cli_common.pyscript94.3 KB
utils/cli_version_cache.pyscript14.0 KB
utils/clock.pyscript3.5 KB
utils/code_fix_batcher.pyscript31.4 KB
utils/codex_trust.pyscript12.9 KB
utils/commit_apply.pyscript73.3 KB
utils/commit_attribution.pyscript21.2 KB
utils/commit_brief.pyscript32.2 KB
utils/commit_front_door.pyscript81.9 KB
utils/commit_paths.pyscript18.0 KB
utils/commit_range.pyscript61.2 KB
utils/commit_report.pyscript19.9 KB
utils/commit_review.pyscript413 B
utils/commit_spelling.pyscript7.9 KB
utils/commit_write_guard.pyscript9.9 KB
utils/concurrency.pyscript11.6 KB
utils/consolidation.pyscript89.0 KB
utils/detach.pyscript187.9 KB
utils/dispatch_spawn.pyscript62.4 KB
utils/doctor.pyscript76.7 KB
utils/env_vars.pyscript20.5 KB
utils/failure_capture.pyscript10.1 KB
utils/file_discovery.pyscript11.0 KB
utils/filtering.pyscript19.9 KB
utils/fresh_reset.pyscript3.9 KB
utils/git_diff.pyscript23.6 KB
utils/git_utils.pyscript86.6 KB
utils/held_folder.pyscript21.3 KB
utils/host_capabilities.pyscript48.0 KB
utils/host_install.pyscript77.6 KB
utils/host_paths.pyscript12.0 KB
utils/host_routes/__init__.pyscript5.7 KB
utils/host_routes/codex.pyscript18.3 KB
utils/host_routes/npm.pyscript12.2 KB
utils/host_shim.pyscript14.7 KB
utils/html_report_generator.pyscript118.9 KB
utils/human_decisions.pyscript38.3 KB
utils/human_task_plan.pyscript30.2 KB
utils/importance.pyscript6.2 KB
utils/improve_cascade.pyscript63.7 KB
utils/improve_cascade_next.pyscript22.2 KB
utils/improve_steps.pyscript33.1 KB
utils/improve_workflow.pyscript120.9 KB
utils/interactive.pyscript31.6 KB
utils/json_extractor.pyscript83.1 KB
utils/judge_vocab.pyscript5.3 KB
utils/known_checks.pyscript41.6 KB
utils/landlock_dispatch.pyscript103.8 KB
utils/landlock_exec.pyscript21.4 KB
utils/launch_limiter.pyscript13.5 KB
utils/llm_client.pyscript142.7 KB
utils/manual_tasks.pyscript14.9 KB
utils/metrics.pyscript68.1 KB
utils/next_phase.pyscript96.7 KB
utils/next_phase_words.pyscript6.4 KB
utils/output_handler.pyscript23.2 KB
utils/phase_walk.pyscript6.4 KB
utils/plan_updater.pyscript17.9 KB
utils/positionals.pyscript5.8 KB
utils/prior_changes.pyscript28.7 KB
utils/probe_state.pyscript17.9 KB
utils/process_capture.pyscript44.1 KB
utils/process_control.pyscript228.7 KB
utils/process_state.pyscript6.6 KB
utils/progress_tracker.pyscript92.2 KB

references/consolidation.md

View on GitHub
# Suggestion Consolidation (review-plan, optional)

Read this only when the review-plan core's step 9 sends you here, in one of two
cases:
- **Starting a recommended pass:** the log `{plan}/review-plan/orchestrator-run.log`
  carries `[CONSOLIDATION_RECOMMENDED]`, or `MULTI_LLM_RESULT.consolidation_recommended`
  is set. Start at *Decide whether it runs*.
- **Continuing a requested or pending pass:** the user typed `--consolidate`, or
  a review-plan run printed `[CONSOLIDATION_PENDING]` or
  `[CONSOLIDATION_BATCHES_PENDING]`. That pass is already decided, below the
  recommendation threshold too: it passes no recommendation check and asks no
  question. Start at section a) when that `--consolidate` has not run yet, or at
  section b) for the batches a pending marker names.

A run with no `[CONSOLIDATION_RECOMMENDED]` line, no `consolidation_recommended`
field, no `--consolidate` from the user and no `[CONSOLIDATION_PENDING]` or
`[CONSOLIDATION_BATCHES_PENDING]` line skips consolidation entirely.

**Commands.** This file is read as it is, so its commands are spelled from the
mode word on: `<prefix>` stands for the dispatcher prefix every command in the
core starts with (`uv run … --project "<skill dir>" -- python "<skill dir>/multi_llm.py"`,
flags and paths exactly as the core prints them). Copy it from any command in
the core, unchanged, and run each command bare, as the core runs its own.

## Decide whether it runs

The orchestrator emits the marker only at `CONSOLIDATION_RECOMMENDED_THRESHOLD`
(15 valid groups, tunable with `--consolidation-threshold`), so the marker's
presence *is* the decision; there is no second threshold to apply. Report the
recommendation from `MULTI_LLM_RESULT.consolidation_recommended`
(`{valid_groups, threshold, command}`, carried by `status --view run` too) rather
than inferring it from the marker's absence.

- With `--yes` (or `--non-interactive`): **auto-run consolidation once, no
  prompt** (D6 row 4). If consolidation itself fails, keep the unconsolidated
  results, warn, and continue: a failed optional pass is not a failed phase.
- Without `--yes`: tell the user the group count and **ask the user** whether to
  run consolidation. If the user declines, stop here.

## a) Run the consolidation orchestrator

```bash
<prefix> review-plan --plan-file "$PLAN_PATH" --consolidate
```

## b) Run the consolidation batches as subagents

The orchestrator output carries `[CONSOLIDATION_PENDING]` (one batch) or
`[CONSOLIDATION_BATCHES_PENDING]` (several). Then:

1. Load `consolidation_tasks.index.json` from the phase directory (`{plan}/review-plan/`),
   or `consolidation_tasks.json` when there is no index: the same `batches` without
   their prompts. Every task in the full file carries a complete `prompt`, and
   `prompt_path` (`consolidation_batch_N.prompt.txt` in the same directory, exactly
   the bytes of `prompt`) with its line count `prompt_lines`.
2. Count the batches as `{total_batches}` and mark the consolidation phase start:
   ```bash
   <prefix> util metrics start --state-file "$STATE_FILE" --phase "review-plan-consolidation" --total-batches {total_batches}
   ```
3. For each task, run a subagent the way the core's validation contract says: when
   the task carries `prompt_path`, the subagent reads that file in full (through
   line `prompt_lines`) and follows it; otherwise, or if the file is absent, pass
   that task's `prompt` from `consolidation_tasks.json`. The filled
   `consolidate_suggestions.txt` template and that batch's group records are
   already inside it, so write no prompt file of your own.
4. Write the JSON each subagent returns to that task's `output_path`
   (`consolidation_batch_N.json` in the phase directory).
5. After each batch completes, record metrics with a cumulative `--batch-index`:
   ```bash
   <prefix> util metrics record --state-file "$STATE_FILE" --phase "review-plan-consolidation" --label "Consolidation batch {N}" --subagent-type "general-purpose" --tokens {token_count} --tool-uses {tool_uses} --duration-ms {duration_ms} --total-batches {total_batches} --batch-index {N}
   ```
   `token_count`, `tool_uses` and `duration_ms` come from whatever your subagent
   mechanism reports. Omit any flag it does not report. Never pass `0` for an
   unknown value; omit the flag: a made-up zero is averaged into the phase totals
   as a real measurement.

**Strategy A (≤ 4 batches):** run all batch subagents from the main agent in
parallel (one message with several subagent calls).

**Strategy B (> 4 batches):** split the batches into waves of up to 4 (the last
wave may absorb the remainder, up to 6). For each wave, run all its subagents in
parallel, wait for them, then start the next wave.

**Before you continue: verify every batch wrote its file, and do not end your
turn before it has.** Confirm that each task's `output_path`
(`consolidation_batch_N.json`) exists and is not empty, and re-dispatch any task
whose file is missing before reaggregating. **You** write these files, and a host
that runs one headless turn has no second turn in which to write them.

## c) Reaggregate after all batches complete

```bash
<prefix> review-plan --plan-file "$PLAN_PATH" --reaggregate-consolidation
```

Then mark the consolidation phase finish:

```bash
<prefix> util metrics finish --state-file "$STATE_FILE" --phase "review-plan-consolidation"
```

**CRITICAL**: do NOT report consolidation results until reaggregation is complete.

## d) Report every consolidation output path

- `consolidated.json`: consolidated suggestion groups
- `consolidated-report.md`: human-readable consolidation report
- `consolidated-report.html`: HTML version of the consolidation report

That completes the core's step 9.
receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementationStars: 322kUpdated Claude CodeCodexVerifiedInstall
requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirementsStars: 322kUpdated Claude CodeCodexVerifiedInstall
using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallbackStars: 322kUpdated Claude CodeCodexVerifiedInstall
verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions alwaysStars: 322kUpdated Claude CodeCodexVerifiedInstall