Skip to content

hashgraph-online/awesome-codex-plugins / plugins/beastlabai/multi-llm-plugin/skills/multi-llm

multi-llm

Flagged

Orchestrate plan reviews, task generation, implementation and code reviews across several LLM CLIs in parallel, or ask every model one free-text question about a plan. Use when the user says: review my plan, multi-llm, run a code review, generate tasks from a plan, implement this plan, ask the models about my plan.

FileKindSize
SKILL.mdskill md4.0 KB
apply_code_fixes_orchestrator.pyscript56.6 KB
apply_commit_fixes_orchestrator.pyscript2.3 KB
apply_suggestions_orchestrator.pyscript4.5 KB
apply_task_suggestions_orchestrator.pyscript19.1 KB
ask_orchestrator.pyscript162.5 KB
assets/icons/Anthropic.svgother584 B
assets/icons/Codex.svgother1.3 KB
assets/icons/Copilot.svgother1.5 KB
assets/icons/Cursor.svgother793 B
assets/icons/GoogleGemini.svgother623 B
assets/icons/Goose.svgother3.2 KB
assets/icons/Kilocode.svgother531 B
assets/icons/MiniMax.pngbinary1.5 KB
assets/icons/Mistral.svgother717 B
assets/icons/MoonshotAI.svgother957 B
assets/icons/Nvidia.svgother2.4 KB
assets/icons/OhMyPi.svgother434 B
assets/icons/OpenAI.svgother1.7 KB
assets/icons/Opencode.svgother325 B
assets/icons/Pi.svgother440 B
assets/icons/Qwen.svgother1.7 KB
assets/icons/Zhipu.pngbinary442 B
assets/icons/xAI.svgother343 B
assets/omp/overlay.ymlconfig2.4 KB
check_workflow_prerequisites.pyscript74.5 KB
code_review_orchestrator.pyscript324.0 KB
display_decisions.pyscript9.8 KB
finalize_tracking.pyscript1.3 KB
full_workflow_orchestrator.pyscript58.8 KB
generate_tasks_orchestrator.pyscript44.2 KB
hosts/capabilities.yamlconfig49.9 KB
hosts/codex/plugin-skill.md.tmplother2.8 KB
hosts/codex/shim.md.tmplother7.3 KB
hosts/opencode/plugin-command.md.tmplother2.2 KB
hosts/opencode/shim.md.tmplother6.8 KB
hosts/render_capabilities.pyscript48.4 KB
hosts/snippets/poll-loop.mdmarkdown4.4 KB
hosts/snippets/uv-preflight.mdmarkdown2.2 KB
hosts/uv-missing.txtmarkdown1.5 KB
implement_orchestrator.pyscript156.0 KB
improve_code_orchestrator.pyscript1.6 KB
improve_commits_orchestrator.pyscript2.5 KB
improve_plan_orchestrator.pyscript1.6 KB
init_config.pyscript57.6 KB
instructions/apply-code-fixes.mdmarkdown52.0 KB
instructions/apply-commit-fixes.mdmarkdown10.3 KB
instructions/apply-suggestions.mdmarkdown48.2 KB
instructions/apply-task-suggestions.mdmarkdown53.9 KB
instructions/ask.mdmarkdown27.6 KB
instructions/full-workflow.mdmarkdown24.6 KB
instructions/generate-tasks.mdmarkdown18.1 KB
instructions/implement.mdmarkdown54.0 KB
instructions/improve-code.mdmarkdown25.0 KB
instructions/improve-commits.mdmarkdown24.8 KB
instructions/improve-plan.mdmarkdown26.2 KB
instructions/init-config.mdmarkdown6.6 KB
instructions/review-code.mdmarkdown43.5 KB
instructions/review-commits.mdmarkdown11.4 KB
instructions/review-plan.mdmarkdown38.4 KB
instructions/review-tasks.mdmarkdown37.8 KB
modes.pyscript58.8 KB
multi_llm.pyscript101.1 KB
prompts/ask.txtmarkdown948 B
prompts/code_review.txtmarkdown4.5 KB
prompts/commit_review.txtmarkdown6.2 KB
prompts/consolidate_suggestions.txtmarkdown3.1 KB
prompts/implementation_task.txtmarkdown1.4 KB
prompts/plan_review.txtmarkdown8.0 KB
prompts/task_review.txtmarkdown8.3 KB
providers.yamlconfig19.6 KB
pyproject.tomlmanifest6.8 KB
references/apply-write-executor.mdmarkdown3.0 KB
references/consolidation.mdmarkdown5.6 KB
references/host-capabilities.mdmarkdown13.5 KB
references/human-decision-batch.mdmarkdown17.2 KB
references/implement-prerequisites.mdmarkdown6.0 KB
references/implement-write-executor.mdmarkdown4.9 KB
references/interrupted-run-recovery.mdmarkdown3.1 KB
references/own-check.mdmarkdown4.3 KB
references/resume-detection.mdmarkdown8.5 KB
references/revalidation.mdmarkdown6.9 KB
references/review-code-apply-fixes.mdmarkdown4.3 KB
references/salvage-handling.mdmarkdown12.5 KB
references/summary-format.mdmarkdown10.0 KB
references/uv-check.mdmarkdown5.7 KB
references/wave-batching.mdmarkdown5.2 KB
review_commits_orchestrator.pyscript2.3 KB
review_plan_orchestrator.pyexecutablescript22.3 KB
review_tasks_orchestrator.pyscript10.7 KB
schemas/cascade_record.schema.jsonconfig6.6 KB
schemas/code_review_issues.schema.jsonconfig2.0 KB
schemas/consolidated.schema.jsonconfig6.8 KB
schemas/consolidation_batch.schema.jsonconfig3.7 KB
schemas/generated_tasks.schema.jsonconfig3.1 KB
schemas/host_capabilities.schema.jsonconfig38.3 KB
schemas/human_decision_verdicts.schema.jsonconfig2.6 KB
schemas/result_line.schema.jsonconfig10.1 KB
schemas/review_suggestions.schema.jsonconfig2.4 KB
schemas/state_file.schema.jsonconfig21.5 KB
schemas/status_file.schema.jsonconfig24.9 KB
schemas/status_json.schema.jsonconfig10.8 KB
schemas/subagent_tasks.schema.jsonconfig20.5 KB
schemas/task_decomposition.schema.jsonconfig1.2 KB
schemas/write_batch_result.schema.jsonconfig8.8 KB
templates/config/providers.override.yamlconfig17.2 KB
templates/consolidated_report_template.htmlother44.1 KB
templates/pr_report_template.htmlother181.2 KB
templates/report_template.htmlother67.5 KB
templates/shared/report_core.cssother4.1 KB
templates/shared/report_core.jsscript25.5 KB
update_plan_tasks.pyscript20.9 KB
utils/__init__.pyscript284 B
utils/advisory.pyscript4.8 KB
utils/apply_orchestrator_base.pyscript220.6 KB
utils/apply_output_helpers.pyscript10.9 KB
utils/apply_path_helpers.pyscript683 B
utils/apply_report.pyscript51.2 KB
utils/apply_selection_helpers.pyscript6.8 KB
utils/apply_suggestion_phase.pyscript11.7 KB
utils/atomic_io.pyscript12.9 KB
utils/attempt_classifier.pyscript31.4 KB
utils/backup.pyscript2.0 KB
utils/batch_done.pyscript79.2 KB
utils/batch_ledger.pyscript27.4 KB
utils/batch_prompt.pyscript29.3 KB
utils/batch_prompt_builder.pyscript12.2 KB
utils/batch_reply.pyscript32.9 KB
utils/cascade_archive.pyscript50.6 KB
utils/cascade_ledger.pyscript39.3 KB
utils/cascade_provenance.pyscript22.0 KB
utils/check_records.pyscript59.3 KB
utils/check_reply.pyscript9.5 KB
utils/check_tree.pyscript1.2 KB
utils/cli_common.pyscript91.2 KB
utils/cli_version_cache.pyscript14.0 KB
utils/clock.pyscript3.5 KB
utils/code_fix_batcher.pyscript30.6 KB
utils/codex_trust.pyscript12.9 KB
utils/commit_apply.pyscript73.3 KB
utils/commit_attribution.pyscript21.2 KB
utils/commit_brief.pyscript32.2 KB
utils/commit_front_door.pyscript81.6 KB
utils/commit_paths.pyscript18.0 KB
utils/commit_range.pyscript61.2 KB
utils/commit_report.pyscript19.9 KB
utils/commit_review.pyscript413 B
utils/commit_spelling.pyscript7.9 KB
utils/commit_write_guard.pyscript9.9 KB
utils/concurrency.pyscript11.6 KB
utils/consolidation.pyscript87.3 KB
utils/detach.pyscript186.8 KB
utils/dispatch_spawn.pyscript56.3 KB
utils/doctor.pyscript76.7 KB
utils/env_vars.pyscript20.5 KB
utils/failure_capture.pyscript10.1 KB
utils/file_discovery.pyscript11.0 KB
utils/filtering.pyscript19.9 KB
utils/git_diff.pyscript23.6 KB
utils/git_utils.pyscript86.6 KB
utils/held_folder.pyscript21.3 KB
utils/host_capabilities.pyscript48.0 KB
utils/host_install.pyscript77.6 KB
utils/host_paths.pyscript12.0 KB
utils/host_routes/__init__.pyscript5.7 KB
utils/host_routes/codex.pyscript18.3 KB
utils/host_routes/npm.pyscript12.2 KB
utils/host_shim.pyscript14.7 KB
utils/html_report_generator.pyscript118.9 KB
utils/human_decisions.pyscript38.3 KB
utils/human_task_plan.pyscript30.2 KB
utils/importance.pyscript6.2 KB
utils/improve_cascade.pyscript63.7 KB
utils/improve_cascade_next.pyscript22.2 KB
utils/improve_steps.pyscript32.4 KB
utils/improve_workflow.pyscript114.4 KB
utils/interactive.pyscript31.6 KB
utils/json_extractor.pyscript61.5 KB
utils/judge_vocab.pyscript5.3 KB
utils/known_checks.pyscript37.5 KB
utils/landlock_dispatch.pyscript103.8 KB
utils/landlock_exec.pyscript21.4 KB
utils/launch_limiter.pyscript13.5 KB
utils/llm_client.pyscript126.9 KB
utils/manual_tasks.pyscript14.9 KB
utils/metrics.pyscript68.1 KB
utils/next_phase.pyscript96.1 KB
utils/next_phase_words.pyscript6.4 KB
utils/output_handler.pyscript23.2 KB
utils/phase_walk.pyscript6.4 KB
utils/plan_updater.pyscript17.9 KB
utils/positionals.pyscript5.8 KB
utils/prior_changes.pyscript28.7 KB
utils/probe_state.pyscript17.9 KB
utils/process_capture.pyscript37.6 KB
utils/process_control.pyscript228.7 KB
utils/process_state.pyscript6.6 KB
utils/progress_tracker.pyscript92.2 KB
utils/prompt_channels.pyscript34.1 KB
utils/prompt_files.pyscript24.5 KB

display_decisions.py

View on GitHub
#!/usr/bin/env python3
"""
Display a formatted summary of suggestion/fix decisions from orchestrator output.

Reads orchestrator_output.json and shows which items will be applied, skipped,
need human review, etc., with their titles.

Usage:
    uv run --project "{SKILL_DIR}" -- python "{SKILL_DIR}/multi_llm.py" util decisions \
        --output-file plans/my-feature/apply-suggestions/orchestrator_output.json

    # Or auto-detect from plan path + phase:
    uv run --project "{SKILL_DIR}" -- python "{SKILL_DIR}/multi_llm.py" util decisions \
        --plan-file plans/my-feature.md --phase apply-suggestions
"""

import argparse
import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))

from utils.stream_bootstrap import bootstrap_streams
from utils import importance as _importance
from utils.human_decisions import judge_source_label
from utils.judge_vocab import JUDGE_SKIPPED, upgrade_legacy
from utils.output_handler import get_phase_dir


def load_output(path: str) -> dict:
    # An output written before the judge rename says `claude_skipped`.
    with open(path, "r", encoding="utf-8") as f:
        return upgrade_legacy(json.load(f))


def get_title(item: dict) -> str:
    return item.get("title") or item.get("theme") or "Untitled"


def get_importance(item: dict) -> str:
    return _importance.normalize(item.get("importance"))


def importance_badge(imp: str) -> str:
    return {"HIGH": "HIGH", "MEDIUM": "MED", "LOW": "LOW"}.get(imp, imp)


def format_item_line(item: dict, index: int) -> str:
    title = get_title(item)
    imp = importance_badge(get_importance(item))
    parts = [f"  {index}. [{imp}] {title}"]
    if item.get("auto_approved"):
        parts[0] += f"  (auto-approved: {item.get('auto_approval_reason', '')})"
    if item.get("edited"):
        parts[0] += "  (description edited)"
    return parts[0]


def display(data: dict, phase: str) -> None:
    is_task_suggestions = phase == "apply-task-suggestions"
    is_code_fixes = "fix" in phase
    if is_task_suggestions:
        item_noun_plural = "task suggestions"
    elif is_code_fixes:
        item_noun_plural = "fixes"
    else:
        item_noun_plural = "suggestions"

    to_apply = data.get("to_apply", [])
    needs_human = data.get("needs_human_review", [])
    skipped_items = data.get("skipped_items", [])
    user_skipped_items = data.get("user_skipped_items", [])
    edited_descriptions = data.get("edited_descriptions", [])
    summary = data.get("summary", {})

    # Build set of edited item IDs for marking in the apply list
    edited_ids = set()
    for ed in edited_descriptions:
        edited_ids.add(ed.get("id", ""))
        edited_ids.add(ed.get("title", ""))

    # Mark edited items in to_apply
    for item in to_apply:
        if item.get("title") in edited_ids:
            item["edited"] = True

    total_groups = summary.get("total_groups", summary.get("total_issues", 0))
    valid_count = len(to_apply)
    human_count = len(needs_human)
    user_skipped_count = len(user_skipped_items)

    # Header
    if is_task_suggestions:
        header = "Task Suggestion Decision Summary"
    elif is_code_fixes:
        header = "Code Fix Decision Summary"
    else:
        header = "Suggestion Decision Summary"
    print(f"\n{'=' * 60}")
    print(f"  {header}")
    print(f"{'=' * 60}")
    print(f"  Total groups: {total_groups}")
    print()

    counter = 1

    # Will Apply
    auto_count = sum(1 for i in to_apply if i.get("auto_approved"))
    label = f"WILL APPLY ({valid_count} {item_noun_plural})"
    if auto_count:
        label += f"  [{auto_count} auto-approved]"
    print(f"  + {label}")
    if to_apply:
        for item in to_apply:
            print(format_item_line(item, counter))
            counter += 1
    else:
        print("    (none)")
    print()

    # Needs Human Review
    if needs_human:
        print(f"  ? NEEDS HUMAN REVIEW ({human_count} {item_noun_plural})")
        for item in needs_human:
            title = get_title(item)
            imp = importance_badge(get_importance(item))
            reason = item.get("validation_reason", "")
            line = f"  {counter}. [{imp}] {title}"
            if reason:
                line += f"  -- {reason}"
            print(line)
            counter += 1
        print()

    # Invalid / Validation Skipped
    invalid_items = [i for i in skipped_items if i.get("status") == "invalid"]
    bulk_skipped = [i for i in skipped_items if i.get("status") == "bulk_skipped"]
    priority_filtered = [i for i in skipped_items if i.get("status") == "below_priority"]
    # The judge's own skips. Bucketed apart from `invalid` because they are the
    # opposite kind of outcome: validation said the item was wrong, whereas this
    # is a decision the judge agent made on the user's behalf about an item
    # validation could not settle — and this command is the one surface whose
    # whole job is to say what was decided.
    judge_skipped = [i for i in skipped_items if i.get("status") == JUDGE_SKIPPED]
    other_skipped = [i for i in skipped_items
                     if i.get("status") not in ("invalid", "bulk_skipped",
                                                "below_priority", JUDGE_SKIPPED)]

    if invalid_items:
        print(f"  x INVALID ({len(invalid_items)} {item_noun_plural})")
        for item in invalid_items:
            title = get_title(item)
            imp = importance_badge(get_importance(item))
            reason = item.get("reason", "")
            line = f"  {counter}. [{imp}] {title}"
            if reason:
                line += f"  -- {reason}"
            print(line)
            counter += 1
        print()

    if bulk_skipped:
        print(f"  - BULK SKIPPED ({len(bulk_skipped)} {item_noun_plural})")
        for item in bulk_skipped:
            title = get_title(item)
            imp = importance_badge(get_importance(item))
            reason = item.get("reason", "")
            line = f"  {counter}. [{imp}] {title}"
            if reason:
                line += f"  -- {reason}"
            print(line)
            counter += 1
        print()

    if judge_skipped:
        print(f"  ! SKIPPED BY THE JUDGE AGENT ({len(judge_skipped)} {item_noun_plural})")
        for item in judge_skipped:
            title = get_title(item)
            imp = importance_badge(get_importance(item))
            reason = item.get("reason", "")
            source = item.get("decision_source", "")
            line = f"  {counter}. [{imp}] {title}"
            if source:
                line += f"  ({judge_source_label(source)})"
            if reason:
                line += f"  -- {reason}"
            print(line)
            counter += 1
        print()

    if priority_filtered:
        print(f"  - BELOW PRIORITY ({len(priority_filtered)} {item_noun_plural})")
        for item in priority_filtered:
            title = get_title(item)
            imp = importance_badge(get_importance(item))
            print(f"  {counter}. [{imp}] {title}")
            counter += 1
        print()

    if other_skipped:
        print(f"  - SKIPPED ({len(other_skipped)} {item_noun_plural})")
        for item in other_skipped:
            title = get_title(item)
            imp = importance_badge(get_importance(item))
            reason = item.get("reason", "")
            line = f"  {counter}. [{imp}] {title}"
            if reason:
                line += f"  -- {reason}"
            print(line)
            counter += 1
        print()

    # User Skipped
    if user_skipped_items:
        print(f"  ~ USER SKIPPED ({user_skipped_count} {item_noun_plural})")
        for item in user_skipped_items:
            title = get_title(item)
            imp = importance_badge(get_importance(item))
            print(f"  {counter}. [{imp}] {title}")
            counter += 1
        print()

    # Edited descriptions note
    edited_in_apply = [i for i in to_apply if i.get("edited")]
    if edited_in_apply:
        print(f"  * {len(edited_in_apply)} {item_noun_plural} have user-edited descriptions")
        print()

    print(f"{'=' * 60}")
    print()


def resolve_output_file(args: argparse.Namespace) -> str:
    if args.output_file:
        return args.output_file

    if not args.plan_file or not args.phase:
        print("ERROR: Provide --output-file or both --plan-file and --phase", file=sys.stderr)
        sys.exit(1)

    phase_map = {
        "apply-suggestions": "apply-suggestions",
        "apply-code-fixes": "apply-fixes",
        "apply-fixes": "apply-fixes",
        "apply-task-suggestions": "apply-task-suggestions",
    }
    phase_dir_name = phase_map.get(args.phase, args.phase)
    phase_dir = get_phase_dir(Path(args.plan_file), phase_dir_name)
    return str(phase_dir / "orchestrator_output.json")


def main(argv=None) -> int:
    bootstrap_streams()
    parser = argparse.ArgumentParser(description="Display decision summary from orchestrator output")
    parser.add_argument("--output-file", help="Path to orchestrator_output.json")
    parser.add_argument("--plan-file", help="Path to plan file (used with --phase)")
    parser.add_argument(
        "--phase",
        choices=["apply-suggestions", "apply-code-fixes", "apply-task-suggestions"],
        help="Phase name: apply-suggestions, apply-code-fixes, or apply-task-suggestions",
    )
    args = parser.parse_args(argv)

    output_path = resolve_output_file(args)
    if args.phase:
        phase = args.phase
    elif "task-suggestions" in output_path:
        phase = "apply-task-suggestions"
    elif "fix" in output_path:
        phase = "apply-code-fixes"
    else:
        phase = "apply-suggestions"

    if not Path(output_path).exists():
        print(f"ERROR: Output file not found: {output_path}", file=sys.stderr)
        sys.exit(1)

    data = load_output(output_path)
    display(data, phase)
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))
receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementationStars: 321kUpdated Claude CodeCodexVerifiedInstall
requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirementsStars: 321kUpdated Claude CodeCodexVerifiedInstall
using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallbackStars: 321kUpdated Claude CodeCodexVerifiedInstall
verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions alwaysStars: 321kUpdated Claude CodeCodexVerifiedInstall