Skip to content

hashgraph-online/awesome-codex-plugins / plugins/beastlabai/multi-llm-plugin/skills/multi-llm

multi-llm

Flagged

Orchestrate plan reviews, task generation, implementation and code reviews across several LLM CLIs in parallel, or ask every model one free-text question about a plan. Use when the user says: review my plan, multi-llm, run a code review, generate tasks from a plan, implement this plan, ask the models about my plan.

FileKindSize
SKILL.mdskill md4.0 KB
apply_code_fixes_orchestrator.pyscript56.6 KB
apply_commit_fixes_orchestrator.pyscript2.3 KB
apply_suggestions_orchestrator.pyscript4.5 KB
apply_task_suggestions_orchestrator.pyscript19.1 KB
ask_orchestrator.pyscript162.5 KB
assets/icons/Anthropic.svgother584 B
assets/icons/Codex.svgother1.3 KB
assets/icons/Copilot.svgother1.5 KB
assets/icons/Cursor.svgother793 B
assets/icons/GoogleGemini.svgother623 B
assets/icons/Goose.svgother3.2 KB
assets/icons/Kilocode.svgother531 B
assets/icons/MiniMax.pngbinary1.5 KB
assets/icons/Mistral.svgother717 B
assets/icons/MoonshotAI.svgother957 B
assets/icons/Nvidia.svgother2.4 KB
assets/icons/OhMyPi.svgother434 B
assets/icons/OpenAI.svgother1.7 KB
assets/icons/Opencode.svgother325 B
assets/icons/Pi.svgother440 B
assets/icons/Qwen.svgother1.7 KB
assets/icons/Zhipu.pngbinary442 B
assets/icons/xAI.svgother343 B
assets/omp/overlay.ymlconfig2.4 KB
check_workflow_prerequisites.pyscript74.5 KB
code_review_orchestrator.pyscript324.0 KB
display_decisions.pyscript9.8 KB
finalize_tracking.pyscript1.3 KB
full_workflow_orchestrator.pyscript58.8 KB
generate_tasks_orchestrator.pyscript44.2 KB
hosts/capabilities.yamlconfig49.9 KB
hosts/codex/plugin-skill.md.tmplother2.8 KB
hosts/codex/shim.md.tmplother7.3 KB
hosts/opencode/plugin-command.md.tmplother2.2 KB
hosts/opencode/shim.md.tmplother6.8 KB
hosts/render_capabilities.pyscript48.4 KB
hosts/snippets/poll-loop.mdmarkdown4.4 KB
hosts/snippets/uv-preflight.mdmarkdown2.2 KB
hosts/uv-missing.txtmarkdown1.5 KB
implement_orchestrator.pyscript156.0 KB
improve_code_orchestrator.pyscript1.6 KB
improve_commits_orchestrator.pyscript2.5 KB
improve_plan_orchestrator.pyscript1.6 KB
init_config.pyscript57.6 KB
instructions/apply-code-fixes.mdmarkdown52.0 KB
instructions/apply-commit-fixes.mdmarkdown10.3 KB
instructions/apply-suggestions.mdmarkdown48.2 KB
instructions/apply-task-suggestions.mdmarkdown53.9 KB
instructions/ask.mdmarkdown27.6 KB
instructions/full-workflow.mdmarkdown24.6 KB
instructions/generate-tasks.mdmarkdown18.1 KB
instructions/implement.mdmarkdown54.0 KB
instructions/improve-code.mdmarkdown25.0 KB
instructions/improve-commits.mdmarkdown24.8 KB
instructions/improve-plan.mdmarkdown26.2 KB
instructions/init-config.mdmarkdown6.6 KB
instructions/review-code.mdmarkdown43.5 KB
instructions/review-commits.mdmarkdown11.4 KB
instructions/review-plan.mdmarkdown38.4 KB
instructions/review-tasks.mdmarkdown37.8 KB
modes.pyscript58.8 KB
multi_llm.pyscript101.1 KB
prompts/ask.txtmarkdown948 B
prompts/code_review.txtmarkdown4.5 KB
prompts/commit_review.txtmarkdown6.2 KB
prompts/consolidate_suggestions.txtmarkdown3.1 KB
prompts/implementation_task.txtmarkdown1.4 KB
prompts/plan_review.txtmarkdown8.0 KB
prompts/task_review.txtmarkdown8.3 KB
providers.yamlconfig19.6 KB
pyproject.tomlmanifest6.8 KB
references/apply-write-executor.mdmarkdown3.0 KB
references/consolidation.mdmarkdown5.6 KB
references/host-capabilities.mdmarkdown13.5 KB
references/human-decision-batch.mdmarkdown17.2 KB
references/implement-prerequisites.mdmarkdown6.0 KB
references/implement-write-executor.mdmarkdown4.9 KB
references/interrupted-run-recovery.mdmarkdown3.1 KB
references/own-check.mdmarkdown4.3 KB
references/resume-detection.mdmarkdown8.5 KB
references/revalidation.mdmarkdown6.9 KB
references/review-code-apply-fixes.mdmarkdown4.3 KB
references/salvage-handling.mdmarkdown12.5 KB
references/summary-format.mdmarkdown10.0 KB
references/uv-check.mdmarkdown5.7 KB
references/wave-batching.mdmarkdown5.2 KB
review_commits_orchestrator.pyscript2.3 KB
review_plan_orchestrator.pyexecutablescript22.3 KB
review_tasks_orchestrator.pyscript10.7 KB
schemas/cascade_record.schema.jsonconfig6.6 KB
schemas/code_review_issues.schema.jsonconfig2.0 KB
schemas/consolidated.schema.jsonconfig6.8 KB
schemas/consolidation_batch.schema.jsonconfig3.7 KB
schemas/generated_tasks.schema.jsonconfig3.1 KB
schemas/host_capabilities.schema.jsonconfig38.3 KB
schemas/human_decision_verdicts.schema.jsonconfig2.6 KB
schemas/result_line.schema.jsonconfig10.1 KB
schemas/review_suggestions.schema.jsonconfig2.4 KB
schemas/state_file.schema.jsonconfig21.5 KB
schemas/status_file.schema.jsonconfig24.9 KB
schemas/status_json.schema.jsonconfig10.8 KB
schemas/subagent_tasks.schema.jsonconfig20.5 KB
schemas/task_decomposition.schema.jsonconfig1.2 KB
schemas/write_batch_result.schema.jsonconfig8.8 KB
templates/config/providers.override.yamlconfig17.2 KB
templates/consolidated_report_template.htmlother44.1 KB
templates/pr_report_template.htmlother181.2 KB
templates/report_template.htmlother67.5 KB
templates/shared/report_core.cssother4.1 KB
templates/shared/report_core.jsscript25.5 KB
update_plan_tasks.pyscript20.9 KB
utils/__init__.pyscript284 B
utils/advisory.pyscript4.8 KB
utils/apply_orchestrator_base.pyscript220.6 KB
utils/apply_output_helpers.pyscript10.9 KB
utils/apply_path_helpers.pyscript683 B
utils/apply_report.pyscript51.2 KB
utils/apply_selection_helpers.pyscript6.8 KB
utils/apply_suggestion_phase.pyscript11.7 KB
utils/atomic_io.pyscript12.9 KB
utils/attempt_classifier.pyscript31.4 KB
utils/backup.pyscript2.0 KB
utils/batch_done.pyscript79.2 KB
utils/batch_ledger.pyscript27.4 KB
utils/batch_prompt.pyscript29.3 KB
utils/batch_prompt_builder.pyscript12.2 KB
utils/batch_reply.pyscript32.9 KB
utils/cascade_archive.pyscript50.6 KB
utils/cascade_ledger.pyscript39.3 KB
utils/cascade_provenance.pyscript22.0 KB
utils/check_records.pyscript59.3 KB
utils/check_reply.pyscript9.5 KB
utils/check_tree.pyscript1.2 KB
utils/cli_common.pyscript91.2 KB
utils/cli_version_cache.pyscript14.0 KB
utils/clock.pyscript3.5 KB
utils/code_fix_batcher.pyscript30.6 KB
utils/codex_trust.pyscript12.9 KB
utils/commit_apply.pyscript73.3 KB
utils/commit_attribution.pyscript21.2 KB
utils/commit_brief.pyscript32.2 KB
utils/commit_front_door.pyscript81.6 KB
utils/commit_paths.pyscript18.0 KB
utils/commit_range.pyscript61.2 KB
utils/commit_report.pyscript19.9 KB
utils/commit_review.pyscript413 B
utils/commit_spelling.pyscript7.9 KB
utils/commit_write_guard.pyscript9.9 KB
utils/concurrency.pyscript11.6 KB
utils/consolidation.pyscript87.3 KB
utils/detach.pyscript186.8 KB
utils/dispatch_spawn.pyscript56.3 KB
utils/doctor.pyscript76.7 KB
utils/env_vars.pyscript20.5 KB
utils/failure_capture.pyscript10.1 KB
utils/file_discovery.pyscript11.0 KB
utils/filtering.pyscript19.9 KB
utils/git_diff.pyscript23.6 KB
utils/git_utils.pyscript86.6 KB
utils/held_folder.pyscript21.3 KB
utils/host_capabilities.pyscript48.0 KB
utils/host_install.pyscript77.6 KB
utils/host_paths.pyscript12.0 KB
utils/host_routes/__init__.pyscript5.7 KB
utils/host_routes/codex.pyscript18.3 KB
utils/host_routes/npm.pyscript12.2 KB
utils/host_shim.pyscript14.7 KB
utils/html_report_generator.pyscript118.9 KB
utils/human_decisions.pyscript38.3 KB
utils/human_task_plan.pyscript30.2 KB
utils/importance.pyscript6.2 KB
utils/improve_cascade.pyscript63.7 KB
utils/improve_cascade_next.pyscript22.2 KB
utils/improve_steps.pyscript32.4 KB
utils/improve_workflow.pyscript114.4 KB
utils/interactive.pyscript31.6 KB
utils/json_extractor.pyscript61.5 KB
utils/judge_vocab.pyscript5.3 KB
utils/known_checks.pyscript37.5 KB
utils/landlock_dispatch.pyscript103.8 KB
utils/landlock_exec.pyscript21.4 KB
utils/launch_limiter.pyscript13.5 KB
utils/llm_client.pyscript126.9 KB
utils/manual_tasks.pyscript14.9 KB
utils/metrics.pyscript68.1 KB
utils/next_phase.pyscript96.1 KB
utils/next_phase_words.pyscript6.4 KB
utils/output_handler.pyscript23.2 KB
utils/phase_walk.pyscript6.4 KB
utils/plan_updater.pyscript17.9 KB
utils/positionals.pyscript5.8 KB
utils/prior_changes.pyscript28.7 KB
utils/probe_state.pyscript17.9 KB
utils/process_capture.pyscript37.6 KB
utils/process_control.pyscript228.7 KB
utils/process_state.pyscript6.6 KB
utils/progress_tracker.pyscript92.2 KB
utils/prompt_channels.pyscript34.1 KB
utils/prompt_files.pyscript24.5 KB

utils/launch_limiter.py

View on GitHub
"""Launch spacing: a minimum interval between two dispatches of one provider.

A CLI with a measured concurrent-start fault declares
``LLMProvider.LAUNCH_STAGGER_S`` (cursor-agent, 2 s); every other adapter
declares 0 and never meets this module. One :class:`LaunchLimiter` serves a
whole fan-out (every slot, and every attempt of every slot) and spaces the
*dispatches*, not the slots' admissions:

- a slot takes its turn only once it holds its semaphores (:meth:`LaunchLimiter.slot`),
  so slots that queued behind other providers cannot dispatch together;
- every dispatch takes its turn right before it starts (:func:`attempt_turn`,
  :func:`attempt_turn_blocking`): a slot's first dispatch confirms the slot's
  turn, after whatever preparation the lane did, or waits again when another
  slot of the provider took one meanwhile, and every ladder retry takes a fresh
  one, so a retry cannot land beside another slot's first attempt;
- the wait is charged to whatever deadline the caller is running against, and
  a turn is never granted once that deadline has passed.

A blocking waiter (Lane B's retry, on a worker thread) is not stopped when its
asyncio slot is cancelled, just as the dispatch it precedes is not: it keeps
its place and takes its turn. No adapter that declares a launch interval
serves the executor's profiles today.

See docs/development/delegation.md#read-and-analyze-executor.

Pure apart from the clock and the sleep: nothing here imports another skill
module, and the tracker transitions are callbacks.
"""

from __future__ import annotations

import asyncio
import contextlib
import contextvars
import threading
import time
from typing import AsyncIterator, Callable, Dict, List, Mapping, Optional

#: How early a woken waiter may claim its turn: event-loop timers fire up to
#: one clock resolution before the time they were set for.
_EPSILON = 1e-6

#: The launch turn of the fan-out slot this code runs in, or ``None`` outside
#: one (a direct call, or a provider that is not spaced). Set for the slot's
#: coroutine only; ``asyncio.to_thread`` copies it into the worker thread, which
#: is how Lane B's blocking retry loop finds it.
_CURRENT: contextvars.ContextVar[Optional["SlotLaunch"]] = contextvars.ContextVar(
    "multi_llm_slot_launch", default=None
)


class LaunchLimiter:
    """Per-provider minimum spacing between dispatches, shared by one fan-out.

    A turn is taken at the moment it is claimed, never reserved ahead: a waiter
    sleeps until its estimated turn (its place in the line times the interval)
    and claims only when it has waited longest for that provider and the
    interval since the last claim has passed. So waiters are served in arrival
    order, and one that stops waiting (a cancelled slot, a spent budget) takes
    no turn: it wakes every waiter behind it, which then works its turn out
    again from its new place in the line. The state sits behind a thread lock
    because Lane B's retries wait on worker threads, and a wake reaches either
    kind of waiter (an event-loop task or a worker thread).
    """

    def __init__(
        self,
        intervals: Mapping[str, float],
        *,
        clock: Optional[Callable[[], float]] = None,
    ) -> None:
        self._intervals: Dict[str, float] = {
            name: float(seconds)
            for name, seconds in intervals.items()
            if seconds and float(seconds) > 0
        }
        self._clock = clock
        self._lock = threading.Lock()
        self._last: Dict[str, float] = {}
        self._holder: Dict[str, object] = {}
        self._waiting: Dict[str, List[_Waiter]] = {}

    def spaces(self, provider: str) -> bool:
        """Whether *provider* has a launch interval at all."""
        return provider in self._intervals

    def now(self) -> float:
        """The limiter's clock: the injected one, else the clock the sleep counts in.

        On the event loop's thread that is the loop's own clock (what
        ``asyncio.sleep`` waits on), on any other thread ``time.monotonic``;
        a stock loop's clock *is* ``time.monotonic``.
        """
        if self._clock is not None:
            return self._clock()
        try:
            return asyncio.get_running_loop().time()
        except RuntimeError:
            return time.monotonic()

    def _next_wait(
        self,
        provider: str,
        waiter: object,
        give_up: Optional[float],
        wake: Callable[[], None],
    ) -> Optional[float]:
        """Claim *provider*'s turn for *waiter* (``0.0``), the seconds to sleep
        before asking again, or ``None`` once *give_up* has passed.

        *wake* is how a departing waiter ahead of this one cuts its sleep short.
        Expiry is checked before the turn: a turn granted at the deadline is one
        no dispatch can use, and it would still push every waiter behind it
        back by an interval.
        """
        interval = self._intervals[provider]
        with self._lock:
            now = self.now()
            if give_up is not None and now + _EPSILON >= give_up:
                return None
            queue = self._waiting.setdefault(provider, [])
            position = _index(queue, waiter)
            if position is None:
                queue.append(_Waiter(waiter, wake))
                position = len(queue) - 1
            last = self._last.get(provider)
            ready_at = now if last is None else last + interval
            if position == 0 and now + _EPSILON >= ready_at:
                queue.pop(0)
                self._last[provider] = now
                self._holder[provider] = waiter
                return 0.0
            wait = max(0.0, ready_at - now) + position * interval
            return wait if give_up is None else min(wait, give_up - now)

    def _leave(self, provider: str, waiter: object) -> None:
        """Take *waiter* out of the line (a no-op once it claimed), and wake
        everyone behind it: each of them is now one place closer."""
        with self._lock:
            queue = self._waiting.get(provider) or []
            position = _index(queue, waiter)
            if position is None:
                return
            queue.pop(position)
            behind = [entry.wake for entry in queue[position:]]
        for wake in behind:
            wake()

    def confirm(self, provider: str, holder: object) -> bool:
        """Move *holder*'s turn to now, its actual dispatch; ``False`` when
        another waiter claimed a turn since, and *holder* must wait again."""
        with self._lock:
            if self._holder.get(provider) is not holder:
                return False
            self._last[provider] = max(self._last[provider], self.now())
            return True

    async def wait_turn(
        self,
        provider: str,
        waiter: object,
        *,
        budget: Optional[float] = None,
        on_wait: Optional[Callable[[float], None]] = None,
    ) -> bool:
        """Wait for and claim *provider*'s turn; ``False`` when *budget*
        seconds ran out first. An unspaced provider returns at once."""
        if not self.spaces(provider):
            return True
        give_up = None if budget is None else self.now() + max(0.0, budget)
        loop = asyncio.get_running_loop()
        woken = asyncio.Event()

        def wake() -> None:
            # From any thread: the departing waiter may be a worker thread's.
            with contextlib.suppress(RuntimeError):  # the loop has closed
                loop.call_soon_threadsafe(woken.set)

        try:
            while True:
                woken.clear()
                wait = self._next_wait(provider, waiter, give_up, wake)
                if not wait:
                    return wait is not None
                if on_wait is not None:
                    on_wait(wait)
                with contextlib.suppress(asyncio.TimeoutError):
                    await asyncio.wait_for(woken.wait(), wait)
        finally:
            self._leave(provider, waiter)

    def wait_turn_blocking(
        self,
        provider: str,
        waiter: object,
        *,
        budget: Optional[float] = None,
        on_wait: Optional[Callable[[float], None]] = None,
        pause: Callable[[threading.Event, float], object] = threading.Event.wait,
    ) -> bool:
        """:meth:`wait_turn` for a worker thread.

        *pause* sleeps for at most the seconds given, or until the event is
        set (a waiter ahead left the line): ``threading.Event.wait``.
        """
        if not self.spaces(provider):
            return True
        give_up = None if budget is None else self.now() + max(0.0, budget)
        woken = threading.Event()
        try:
            while True:
                woken.clear()
                wait = self._next_wait(provider, waiter, give_up, woken.set)
                if not wait:
                    return wait is not None
                if on_wait is not None:
                    on_wait(wait)
                pause(woken, wait)
        finally:
            self._leave(provider, waiter)

    @contextlib.asynccontextmanager
    async def slot(
        self,
        provider: str,
        *,
        label: str,
        on_wait: Optional[Callable[[float], None]] = None,
        after_wait: Optional[Callable[[], None]] = None,
    ) -> AsyncIterator[Optional["SlotLaunch"]]:
        """One fan-out slot's launch turn, entered once its semaphores are held.

        Waits for and claims *provider*'s turn, then installs the slot's
        :class:`SlotLaunch` for the coroutine run inside, where a ladder's
        attempts confirm or renew it. *on_wait* gets every wait (the tracker's
        ``staggering``, with the seconds), *after_wait* runs once a wait has
        ended (the row back to ``queued`` until the dispatch). An unspaced
        provider enters at once and installs nothing.
        """
        if not self.spaces(provider):
            yield None
            return
        launch = SlotLaunch(self, provider, label=label, on_wait=on_wait)
        if await launch.claim() and after_wait is not None:
            after_wait()
        token = _CURRENT.set(launch)
        try:
            yield launch
        finally:
            _CURRENT.reset(token)


class SlotLaunch:
    """One fan-out slot's place in its provider's launch spacing."""

    def __init__(
        self,
        limiter: LaunchLimiter,
        provider: str,
        *,
        label: str,
        on_wait: Optional[Callable[[float], None]] = None,
    ) -> None:
        self.limiter = limiter
        self.provider = provider
        self.label = label
        self._on_wait = on_wait
        # The slot holds a turn that no dispatch has used yet.
        self._unconfirmed = False

    def _announcer(self, what: str, *, tracked: bool) -> Callable[[float], None]:
        """One line per wait, the first time it sleeps; the tracker every time."""
        said = []

        def announce(seconds: float) -> None:
            if not said:
                said.append(True)
                print(
                    f"[{self.label}] Waiting {seconds:.1f}s {what} "
                    f"(staggered launch)..."
                )
            if tracked and self._on_wait is not None:
                self._on_wait(seconds)

        return announce

    async def claim(self) -> bool:
        """The slot's own turn; ``True`` when it had to wait for it."""
        waited = []
        announce = self._announcer("before starting", tracked=True)

        def on_wait(seconds: float) -> None:
            waited.append(seconds)
            announce(seconds)

        await self.limiter.wait_turn(self.provider, self, on_wait=on_wait)
        self._unconfirmed = True
        return bool(waited)

    def _confirmed(self) -> bool:
        if not self._unconfirmed:
            return False
        self._unconfirmed = False
        return self.limiter.confirm(self.provider, self)

    async def attempt(self, budget: Optional[float]) -> bool:
        """The turn for the slot's next dispatch, within *budget* seconds."""
        if self._confirmed():
            return True
        return await self.limiter.wait_turn(
            self.provider, self, budget=budget,
            on_wait=self._announcer("for its launch turn", tracked=True),
        )

    def attempt_blocking(self, budget: Optional[float]) -> bool:
        """:meth:`attempt` on a worker thread, where no tracker is written."""
        if self._confirmed():
            return True
        return self.limiter.wait_turn_blocking(
            self.provider, self, budget=budget,
            on_wait=self._announcer("for its launch turn", tracked=False),
        )


async def attempt_turn(budget: Optional[float] = None) -> bool:
    """The current slot's launch turn for its next dispatch, if it is spaced.

    Called right before every attempt a ladder dispatches, with what is left
    of the slot's deadline as *budget*; ``True`` outside a spaced slot.
    """
    launch = _CURRENT.get()
    return True if launch is None else await launch.attempt(budget)


def attempt_turn_blocking(budget: Optional[float] = None) -> bool:
    """:func:`attempt_turn` for a ladder that runs on a worker thread."""
    launch = _CURRENT.get()
    return True if launch is None else launch.attempt_blocking(budget)


class _Waiter:
    """One place in a provider's line: who waits, and how to wake them."""

    __slots__ = ("who", "wake")

    def __init__(self, who: object, wake: Callable[[], None]) -> None:
        self.who = who
        self.wake = wake


def _index(queue: List[_Waiter], waiter: object) -> Optional[int]:
    for position, queued in enumerate(queue):
        if queued.who is waiter:
            return position
    return None
receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementationStars: 321kUpdated Claude CodeCodexVerifiedInstall
requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirementsStars: 321kUpdated Claude CodeCodexVerifiedInstall
using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallbackStars: 321kUpdated Claude CodeCodexVerifiedInstall
verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions alwaysStars: 321kUpdated Claude CodeCodexVerifiedInstall