Skip to content

praneybehl/llm-wiki-plugin / skills/llm-wiki

llm-wiki

Verified

Build and maintain an LLM-curated knowledge base from papers, articles, transcripts, notes and project findings. Ingest sources into linked Markdown, answer questions with citations, lint or upgrade a wiki, and preserve useful synthesis across sessions and agents. Also capture verified task experience, consolidate success/failure patterns, and propose, evaluate, apply or roll back evidence-linked skill improvements. Trigger on "add this to my wiki", "what does the wiki say", "learn from this task", "improve this procedure", /wiki:learn or /wiki:evolve. Works on a local folder or a hosted wiki URL (MCP). Use index-first navigation, atomic pages and hybrid search.

FileKindSize
SKILL.mdskill md24.6 KB
assets/SCHEMA.md.templateother7.4 KB
assets/decision-questions.jsonconfig3.1 KB
assets/evolution_README.md.templateother2.3 KB
assets/experience.json.templateother660 B
assets/graph_README.md.templateother2.0 KB
assets/graph_gitignore.templateother27 B
assets/index.md.templateother775 B
assets/log.md.templateother499 B
assets/ontology.yaml.templateother3.2 KB
assets/page.md.templateother805 B
assets/pattern.md.templateother839 B
assets/wiki-cache_gitignore.templateother14 B
references/agent-memory-integration.mdmarkdown7.5 KB
references/architecture.mdmarkdown9.6 KB
references/decision-models.mdmarkdown6.5 KB
references/evolution-workflow.mdmarkdown19.7 KB
references/graph-workflow.mdmarkdown7.1 KB
references/hosted-mode.mdmarkdown11.4 KB
references/ingest-workflow.mdmarkdown11.1 KB
references/lint-workflow.mdmarkdown8.3 KB
references/page-conventions.mdmarkdown5.8 KB
references/query-workflow.mdmarkdown8.6 KB
references/retrieval-setup.mdmarkdown5.6 KB
references/scaling-playbook.mdmarkdown6.8 KB
scripts/init_wiki.pyscript9.5 KB
scripts/setup_wiki.pyexecutablescript2.5 KB
scripts/test_link_resolution.pyscript6.2 KB
scripts/wiki_decide.pyexecutablescript31.6 KB
scripts/wiki_evolve.pyscript29.1 KB
scripts/wiki_evolve_agent.pyscript15.1 KB
scripts/wiki_evolve_api.pyscript7.0 KB
scripts/wiki_evolve_claude.pyscript2.4 KB
scripts/wiki_evolve_loop.pyscript43.4 KB
scripts/wiki_evolve_stream.pyscript13.0 KB
scripts/wiki_graph_extract.pyscript21.0 KB
scripts/wiki_graph_lint.pyscript22.0 KB
scripts/wiki_graph_query.pyscript9.3 KB
scripts/wiki_lint.pyscript14.5 KB
scripts/wiki_remote.pyexecutablescript76.6 KB
scripts/wiki_search.pyscript35.9 KB
scripts/wiki_stats.pyscript5.9 KB

scripts/wiki_decide.py

View on GitHub
#!/usr/bin/env python3
"""
wiki_decide.py: optional typed-decision checks for an LLM Wiki (stdlib only).

A decision model reads a state and answers typed questions with a choice and
probabilities. It never writes text. This tool is off by default. It sends
wiki text to a backend only when you select that backend with --backend or
the LLM_WIKI_DECISIONS environment variable.

Backends:
    local       llama-server with LiquidAI/d1-2B-GGUF (default http://127.0.0.1:8080). No key.
    typesafe    TypeSafe Jev (https://api.typesafe.ai). Needs TYPESAFE_API_KEY.
    openrouter  Jev through OpenRouter (https://openrouter.ai/api). Needs OPENROUTER_API_KEY.
    liquid      Liquid decisions API (https://api.liquid.ai/decisions). Needs LIQUID_API_KEY.
    hosted      Your hosted wiki server (/api/v1/decide). Uses the wiki_remote.py profile, URL,
                token and wiki. --profile wins over LLM_WIKI_URL and LLM_WIKI_TOKEN.
                LLM_WIKI_CONFIG selects another config file.

Environment:
    LLM_WIKI_DECISIONS          Backend name (default: off).
    LLM_WIKI_DECISIONS_URL      Backend base URL.
    LLM_WIKI_DECISIONS_MODEL    Model name.

Commands:
    ask [FILE]                     Send a raw {state, questions} JSON request (FILE or stdin).
    screen FILE                    Check if text gives orders to an AI system (D5).
    shortlist SOURCE [--wiki DIR]  Rank wiki pages that a new source changes (D4).
    contradictions [--wiki DIR]    Find section pairs that contradict (D3).
    check                          Test the backend connection.

On a backend error the checks fail open: the output keeps the unchecked
result, sets "failed_open": true, and a note goes to stderr.

Examples:
    LLM_WIKI_DECISIONS=local python wiki_decide.py check
    python wiki_decide.py --backend openrouter screen raw/article.md
    python wiki_decide.py --backend local shortlist raw/paper.md --wiki wiki
"""

from __future__ import annotations

import argparse
import http.client
import math
import json
import os
import re
import subprocess
import shutil
import sys
import time
import urllib.error
import urllib.request
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from typing import Callable

SCRIPT_DIR = Path(__file__).resolve().parent
QUESTIONS_PATH = SCRIPT_DIR.parent / "assets" / "decision-questions.json"

BACKENDS = {
    "local": {"url": "http://127.0.0.1:8080", "model": "d1-2B", "key": None},
    "typesafe": {"url": "https://api.typesafe.ai", "model": "jev-1.14.0", "key": "TYPESAFE_API_KEY"},
    "openrouter": {"url": "https://openrouter.ai/api", "model": "typesafe/jev-1.14", "key": "OPENROUTER_API_KEY"},
    "liquid": {"url": "https://api.liquid.ai/decisions", "model": "d1.1", "key": "LIQUID_API_KEY"},
    "hosted": {"url": None, "model": "server", "key": None},
}
BACKEND_CHOICES = ("off", *BACKENDS)
DEFAULT_TIMEOUT = 8.0
MAX_WORKERS = 8


def load_questions() -> dict:
    return json.loads(QUESTIONS_PATH.read_text(encoding="utf-8"))["checks"]


QUESTIONS = load_questions()


class DecisionError(Exception):
    """A backend call failed. `code` is one of: unavailable, insufficient_credits,
    timeout, http, invalid_answer, bad_request."""

    def __init__(self, code: str, message: str, status: int | None = None):
        super().__init__(message)
        self.code = code
        self.status = status


Transport = Callable[[dict], dict]


class DecisionClient:
    """Posts System One requests to one backend. `transport` replaces HTTP in tests."""

    def __init__(
        self,
        backend: str,
        url: str,
        model: str,
        key: str | None = None,
        timeout: float = DEFAULT_TIMEOUT,
        transport: Transport | None = None,
        wiki: str | None = None,
    ):
        self.backend = backend
        self.url = url
        self.model = model
        self.key = key
        self.timeout = timeout
        self.transport = transport
        self.wiki = wiki

    @property
    def endpoint(self) -> str:
        base = self.url.rstrip("/")
        if self.backend == "hosted":
            return base if base.endswith("/api/v1/decide") else base + "/api/v1/decide"
        return base if base.endswith("/v1/systemone") else base + "/v1/systemone"

    def ask(self, request: dict) -> dict:
        """Return {"answers", "model", "latency_ms"}. Raises DecisionError."""
        if not isinstance(request, dict) or "state" not in request or not isinstance(request.get("questions"), dict):
            raise DecisionError("bad_request", "request needs state and questions")
        started = time.monotonic()
        if self.transport is not None:
            raw = self.transport({"model": self.model, **request})
        else:
            raw = self._post(request)
        payload = raw.get("result") if isinstance(raw, dict) and not isinstance(raw.get("answers"), dict) and isinstance(raw.get("result"), dict) else raw
        if not isinstance(payload, dict) or not isinstance(payload.get("answers"), dict):
            raise DecisionError("invalid_answer", "backend response has no answers object")
        return {
            "answers": payload["answers"],
            "model": payload.get("model") if isinstance(payload.get("model"), str) else self.model,
            "latency_ms": int((time.monotonic() - started) * 1000),
        }

    def _post(self, request: dict) -> dict:
        body = {"state": request["state"], "questions": request["questions"]}
        if self.backend != "hosted":
            body = {"model": self.model, **body}
        headers = {"Content-Type": "application/json", "Accept": "application/json"}
        if self.key:
            headers["Authorization"] = f"Bearer {self.key}"
        if self.backend == "hosted" and self.wiki:
            headers["X-LLM-Wiki"] = self.wiki
        req = urllib.request.Request(self.endpoint, data=json.dumps(body).encode("utf-8"), headers=headers, method="POST")
        try:
            with urllib.request.urlopen(req, timeout=self.timeout) as resp:
                text = resp.read().decode("utf-8", errors="replace")
        except urllib.error.HTTPError as exc:
            try:
                detail = exc.read().decode("utf-8", errors="replace")[:300]
            except (OSError, http.client.HTTPException):
                detail = ""
            code = "insufficient_credits" if exc.code == 402 or "insufficient" in detail.lower() and "credit" in detail.lower() else "http"
            raise DecisionError(code, f"HTTP {exc.code}: {detail}", exc.code) from None
        except http.client.HTTPException as exc:
            # IncompleteRead, BadStatusLine, RemoteDisconnected and other protocol errors.
            raise DecisionError("unavailable", f"bad response from {self.endpoint}: {exc!r}") from None
        except (TimeoutError, OSError) as exc:
            reason = getattr(exc, "reason", exc)
            if isinstance(reason, TimeoutError) or "timed out" in str(reason).lower():
                raise DecisionError("timeout", f"no answer after {self.timeout} s") from None
            raise DecisionError("unavailable", f"cannot reach {self.endpoint}: {reason}") from None
        try:
            return json.loads(text)
        except json.JSONDecodeError:
            raise DecisionError("invalid_answer", "backend response is not JSON") from None


def note(message: str) -> None:
    print(f"decisions: {message}", file=sys.stderr)


def read_remote_profile(profile: str | None, url_override: str | None = None) -> tuple[str, str, str | None]:
    """Return (url, token, wiki) with the same config file and rules as wiki_remote.py.

    An explicit profile wins over LLM_WIKI_URL and LLM_WIKI_TOKEN and must exist.
    LLM_WIKI_CONFIG selects another config file. Raises DecisionError("unavailable").
    """
    if str(SCRIPT_DIR) not in sys.path:
        sys.path.insert(0, str(SCRIPT_DIR))
    import wiki_remote

    args = argparse.Namespace(profile=profile, url_override=url_override, wiki=None)
    try:
        url, token, wiki, _ = wiki_remote.resolve_connection(args)
    except wiki_remote.CliError as exc:
        raise DecisionError("unavailable", str(exc)) from None
    return url, token, wiki


def client_from_config(
    backend: str | None = None,
    url: str | None = None,
    model: str | None = None,
    timeout: float | None = None,
    profile: str | None = None,
) -> DecisionClient | None:
    """Build a client from flags, then environment. Returns None when decisions are off.
    Raises DecisionError("unavailable") when the selected backend lacks a key or URL."""
    name = (backend or os.environ.get("LLM_WIKI_DECISIONS") or "off").strip().lower()
    if name in ("", "off", "none", "0", "false"):
        return None
    if name not in BACKENDS:
        raise DecisionError("bad_request", f"unknown backend {name!r}; use one of {', '.join(BACKEND_CHOICES)}")
    spec = BACKENDS[name]
    url = url or os.environ.get("LLM_WIKI_DECISIONS_URL") or spec["url"]
    model = model or os.environ.get("LLM_WIKI_DECISIONS_MODEL") or spec["model"]
    key = os.environ.get(spec["key"]) if spec["key"] else None
    if spec["key"] and not key:
        raise DecisionError("unavailable", f"set {spec['key']} to use the {name} backend")
    wiki = None
    if name == "hosted":
        # A decisions URL (flag or LLM_WIKI_DECISIONS_URL) replaces the profile URL, like --url in wiki_remote.py.
        url, key, wiki = read_remote_profile(profile, url)
    return DecisionClient(name, url, model, key, timeout or DEFAULT_TIMEOUT, wiki=wiki)


# ---------------------------------------------------------------------------
# Answer reading
# ---------------------------------------------------------------------------

def read_choice(answers: dict, name: str, keys: list[str]) -> dict:
    """Validate a choice answer against the sent keys. Probabilities are renormalized
    over `keys`. Raises DecisionError("invalid_answer")."""
    answer = answers.get(name) if isinstance(answers, dict) else None
    if not isinstance(answer, dict):
        raise DecisionError("invalid_answer", f"no answer for {name}")
    choice = answer.get("choice")
    if not isinstance(choice, str) or choice not in keys:
        raise DecisionError("invalid_answer", f"answer for {name} has an unknown choice {choice!r}")

    def number(value) -> float | None:
        if not isinstance(value, (int, float)) or isinstance(value, bool):
            return None
        return float(value)

    probs = {}
    raw_probs = answer.get("probabilities")
    if isinstance(raw_probs, dict):
        for k in keys:
            if raw_probs.get(k) is None:
                continue
            p = number(raw_probs[k])
            if p is None or not math.isfinite(p) or p < 0:
                raise DecisionError("invalid_answer", f"answer for {name} has a bad probability for {k}: {raw_probs[k]!r}")
            probs[k] = p
        if probs:
            total = sum(probs.values())
            if not math.isfinite(total) or total <= 0:
                raise DecisionError("invalid_answer", f"answer for {name} has probabilities that sum to {total}")
            probs = {k: v / total for k, v in probs.items()}
    confidence = answer.get("confidence")
    if confidence is not None:
        confidence = number(confidence)
        if confidence is None or not math.isfinite(confidence) or not 0.0 <= confidence <= 1.0:
            raise DecisionError("invalid_answer", f"answer for {name} has a confidence outside 0 to 1: {answer.get('confidence')!r}")
    if not probs:
        if confidence is None:
            raise DecisionError("invalid_answer", f"answer for {name} has no probabilities")
        probs[choice] = confidence
        if len(keys) == 2:
            probs[next(k for k in keys if k != choice)] = 1 - confidence
    if confidence is None:
        confidence = probs.get(choice, 0.0)
    return {"choice": choice, "confidence": confidence, "probabilities": probs}


def choice_question(check: str) -> dict:
    t = QUESTIONS[check]
    return {t["question"]: {"type": t["type"], "instructions": t["instructions"], "criteria": t["criteria"]}}


def positive_probability(answers: dict, check: str) -> float:
    t = QUESTIONS[check]
    ans = read_choice(answers, t["question"], list(t["criteria"]))
    return ans["probabilities"].get(t["positive"], 0.0)


def ask_many(client: DecisionClient, requests: list[dict]) -> list[dict]:
    """Run requests in parallel threads. Raises the first DecisionError."""
    if not requests:
        return []
    with ThreadPoolExecutor(max_workers=min(MAX_WORKERS, len(requests))) as pool:
        return list(pool.map(client.ask, requests))


def block(client: DecisionClient | None, threshold: float, started: float, **extra) -> dict:
    return {
        "backend": client.backend if client else "off",
        "model": client.model if client else None,
        "threshold": threshold,
        "latency_ms": int((time.monotonic() - started) * 1000),
        **extra,
    }


# ---------------------------------------------------------------------------
# D1 relevance gate
# ---------------------------------------------------------------------------

def relevance_request(query: str, title: str, heading: str, text: str) -> dict:
    t = QUESTIONS["relevance"]
    return {
        "state": {"query": query, "title": title, "heading": heading, "text": text[: t["max_text_chars"]]},
        "questions": choice_question("relevance"),
    }


def score_relevance(client: DecisionClient, query: str, rows: list[dict], texts: list[str]) -> list[float]:
    """P(relevant) for each row. `rows` carry `title` and `heading_path`. Raises DecisionError."""
    requests = [
        relevance_request(query, row.get("title") or "", " > ".join(row.get("heading_path") or []), text)
        for row, text in zip(rows, texts)
    ]
    return [positive_probability(r["answers"], "relevance") for r in ask_many(client, requests)]


def gate_pool(
    client: DecisionClient | None,
    query: str,
    rows: list[dict],
    texts: list[str],
    threshold: float,
    top: int,
) -> tuple[list[int], dict[int, dict], dict]:
    """Return (kept pool indices in order, capped at `top`; per-index row decision; decision block).
    Keeps rows with P(relevant) >= threshold. Fails open: the first `top` indices, failed_open true."""
    started = time.monotonic()
    try:
        if client is None:
            raise DecisionError("unavailable", "no backend configured (set LLM_WIKI_DECISIONS)")
        scores = score_relevance(client, query, rows, texts)
    except DecisionError as exc:
        note(f"relevance gate failed open ({exc.code}: {exc})")
        kept = list(range(min(top, len(rows))))
        return kept, {}, block(client, threshold, started, pool=len(rows), kept=len(kept), dropped=0,
                               failed_open=True, error=exc.code)
    passed = [i for i, score in enumerate(scores) if score >= threshold]
    row_decisions = {
        i: {"relevant": round(scores[i], 4), "backend": client.backend, "model": client.model} for i in passed
    }
    return passed[:top], row_decisions, block(client, threshold, started, pool=len(rows), kept=len(passed),
                                              dropped=len(rows) - len(passed), failed_open=False)


# ---------------------------------------------------------------------------
# D2 evidence check
# ---------------------------------------------------------------------------

FENCE_RE = re.compile(r"^\s*(```|~~~)")
BLOCK_PREFIX_RE = re.compile(r"^(#{1,6}\s+|>\s?|[-*+]\s+|\d+[.)]\s+)")
SENTENCE_END_RE = re.compile(r"(?<=[.!?])\s+")


def normalize_for_match(text: str) -> str:
    return re.sub(r"\s+", " ", text.lower()).strip()


def evidence_in_text(evidence: str, body: str) -> bool:
    """True when `evidence` occurs in `body`, ignoring case and whitespace."""
    needle = normalize_for_match(evidence or "")
    return bool(needle) and needle in normalize_for_match(body)


def split_sentences(text: str, max_sentences: int = 60, max_chars: int = 300) -> list[str]:
    """Split Markdown into sentences. Mirrors splitSentences in server/src/decisions."""
    blocks: list[str] = []
    para: list[str] = []
    in_fence = False

    def flush() -> None:
        if para:
            blocks.append(" ".join(para))
        para.clear()

    for raw_line in text.split("\n"):
        if FENCE_RE.match(raw_line):
            flush()
            in_fence = not in_fence
            continue
        if in_fence:
            continue
        line = raw_line.strip()
        if not line or line == "---":
            flush()
            continue
        if BLOCK_PREFIX_RE.match(line):
            flush()
            blocks.append(BLOCK_PREFIX_RE.sub("", line, count=1))
            continue
        para.append(line)
    flush()
    out: list[str] = []
    for blk in blocks:
        for piece in SENTENCE_END_RE.split(blk):
            s = re.sub(r"\s+", " ", piece).strip()
            if not s:
                continue
            out.append(s[:max_chars])
            if len(out) >= max_sentences:
                return out
    return out


def evidence_request(claim: str, evidence: str, sentences: list[str]) -> dict:
    t = QUESTIONS["evidence"]
    criteria = {f"{t['sentence_key_prefix']}{i + 1}": s for i, s in enumerate(sentences)}
    criteria[t["none_key"]] = t["none_criterion"]
    return {
        "state": {"claim": claim, "evidence": evidence},
        "questions": {t["question"]: {"type": t["type"], "instructions": t["instructions"], "criteria": criteria}},
    }


def check_evidence(client: DecisionClient, claim: str, evidence: str, body: str, threshold: float) -> dict:
    """D2 verdict for one edge. Raises DecisionError on backend failure."""
    t = QUESTIONS["evidence"]
    if evidence_in_text(evidence, body):
        return {"supported": True, "choice": "quote", "confidence": 1.0, "sentence": None}
    sentences = split_sentences(body, t["max_sentences"], t["max_sentence_chars"])
    if not sentences:
        return {"supported": False, "choice": t["none_key"], "confidence": 1.0, "sentence": None}
    res = client.ask(evidence_request(claim, evidence, sentences))
    keys = [f"{t['sentence_key_prefix']}{i + 1}" for i in range(len(sentences))] + [t["none_key"]]
    ans = read_choice(res["answers"], t["question"], keys)
    idx = -1 if ans["choice"] == t["none_key"] else int(ans["choice"][len(t["sentence_key_prefix"]):]) - 1
    return {
        "supported": ans["choice"] != t["none_key"] and ans["confidence"] >= threshold,
        "choice": ans["choice"],
        "confidence": round(ans["confidence"], 4),
        "sentence": sentences[idx] if idx >= 0 else None,
    }


# ---------------------------------------------------------------------------
# D3 relation, D4 changes, D5 screen
# ---------------------------------------------------------------------------

def relation_request(entity: str, a: dict, b: dict) -> dict:
    limit = QUESTIONS["relation"]["max_text_chars"]
    return {
        "state": {
            "entity": entity,
            "passage_a": {"page": a["title"], "text": a["text"][:limit]},
            "passage_b": {"page": b["title"], "text": b["text"][:limit]},
        },
        "questions": choice_question("relation"),
    }


def changes_request(source_title: str, source_text: str, page_title: str, page_text: str) -> dict:
    t = QUESTIONS["changes"]
    return {
        "state": {
            "source": {"title": source_title, "text": source_text[: t["max_source_chars"]]},
            "page": {"title": page_title, "text": page_text[: t["max_text_chars"]]},
        },
        "questions": choice_question("changes"),
    }


def screen_request(path: str, text: str) -> dict:
    return {
        "state": {"path": path, "text": text[: QUESTIONS["orders_agent"]["max_text_chars"]]},
        "questions": choice_question("orders_agent"),
    }


def screen_text(client: DecisionClient | None, path: str, text: str, threshold: float) -> dict:
    started = time.monotonic()
    try:
        if client is None:
            raise DecisionError("unavailable", "no backend configured (set LLM_WIKI_DECISIONS)")
        res = client.ask(screen_request(path, text))
        score = positive_probability(res["answers"], "orders_agent")
    except DecisionError as exc:
        note(f"screen failed open ({exc.code}: {exc})")
        return {"path": path, "score": None, "flagged": False,
                **block(client, threshold, started, failed_open=True, error=exc.code)}
    return {"path": path, "score": round(score, 4), "flagged": score >= threshold,
            **block(client, threshold, started, failed_open=False)}


# ---------------------------------------------------------------------------
# Wiki helpers (reuse the search parser)
# ---------------------------------------------------------------------------

def load_search_module():
    sys.path.insert(0, str(SCRIPT_DIR))
    import wiki_search  # noqa: E402

    return wiki_search


def strip_frontmatter(text: str) -> tuple[str, str]:
    """Return (title, body) of a Markdown file."""
    ws = load_search_module()
    meta, body = ws.parse_frontmatter(text)
    title = meta.get("title") if isinstance(meta, dict) else None
    if not title:
        m = re.search(r"^#\s+(.+)$", body, re.MULTILINE)
        title = m.group(1).strip() if m else ""
    return title or "", body


def search_candidates(source_query: str, wiki: Path, top: int, no_embed: bool) -> list[dict]:
    script = SCRIPT_DIR / "wiki_search.py"
    argv = [source_query, "--wiki", str(wiki), "--json", "--top", str(top), "--per-page", "1"]
    if no_embed:
        argv.append("--no-embed")
    uv = shutil.which("uv")
    cmd = [uv, "run", "--quiet", "--script", str(script), *argv] if uv and not no_embed else [sys.executable, str(script), *argv]
    proc = subprocess.run(cmd, capture_output=True, text=True)
    if proc.returncode != 0 or not proc.stdout.strip():
        return []
    return json.loads(proc.stdout).get("results", [])


def cmd_shortlist(args, client: DecisionClient | None) -> int:
    source_path = Path(args.source)
    text = source_path.read_text(encoding="utf-8")
    title, body = strip_frontmatter(text)
    title = title or source_path.stem
    t = QUESTIONS["changes"]
    query = f"{title} {body[: t['max_source_chars']]}"
    rows = search_candidates(query, args.wiki, args.top, args.no_embed)
    threshold = args.threshold if args.threshold is not None else t["default_threshold"]
    started = time.monotonic()
    candidates = []
    for rank, row in enumerate(rows, 1):
        page_file = args.wiki / row["rel_path"]
        try:
            _, page_body = strip_frontmatter(page_file.read_text(encoding="utf-8"))
        except OSError:
            page_body = row.get("snippet", "")
        candidates.append({"rel_path": row["rel_path"], "title": row["title"], "search_rank": rank, "text": page_body})
    try:
        if client is None:
            raise DecisionError("unavailable", "no backend configured (set LLM_WIKI_DECISIONS)")
        results = ask_many(client, [changes_request(title, body, c["title"], c["text"]) for c in candidates])
        scores = [positive_probability(r["answers"], "changes") for r in results]
    except DecisionError as exc:
        note(f"shortlist failed open ({exc.code}: {exc}); pages keep search order")
        out = [{k: v for k, v in c.items() if k != "text"} | {"changes": None} for c in candidates]
        print(json.dumps({"source": str(source_path), "candidates": out,
                          "decision": block(client, threshold, started, failed_open=True, error=exc.code)}, indent=2))
        return 0
    ranked = sorted(
        ({k: v for k, v in c.items() if k != "text"} | {"changes": round(s, 4), "shortlisted": s >= threshold}
         for c, s in zip(candidates, scores)),
        key=lambda c: (-c["changes"], c["search_rank"]),
    )
    print(json.dumps({"source": str(source_path), "candidates": ranked,
                      "decision": block(client, threshold, started, failed_open=False,
                                        shortlisted=sum(1 for c in ranked if c["shortlisted"]))}, indent=2))
    return 0


def contradiction_pairs(wiki: Path, max_pairs: int) -> list[tuple[str, dict, dict]]:
    """Pairs of sections from different pages that link the same entity page."""
    ws = load_search_module()
    pages = ws.collect_pages(wiki)
    by_path, by_slug = ws.build_link_index(pages)
    entity_titles = {p["slug"]: p["meta"].get("title") or p["slug"] for p in pages if p["meta"].get("type") == "entity"}
    mentions: dict[str, list[dict]] = {}
    for page in pages:
        title = page["meta"].get("title") or page["slug"]
        for section in page["sections"]:
            linked = set()
            for m in ws.WIKILINK_RE.finditer(section["text"]):
                slug = ws.resolve_link(m.group(1), by_path, by_slug)
                if slug in entity_titles and slug != page["slug"]:
                    linked.add(slug)
            text = section["text"].strip()
            if not text:
                continue
            for slug in sorted(linked):
                mentions.setdefault(slug, []).append({
                    "rel_path": page["rel_path"],
                    "title": title,
                    "heading": " > ".join(section["heading_path"]),
                    "text": text,
                })
    pairs = []
    for slug in sorted(mentions):
        items = mentions[slug]
        for i in range(len(items)):
            for j in range(i + 1, len(items)):
                if items[i]["rel_path"] == items[j]["rel_path"]:
                    continue
                pairs.append((entity_titles[slug], items[i], items[j]))
                if len(pairs) >= max_pairs:
                    return pairs
    return pairs


def cmd_contradictions(args, client: DecisionClient | None) -> int:
    t = QUESTIONS["relation"]
    threshold = args.threshold if args.threshold is not None else t["default_threshold"]
    pairs = contradiction_pairs(args.wiki, args.max_pairs)
    started = time.monotonic()
    try:
        if client is None:
            raise DecisionError("unavailable", "no backend configured (set LLM_WIKI_DECISIONS)")
        results = ask_many(client, [relation_request(e, a, b) for e, a, b in pairs])
        answers = [read_choice(r["answers"], t["question"], list(t["criteria"])) for r in results]
    except DecisionError as exc:
        note(f"contradiction screen failed open ({exc.code}: {exc}); no pairs checked")
        print(json.dumps({"pairs": len(pairs), "flagged": [],
                          "decision": block(client, threshold, started, failed_open=True, error=exc.code)}, indent=2))
        return 0
    flagged = []
    for (entity, a, b), ans in zip(pairs, answers):
        p = ans["probabilities"].get(t["positive"], 0.0)
        if p >= threshold:
            flagged.append({
                "entity": entity,
                "a": {"rel_path": a["rel_path"], "heading": a["heading"]},
                "b": {"rel_path": b["rel_path"], "heading": b["heading"]},
                "contradicts": round(p, 4),
                "choice": ans["choice"],
            })
    flagged.sort(key=lambda f: -f["contradicts"])
    print(json.dumps({"pairs": len(pairs), "flagged": flagged,
                      "decision": block(client, threshold, started, failed_open=False)}, indent=2))
    return 0


def cmd_screen(args, client: DecisionClient | None) -> int:
    text = Path(args.file).read_text(encoding="utf-8", errors="replace")
    threshold = args.threshold if args.threshold is not None else QUESTIONS["orders_agent"]["default_threshold"]
    print(json.dumps(screen_text(client, str(args.file), text, threshold), indent=2))
    return 0


def cmd_ask(args, client: DecisionClient | None) -> int:
    raw = sys.stdin.read() if args.file in (None, "-") else Path(args.file).read_text(encoding="utf-8")
    try:
        request = json.loads(raw)
    except json.JSONDecodeError as exc:
        note(f"request is not JSON ({exc})")
        return 2
    if client is None:
        note("no backend configured (set LLM_WIKI_DECISIONS or --backend)")
        return 1
    try:
        res = client.ask(request)
    except DecisionError as exc:
        note(f"{client.backend} failed ({exc.code}: {exc})")
        return 1
    print(json.dumps({**res, "backend": client.backend}, indent=2))
    return 0


def cmd_check(args, client: DecisionClient | None) -> int:
    if client is None:
        print(json.dumps({"ok": False, "backend": "off", "error": "no backend configured"}))
        return 1
    started = time.monotonic()
    try:
        res = client.ask(relevance_request("what is a wiki", "Wiki", "Overview", "A wiki is a set of linked pages that people edit."))
        positive_probability(res["answers"], "relevance")
    except DecisionError as exc:
        print(json.dumps({"ok": False, "backend": client.backend, "model": client.model,
                          "url": client.endpoint, "error": exc.code, "message": str(exc)}))
        return 1
    print(json.dumps({"ok": True, "backend": client.backend, "model": client.model, "url": client.endpoint,
                      "latency_ms": int((time.monotonic() - started) * 1000)}))
    return 0


def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--backend", choices=BACKEND_CHOICES, help="Decision backend (default: LLM_WIKI_DECISIONS or off).")
    parser.add_argument("--model", help="Model name (default: LLM_WIKI_DECISIONS_MODEL or the backend default).")
    parser.add_argument("--url", help="Backend base URL (default: LLM_WIKI_DECISIONS_URL or the backend default).")
    parser.add_argument("--timeout", type=float, default=DEFAULT_TIMEOUT, help="Request timeout in seconds (default: 8).")
    parser.add_argument("--profile", help="wiki_remote.py profile for the hosted backend.")
    sub = parser.add_subparsers(dest="command", required=True)

    p = sub.add_parser("ask", help="Send a raw {state, questions} request.")
    p.add_argument("file", nargs="?", help="Request JSON file (default: stdin).")

    p = sub.add_parser("screen", help="Check if text gives orders to an AI system (D5).")
    p.add_argument("file")
    p.add_argument("--threshold", type=float, help="Flag at or above this probability (default: 0.7).")

    p = sub.add_parser("shortlist", help="Rank wiki pages that a new source changes (D4).")
    p.add_argument("source")
    p.add_argument("--wiki", type=Path, default=Path("wiki"))
    p.add_argument("--top", type=int, default=QUESTIONS["changes"]["pool"], help="Search candidates (default: 30).")
    p.add_argument("--threshold", type=float, help="Shortlist at or above this probability (default: 0.5).")
    p.add_argument("--no-embed", action="store_true", help="Use lexical search for candidates.")

    p = sub.add_parser("contradictions", help="Find section pairs that contradict (D3).")
    p.add_argument("--wiki", type=Path, default=Path("wiki"))
    p.add_argument("--max-pairs", type=int, default=40, help="Most pairs to check (default: 40).")
    p.add_argument("--threshold", type=float, help="Report at or above this probability (default: 0.5).")

    sub.add_parser("check", help="Test the backend connection.")
    return parser


def main(argv: list[str] | None = None) -> int:
    args = build_parser().parse_args(argv)
    try:
        client = client_from_config(args.backend, args.url, args.model, args.timeout, args.profile)
    except DecisionError as exc:
        note(str(exc))
        if args.command in ("ask", "check"):
            return 1
        client = None
    if args.command in ("shortlist", "contradictions") and not args.wiki.exists():
        print(f"Wiki directory not found: {args.wiki}", file=sys.stderr)
        return 1
    handlers = {
        "ask": cmd_ask,
        "screen": cmd_screen,
        "shortlist": cmd_shortlist,
        "contradictions": cmd_contradictions,
        "check": cmd_check,
    }
    return handlers[args.command](args, client)


if __name__ == "__main__":
    sys.exit(main())
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall