Skip to content

GodModeAI2025/SkillSafeWerkstatt / maintain-llm-wiki

maintain-llm-wiki

Unverified

Create, maintain, inventory, migrate, recover, quality-review, optionally clean, release, or export an evidence-linked SkillSafeWerkstatt Markdown wiki. Use whenever the user wants to initialize or extend this wiki, ingest documents as Markdown, curate claims, clusters or concepts, clean metadata, restore files, record who reviewed which pages, resolve a OneDrive or SharePoint conflict copy, publish a release, or export a frozen read-only knowledge skill or an Open Knowledge Format bundle in Claude Code, Claude Cowork, or Codex.

FileKindSize
SKILL.mdskill md57.2 KB
assets/graph-template.htmlother27.8 KB
assets/knowledge-skill/SKILL.template.mdmarkdown35.6 KB
assets/knowledge-skill/assess_quality.pyscript1.5 KB
assets/knowledge-skill/identity_status.pyscript5.9 KB
assets/knowledge-skill/search_knowledge.pyscript4.5 KB
assets/knowledge-skill/verify_knowledge.pyscript4.8 KB
assets/page-template.htmlother7.4 KB
evals/evals.jsonconfig32.6 KB
references/adapted-vault-standards.mdmarkdown3.3 KB
references/frontmatter-operations.mdmarkdown9.8 KB
references/wiki-contract.mdmarkdown73.0 KB
scripts/adopt_directory.pyscript13.2 KB
scripts/apply_identity.pyscript3.1 KB
scripts/bm25.pyscript8.2 KB
scripts/build_graph.pyscript27.0 KB
scripts/claim_id.pyscript1.9 KB
scripts/describe_actions.pyscript13.2 KB
scripts/design_contract.pyscript217 B
scripts/export_okf_bundle.pyscript17.9 KB
scripts/export_wiki_skill.pyscript23.7 KB
scripts/findability_questions.pyscript17.6 KB
scripts/freshness.pyscript3.6 KB
scripts/frontmatter_actions.pyscript26.1 KB
scripts/frontmatter_contract.pyscript10.0 KB
scripts/identity_contract.pyscript15.2 KB
scripts/init_wiki.pyscript18.2 KB
scripts/initialize_wiki.pyscript12.2 KB
scripts/inventory_wiki.pyscript6.7 KB
scripts/lint_wiki.pyscript56.0 KB
scripts/move_wiki_page.pyscript6.7 KB
scripts/navigation.pyscript11.0 KB
scripts/okf_contract.pyscript14.3 KB
scripts/page_batch.pyscript11.4 KB
scripts/plan_identity.pyscript2.2 KB
scripts/plan_language_migration.pyscript4.9 KB
scripts/portable_io.pyscript3.3 KB
scripts/quality_core.pyscript15.9 KB
scripts/record_quality_review.pyscript3.4 KB
scripts/register_source.pyscript12.2 KB
scripts/release_wiki.pyscript15.9 KB
scripts/report_okf.pyscript3.8 KB
scripts/resolve_conflict_copy.pyscript11.0 KB
scripts/restore_wiki.pyscript16.1 KB
scripts/run_locked.pyscript2.2 KB
scripts/search_core.pyscript221.1 KB
scripts/secret_screen.pyscript4.1 KB
scripts/snapshot_wiki.pyscript6.7 KB
scripts/source_coverage.pyscript22.8 KB
scripts/sync_artifacts.pyscript14.2 KB
scripts/trust_contract.pyscript5.2 KB
scripts/validate_extraction.pyscript5.0 KB
scripts/verify_pages.pyscript9.1 KB
scripts/verify_release.pyscript12.1 KB
scripts/wiki_filters.pyscript6.6 KB
scripts/wiki_lock.pyscript59.2 KB
scripts/wordforms.pyscript56.0 KB

scripts/bm25.py

View on GitHub
#!/usr/bin/env python3
"""Field-weighted BM25 ranking over a wiki, using the standard library only.

The previous ranking added hand-picked constants per match. That is neither
length-normalised nor inverse-document-frequency weighted, so long pages ranked
higher simply for being long, and a term appearing on every page counted as much
as one appearing on two.

This implements BM25F: term frequency is accumulated across weighted fields,
each normalised by that field's own average length, and every query term is
weighted by how rare it is in this corpus. It needs corpus statistics, so
callers collect all candidates first and score afterwards.

The wiki's own signals - controlled concepts, an exact phrase, page status - stay
as bonuses on top. They express curation the corpus statistics cannot see.

Every query term is a weighted group of document tokens (see `TermGroup`
and wordforms.py): the literal token at weight 1.0 and its word forms and
heuristic matches at lower weights. A group of the literal token alone scores
exactly as the plain BM25F of that token.
"""

from __future__ import annotations

import math
from typing import Iterable, Mapping, Optional, Sequence


#: Term-frequency saturation. The standard default.
K1 = 1.2

#: Length normalisation per field. The standard default.
B = 0.75

#: Field weights, carried over from the previous ranking so relative emphasis
#: between title, aliases, headings, description and body is unchanged.
FIELD_WEIGHTS = {
    "title": 10.0,
    "aliases": 7.0,
    "headings": 5.0,
    "description": 4.0,
    "body": 1.0,
    # The quantity tokens of a document (search_core.spelled_quantities); a
    # field of its own, so the length of no other field changes.
    "quantities": 1.0,
}

#: Modifiers for signals the corpus statistics cannot see.
#:
#: These are multiplicative on purpose. BM25 scores scale with corpus size and
#: term rarity, so a fixed addition means something different in a wiki of ten
#: pages than in one of a thousand - and a fixed subtraction could drive a real
#: textual match below zero and drop it from the results entirely. A factor
#: reorders comparable documents without ever suppressing or inventing a match.
PHRASE_FACTOR = 1.5
CONCEPT_FACTOR_PER_MATCH = 0.5
STATUS_FACTOR = {"active": 1.15, "draft": 0.95, "superseded": 0.75, "withdrawn": 0.75}


class Corpus:
    """Corpus statistics needed to score any document in it."""

    def __init__(self, documents: Iterable[dict[str, list[str]]]) -> None:
        self.count = 0
        self.field_length_total: dict[str, int] = {field: 0 for field in FIELD_WEIGHTS}
        for fields in documents:
            self.count += 1
            for field in FIELD_WEIGHTS:
                self.field_length_total[field] += len(fields.get(field) or [])

    def average_length(self, field: str) -> float:
        if not self.count:
            return 0.0
        return self.field_length_total.get(field, 0) / self.count

    def inverse_document_frequency_of(self, frequency: int) -> float:
        """Standard BM25 IDF for a document frequency counted by the caller, such as a term group's.

        It stays positive for every observed term.
        """
        return math.log(1.0 + (self.count - frequency + 0.5) / (frequency + 0.5))


class TermGroup:
    """One query term as weighted document tokens.

    `members` maps every token that counts for the term to its weight: 1.0 for
    the literal token, less for a word form or a heuristic member (see
    wordforms.py). `document_frequency` is counted by the caller, because only
    the caller knows which members may make a term look common: the search
    core counts word forms only, and heuristic members only when no document
    holds a word form.
    """

    __slots__ = ("label", "members", "document_frequency")

    def __init__(self, label: str, members: Mapping[str, float], document_frequency: int) -> None:
        self.label = label
        self.members = dict(members)
        self.document_frequency = document_frequency


def group_base_score(
    corpus: Corpus,
    fields: dict[str, list[str]],
    groups: Sequence[TermGroup],
    counted: Optional[dict[str, list[str]]] = None,
    words: Optional[set[str]] = None,
) -> float:
    """BM25F where every query term is a TermGroup.

    The occurrences of a group's members are accumulated across the weighted
    fields exactly like those of one literal token and saturated; the rule
    weight is applied after saturation. Members of different weights are
    layered: the layer up to weight w is filled by the occurrences of every
    member weighing at least w. So a document holding only the literal token
    scores exactly as the plain BM25F of that token, one holding only a
    member of weight w scores w times that (a compound part in a title keeps
    its own weight of an exact title hit, not almost all of it), and every
    further occurrence of any member only adds. A document holding no member
    of any group scores 0.

    `counted`, when given, receives per group label the tokens that counted.
    `words`, when given, must be the set of every token of `fields`; a caller
    that keeps it per document saves building it again.
    """
    if not corpus.count:
        # Without a corpus there is no term rarity and no average length, so a
        # score would be a number without meaning. Rank nothing instead.
        return 0.0
    field_tokens = {field: fields.get(field) or [] for field in FIELD_WEIGHTS}
    if words is None:
        words = set().union(*field_tokens.values())
    normalisers: dict[str, float] = {}
    for field, values in field_tokens.items():
        # Normalise by this field's own average length, so a long body does
        # not outrank a precise title.
        average = corpus.average_length(field)
        normalisers[field] = 1.0 - B + B * (len(values) / average if average else 1.0)
    score = 0.0
    for group in groups:
        # Only the members the document holds are counted, in a fixed order.
        present = sorted(words.intersection(group.members))
        if not present:
            continue
        by_weight: dict[float, float] = {}
        for token in present:
            accumulated = 0.0
            for field, weight in FIELD_WEIGHTS.items():
                frequency = field_tokens[field].count(token)
                if frequency:
                    accumulated += weight * frequency / normalisers[field]
            level = group.members[token]
            by_weight[level] = by_weight.get(level, 0.0) + accumulated
        idf = corpus.inverse_document_frequency_of(group.document_frequency)
        levels = sorted(by_weight, reverse=True)
        layered = 0.0
        for index, level in enumerate(levels):
            layered += by_weight[level]
            below = levels[index + 1] if index + 1 < len(levels) else 0.0
            score += (level - below) * idf * layered / (K1 + layered)
        if counted is not None:
            counted[group.label] = present
    return score


def score(
    corpus: Corpus,
    fields: dict[str, list[str]],
    groups: Sequence[TermGroup],
    *,
    phrase_match: bool = False,
    concept_overlap: int = 0,
    status: str = "",
    explain: Optional[dict[str, float]] = None,
    counted: Optional[dict[str, list[str]]] = None,
    words: Optional[set[str]] = None,
) -> float:
    """BM25F over the query's term groups, scaled by the wiki's curated signals, never below zero.

    `counted` and `words` are passed to `group_base_score`.
    """
    base = group_base_score(corpus, fields, groups, counted, words)
    if base <= 0:
        # Nothing in this document's text answers the query. Curation reorders
        # matches; it never turns a non-match into one.
        if explain is not None:
            explain.update({"bm25": 0.0, "phrase": 1.0, "concepts": 1.0, "status": 1.0})
        return 0.0
    phrase = PHRASE_FACTOR if phrase_match else 1.0
    concepts = 1.0 + CONCEPT_FACTOR_PER_MATCH * max(concept_overlap, 0)
    adjustment = STATUS_FACTOR.get(status, 1.0)
    if explain is not None:
        explain.update(
            {
                "bm25": round(base, 4),
                "phrase": phrase,
                "concepts": concepts,
                "status": adjustment,
            }
        )
    return max(base * phrase * concepts * adjustment, 0.0)
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall