Skip to content

praneybehl/llm-wiki-plugin / skills/llm-wiki

llm-wiki

Verified

Build and maintain an LLM-curated knowledge base from papers, articles, transcripts, notes and project findings. Ingest sources into linked Markdown, answer questions with citations, lint or upgrade a wiki, and preserve useful synthesis across sessions and agents. Also capture verified task experience, consolidate success/failure patterns, and propose, evaluate, apply or roll back evidence-linked skill improvements. Trigger on "add this to my wiki", "what does the wiki say", "learn from this task", "improve this procedure", /wiki:learn or /wiki:evolve. Works on a local folder or a hosted wiki URL (MCP). Use index-first navigation, atomic pages and hybrid search.

FileKindSize
SKILL.mdskill md24.6 KB
assets/SCHEMA.md.templateother7.4 KB
assets/decision-questions.jsonconfig3.1 KB
assets/evolution_README.md.templateother2.3 KB
assets/experience.json.templateother660 B
assets/graph_README.md.templateother2.0 KB
assets/graph_gitignore.templateother27 B
assets/index.md.templateother775 B
assets/log.md.templateother499 B
assets/ontology.yaml.templateother3.2 KB
assets/page.md.templateother805 B
assets/pattern.md.templateother839 B
assets/wiki-cache_gitignore.templateother14 B
references/agent-memory-integration.mdmarkdown7.5 KB
references/architecture.mdmarkdown9.6 KB
references/decision-models.mdmarkdown6.5 KB
references/evolution-workflow.mdmarkdown19.7 KB
references/graph-workflow.mdmarkdown7.1 KB
references/hosted-mode.mdmarkdown11.4 KB
references/ingest-workflow.mdmarkdown11.1 KB
references/lint-workflow.mdmarkdown8.3 KB
references/page-conventions.mdmarkdown5.8 KB
references/query-workflow.mdmarkdown8.6 KB
references/retrieval-setup.mdmarkdown5.6 KB
references/scaling-playbook.mdmarkdown6.8 KB
scripts/init_wiki.pyscript9.5 KB
scripts/setup_wiki.pyexecutablescript2.5 KB
scripts/test_link_resolution.pyscript6.2 KB
scripts/wiki_decide.pyexecutablescript31.6 KB
scripts/wiki_evolve.pyscript29.1 KB
scripts/wiki_evolve_agent.pyscript15.1 KB
scripts/wiki_evolve_api.pyscript7.0 KB
scripts/wiki_evolve_claude.pyscript2.4 KB
scripts/wiki_evolve_loop.pyscript43.4 KB
scripts/wiki_evolve_stream.pyscript13.0 KB
scripts/wiki_graph_extract.pyscript21.0 KB
scripts/wiki_graph_lint.pyscript22.0 KB
scripts/wiki_graph_query.pyscript9.3 KB
scripts/wiki_lint.pyscript14.5 KB
scripts/wiki_remote.pyexecutablescript76.6 KB
scripts/wiki_search.pyscript35.9 KB
scripts/wiki_stats.pyscript5.9 KB

references/ingest-workflow.md

View on GitHub
# Ingest Workflow

When a new source arrives, this is the procedure. The order matters: each step builds context for the next.

Each step shows the local form first. For a hosted wiki, use the **Hosted** form (see `hosted-mode.md`). In hosted mode, collect the page writes of Steps 5 to 9 into one `wiki_commit` when possible. One commit applies all changes, or none.

## Step 0: Check the schema

Before anything else, read `wiki/SCHEMA.md`. The user may have customized the page-type structure, the tag taxonomy, the naming conventions, or the ingest workflow itself. Schema overrides everything documented here.

**Hosted:** call `wiki_overview`. It returns `SCHEMA.md` and `index.md`.

## Step 1: Place the raw source

If the source isn't already in `raw/`, place it there. Use a slugified filename: lowercase, hyphens for spaces, no special characters, with the original extension. For web articles, save as `.md` (Obsidian Web Clipper output is ideal). For PDFs, keep the `.pdf`. For transcripts, save as `.md` or `.txt`.

The slug you pick here will become the slug of the source-summary page in `wiki/sources/`, so make it descriptive and stable.

**Hosted:** upload the file with `python3 scripts/wiki_remote.py raw put <file> --as <slug>.<ext>`. Without a shell, save text sources with `wiki_raw` `put_text`. If the server has no raw storage, keep the file on this machine and record its path and SHA-256 in the source page frontmatter.

## Step 2: Read the source

For short sources (under ~5,000 words / ~25,000 tokens), read the whole thing in one pass.

For long sources (papers over ~30 pages, book chapters, multi-hour transcripts), **chunk-read**: read the table of contents or section headers first to build a mental map, then read sections sequentially, summarizing each section in working memory before moving to the next. Do not load the entire raw source into context at once if it would consume more than ~25% of your context window: that leaves no room for the rest of the operation.

**Hosted:** read a raw file in parts with `wiki_raw` `read`, `offset` and `length`.

For PDFs specifically, prefer the `pdf-reading` skill if available (it handles the chunking automatically). Otherwise extract text first with `pdftotext` or `pdfminer` and then chunk-read the extracted text.

For images embedded in the source: read the surrounding text first, then view only the images that the text suggests are load-bearing (a chart referenced in an argument, a diagram of a system, a figure the source explicitly walks through). Don't blindly load every image: many are decorative.

## Step 3: Discuss the takeaways with the user

Briefly: three or four sentences. Surface what struck you as important, what was surprising, what connects to existing wiki content, and what's worth flagging. The user's reaction shapes the next steps. They might say "skip the methodology section, only the results matter for me" or "we already have a page on this: just update it" or "this contradicts the page on X, flag that prominently".

If the user is operating in a hands-off mode (batch ingest, "just process these 20 papers"), skip the discussion but be more conservative on the wiki edits: make smaller, safer updates and surface anything ambiguous in the log.

## Step 4: Identify what's touched

Before writing anything, do a survey pass against the wiki to determine the impact:

1. Read `wiki/index.md` (or the relevant shard under `wiki/indexes/`) to identify existing pages this source touches. Look for entity names, concept names, and topical overlap.
2. For each potentially-touched page, read the page to confirm. (Read, don't grep: the index summaries can mislead.)
3. List, in working memory: existing pages to update, new pages to create, contradictions to flag.

If the user enabled the optional decision layer, `python scripts/wiki_decide.py shortlist raw/<source-file> --wiki wiki/` ranks the pages to read first. Use it as a hint only (`decision-models.md`).

**Hosted:** use `index.md` from `wiki_overview`, then `wiki_read` for the candidate pages. Keep the `rev` of each page that you will change.

This survey is what prevents duplication. Without it, you'll create a new page on a topic that already has one under a slightly different name.

## Step 5: Write the source-summary page

Create `wiki/sources/<source-slug>.md`. Use the page template (`assets/page.md.template`) as a starting point. Frontmatter should include at minimum:

```yaml
---
type: source
title: "Original title of the source"
authors: ["Author Name"]
url: "https://..." # if applicable
raw: "raw/<source-slug>.<ext>"
ingested: 2026-04-15
tags: [tag1, tag2]
entities: [entity-page-1, entity-page-2]
concepts: [concept-page-1, concept-page-2]
---
```

Frontmatter list values are bare slugs: the `[[wikilink]]` syntax goes in the body, not in YAML.

The body should be the LLM's summary of the source: the key claims, the methodology if relevant, the conclusions, the open questions. Do not paraphrase the entire source; that defeats the purpose. Aim for a summary that captures what a future query would need to know without re-reading the raw file.

Keep the page atomic: under the 400-line soft cap. If the source is so dense that a single summary page can't capture it, split: one page per major section, each linking to the others, with a parent page that gives the overview.

End the body with a "Where this fits" section listing `[[wikilinks]]` to the entity and concept pages this source touches. This is the bidirectional link from source → existing structure.

## Step 6: Update touched pages

For each existing page identified in Step 4, surgically edit it to incorporate what the new source adds. Use `str_replace`, not full rewrites. The goal is to add a sentence or paragraph in the relevant section, with a `[[wikilinks]]` citation to the new source-summary page.

**Hosted:** use a `wiki_commit` `edit` operation with the page `base_rev`. Each `old` text must occur once in the page. After a conflict, read the page again and retry (`hosted-mode.md`).

Common update patterns:

- A new source corroborates an existing claim → add a citation: `..., as established in [[paper-X]] and now corroborated by [[paper-Y]]`.
- A new source contradicts an existing claim → flag prominently: add a "## Contradictions" section if it doesn't exist, and document the contradiction with both sources cited. Do not silently overwrite the older claim.
- A new source adds a new dimension to an existing topic → add a new sub-section, don't dilute the existing prose.
- A new source mentions an entity or concept the page already discusses → update the `sources:` frontmatter and add the cross-reference where relevant in the body.

If updating a page would push it over the 800-line hard cap, that's the signal to split the page (extract the new dimension into its own page, link from the parent). Do that as part of the ingest, not as a deferred lint task.

## Step 7: Create new pages for new entities and concepts

For each new entity or concept the source introduces that doesn't have a page yet, decide whether it warrants its own page. Heuristic: if the source mentions it in passing and no other source is likely to expand on it, just mention it inline on a related page. If the source treats it as a first-class topic or you can foresee future sources building on it, create a page.

New pages need:
- Frontmatter with `type`, `tags`, `sources: [[[<source-slug>]]]`, `created`, `updated`.
- A body that introduces the entity/concept with what this source said about it.
- Inbound links: at least one existing page should `[[wikilink]]` to the new page, otherwise it's an instant orphan. Update the existing page to add the link.

A new page that nothing links to is a bug in the ingest, not just a lint finding.

**Hosted:** use a `create` operation for each new page. It fails if the path exists. Then read the existing page and edit it.

## Step 8: Update the index

Add entries for any new pages to `wiki/index.md` (or the appropriate shard). Each entry is one line: a wikilink to the page and a one-sentence summary. Keep the summary tight: the index is engineered to be cheap to read, and a fat index defeats the index-first navigation principle.

**Hosted:** use an `append` operation on the index page, with `after_heading` for the correct section.

If `index.md` exceeds 300 lines after this update, that's the signal to shard. Do it now while the structure is fresh: see `scaling-playbook.md` for the procedure.

## Step 8b: Refresh the graph layer (only if `wiki/graph/ontology.yaml` exists)

If the wiki has the optional graph layer:

1. Add typed `graph.relationships[]` only when the source explicitly supports them (predicate, source-page slug, evidence quote, confidence, status). When uncertain, prefer a plain `[[wikilink]]` in the body: the body wikilink already produces a `mentions` edge.
2. Run `uv run --script scripts/wiki_graph_lint.py wiki/`. Triage findings with the user before extracting; do not silently rewrite typed edges.
3. Run `uv run --script scripts/wiki_graph_extract.py wiki/` to regenerate `nodes.jsonl`, `edges.jsonl`, `graph.sqlite`, `graph.graphml`.

Skip this entire step if the ingest added no `graph:` metadata and created no new pages: the compiled artifacts are unchanged. Full reference: `references/graph-workflow.md`.

**Hosted:** do not run extract. The server updates the graph in the same commit. Run `wiki_lint` and triage the graph findings.

## Step 9: Append to the log

One line in `wiki/log.md`, with the prefix `## [YYYY-MM-DD] ingest | <source-title>`. Optionally add a sub-line listing the pages touched. If the graph layer was refreshed, add a second sub-line: `   graph: +N nodes, +M typed edges`. The log is parsed by simple unix tools (`grep "^## \[" log.md | tail -10`), so the prefix matters.

**Hosted:** add the same line with an `append` operation on `log.md`. Appends never conflict.

## Step 10: Close the loop with the user

Tell the user what you did, briefly: "Ingested. Created the source page and a new entity page for X; updated the concept pages for Y and Z. Flagged a contradiction with [[paper-A]] regarding the claim about W."

If the source revealed something worth following up on (an obvious gap, a question the source raised but didn't answer, a candidate next source to ingest), say so: this is where the wiki's compounding effect comes from.

## Anti-patterns to avoid

**Loading the whole source into context at once when it's large.** This is the most common scaling failure. Chunk-read.

**Rewriting whole pages instead of surgical edits.** This burns tokens, risks losing nuance, and erodes diff quality if the wiki is in git.

**Creating pages with no inbound links.** Orphans accumulate fast and become invisible. Always link.

**Ingesting silently in batch mode without surfacing surprises.** Batch ingest is fine, but a one-line "ingested 5 sources, 2 new entity pages, 1 contradiction with [[X]]" summary is the minimum.

**Treating prior wiki pages as ground truth instead of the raw sources.** When updating an existing claim, re-read the raw source for that claim before merging the new one. Don't compound on the wiki's own paraphrase.

**Letting the page split decision drift to a future lint pass.** If a page crossed the size cap during this ingest, split it during this ingest.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall