po4yka/llm-wiki-skills / skills/llm-wiki-trust-audit
llm-wiki-trust-audit
VerifiedAudit an LLM-Wiki system for anti-slop controls such as provenance, review gates, protected human synthesis, stale claims, and unsafe agent write permissions. Use when the user asks whether their wiki can be trusted; route setup/permission safety reviews to llm-wiki-security-review and proposal risk audits to llm-wiki-critique-audit.
LLM-Wiki Trust Audit
Goal
Evaluate whether an LLM-Wiki is trustworthy enough to use for decisions and identify the highest-risk failure modes.
When to use
- The user asks whether their LLM-Wiki can be trusted before relying on it for a decision.
- Before granting an agent write access to a vault, or after granting it, to confirm the write-safety boundary still holds.
- After a large ingest, bulk edit, or lint run, to confirm provenance and review gates were not weakened.
- Route setup/permission-only reviews to
llm-wiki-security-reviewand proposal-specific risk reviews tollm-wiki-critique-auditinstead of this skill.
Inputs
- Vault/repository path.
AGENTS.md,CLAUDE.md, skills and schemas.raw/,wiki/,_meta/,_agent/reports/if present.- Git history if available.
- User's risk tolerance and domain sensitivity.
Procedure
1. Inspect the trust model
Check whether the system defines:
- raw source immutability;
- page lifecycle states;
- source backlinks;
- claim types;
- confidence semantics;
- review requirements;
- protected human sections;
- lint cadence;
- rollback/recovery path.
2. Audit provenance
Sample important pages and report:
- no provenance;
- source-level provenance only;
- claim-level provenance;
- generated pages citing generated pages;
- stale or missing source hashes;
- source links that no longer resolve.
Re-verify any page flagged stale or with an unresolved source link before treating its claims as trustworthy.
3. Audit generated content boundaries
Look for:
- AI-generated pages marked reviewed/verified without evidence;
ai_confidencedefaulted lazily;- low-confidence pages not requiring review;
- ambiguous claims in trusted pages;
- human synthesis sections overwritten or unprotected.
4. Audit structural health
Coordinate with wiki-lint if available. Check:
- broken links;
- orphan pages;
- duplicate concepts;
- taxonomy drift;
- stale pages;
- contradiction reports ignored;
- excessive draft backlog.
5. Audit write safety
Check whether agents can:
- edit
raw/; - delete source material;
- bulk rewrite wiki pages without dry-run;
- bypass git/PR review without explicit approval;
- follow prompt-injection instructions from captured content;
- expose sensitive material to external tools.
6. Grade risk
Use this scorecard:
| Area | Risk | Evidence | Fix |
|---|---|---|---|
| Raw immutability | low/medium/high | ||
| Provenance | low/medium/high | ||
| Review gates | low/medium/high | ||
| Human synthesis boundary | low/medium/high | ||
| Staleness | low/medium/high | ||
| Agent write safety | low/medium/high | ||
| Sensitive data | low/medium/high |
7. Recommend fixes
Prioritize fixes that reduce trust risk before productivity features:
- protect raw sources;
- add review states;
- add provenance fields;
- protect human sections;
- add lint reports;
- add dry-run/PR workflow;
- add retrieval upgrades only after trust basics.
Output
## Trust audit summary
## Top risks
## Evidence
## Scorecard
## Fix plan
## Stop conditions
## Optional follow-up skills
Safety gates
- Do not mark the wiki safe without evidence.
- Do not auto-fix truth conflicts during audit.
- Do not expose sensitive file contents in the final report.
- Do not treat beautiful structure as proof of correctness.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |