Skip to content

gaia-research/gaia-skill-tree / .agents/skills/gaia-audit

gaia-audit

Verified

Focused single-target audit of one Gaia skill, named skill, or real-skill catalog item. Use this skill when: auditing a specific skill by ID or name, a skill looks overpromoted, evidence feels thin or dead, a contributor's skill seems outdated or superseded, the rank doesn't match the source quality, evidence is mapped to the wrong Evidence Type, a skill at 3★+ is missing a working GitHub link, someone asks "does this skill deserve its stars?", or you want the smallest possible correction with validation — not a bulk sweep. This is NOT for bulk curation passes (use /gaia-curate for that) or meta-level registry sweeps (use /gaia-meta-audit). It is specifically for "fix this one skill, get it right."

FileKindSize
SKILL.mdskill md11.6 KB
---
name: gaia-audit
description: >
  Focused single-target audit of one Gaia skill, named skill, or real-skill catalog item.
  Use this skill when: auditing a specific skill by ID or name, a skill looks overpromoted,
  evidence feels thin or dead, a contributor's skill seems outdated or superseded, the rank
  doesn't match the source quality, evidence is mapped to the wrong Evidence Type, a skill at 3★+ is
  missing a working GitHub link, someone asks "does this skill deserve its stars?", or you
  want the smallest possible correction with validation — not a bulk sweep.
  This is NOT for bulk curation passes (use /gaia-curate for that) or meta-level registry
  sweeps (use /gaia-meta-audit). It is specifically for "fix this one skill, get it right."
version: 1.1.0
---

# gaia-audit

**Yggdrasil IV:** [Trust degradation and reversible technical-challenge contract](../../../founder/YGGDRASIL_IV_RATIFICATION_2026-10-25.md). This focused audit verifies evidence, distinguishes temporary liveness from independently verified material counterevidence, and prepares a traceable adjudication packet. An automated HTTP check, model claim, inactivity, or competitor marketing cannot change TM/stars or declare a skill obsolete. Class B Steward can close proven technical challenges **only**, while canonical changes require Class C approval.

Perform a focused, single-target audit of one Gaia skill or catalog item and apply the smallest correct source-level fix. The goal is accuracy over coverage: one skill, done properly.

## Why single-target?

Bulk passes introduce unintended side effects and make diffs hard to review. A single-target audit lets you go deep — verifying the actual source material, not just the metadata — and produces a minimal, reviewable change. If you find adjacent issues while auditing, note them but do not fix them in this pass.

## Workflow

### 1. Locate the target

Determine which registry layer owns the skill:
- Generic (starless) canonical skill: `registry/nodes/<id>.json` (or via `registry/gaia.json`)
- Named skill implementation: `registry/named/<contributor>/<skill>.md`
- Real-skill catalog item: `registry/real-skills.json`

Run `rg <skill-id>` across the repo to find every occurrence — frontmatter references, named files, catalog entries, and timeline events — before touching anything.

### 2. Ask the diagnostic questions

Work through these in order. Stop when you find a violation — that's the correction to make.

- Does the skill exist at all in the registry?
- Is it generic (no stars) or named (2★–6★)? Generic skills have no level; only named implementations carry stars.
- Does it map to the claimed Gaia capability? (Check `CONTEXT.md` for canonical vocabulary.)
- Does the evidence actually support the claimed rank (stars)? Re-read each source URL — homepage and directory listings are weak; `SKILL.md` blobs, papers, benchmarks, and release notes are strong.
- Is it outdated, superseded, or a duplicate of something already in the registry?
- For named skills at 3★+: is there an independently confirmed absence of the required `blob/branch/subpath` implementation link? A temporary dead URL is only an observation; a verified Star Bar violation may warrant an approved hard-reset review.
- For fusion/upgrade opportunities: note if related basic skills could cleanly consolidate, but do not act on it in this pass unless the target is explicitly the consolidation candidate.
- Does the skill have structural, documentation, or packaging issues? **Classify against the Four-Tier Audit Taxonomy (GOVERNANCE.md §4.2 / RFC #1809):**
  - **Tier 1 (Malicious / Imposter Squatter)**: Bad-faith aggregation, hijacking other maintainers' repos/stars, or falsified attribution → Expungement from registry.
  - **Tier 2 (Packaging / Install Shape Gap)**: Authentic concept and author, but missing standard packaging or broken links → For ≤2★, tag `installable: false` per CONTRIBUTING.md §12. For 3★+, independently establish the Star Bar facts, request repackaging, and queue the existing 1★ consequence for **approved** adjudication; do not demote from a single URL probe. **Never purge or label as an imposter.**
  - **Tier 3 (Early-Stage / Under-Evidenced Stub)**: Authentic author, thin docs or preliminary evidence → Keep at baseline 1★ (Awakened), request doc enrichment, or tag `needs-info`. **Never purge.**
  - **Tier 4 (Product-Coupled / Non-Generalized)**: Real tool documentation needing generalization (#1766, #1801) → Enforce maker attribution via `check_product_attribution.py` and route to generalization.
- **Mandatory Contributor Check**: If the author is a known contributor, co-founder, or team member, deletion is strictly prohibited without direct maintainer outreach and documented review.

Skip the `rarity` field entirely — it is deprecated and carries no review signal (see `CONTEXT.md` § Rarity).

### 3. Present findings before acting

Write out what you found and the proposed correction before making any edit. This gives the user a chance to redirect. If no correction is warranted, say so explicitly with the reason — "no action" is a valid and useful audit outcome.

### 4. Apply the correction via CLI

All mutations go through the CLI, not direct file edits — this preserves timeline logging and schema integrity.

```bash
# Adjust rank (named skills only — use contributor/skill format, not bare generic IDs)
gaia dev calibrate <contributor/skill_id> <N>★

# Change generic type (basic / fusion)
gaia dev reclassify <generic_id> <type>

# Update named status (awakened / named / etc.)
gaia dev update-named <contributor/skill_id> --status <status>

# Add or replace evidence
gaia dev evidence <skill_id> "<url>" --type <arxiv|repo|github-stars|...> --trust <value>

# Log a demotion event in the timeline
gaia dev timeline <skill_id> --action demote --notes "<reason>"
```

For dead evidence links: create/refresh Steward verification debt and collect repeated source/provenance evidence. **Do not remove evidence, mint demerits or write demotion events merely because a URL fails.** After separately approved adjudication, use governed CLI mutations and complete both canonical and owner-tree timeline receipts.

### 5. Regenerate and verify

```bash
gaia dev build       # sync all derivative files after registry edits
gaia dev validate
gaia dev validate --intake
gaia dev test all
git diff --check
```

## Demotion enforcement rules

These are hard rules — they apply regardless of how prominent or well-known the skill is:

- **3★+ with independently confirmed missing implementation blob** → propose the `META.md` §2.4 **1★ hard-reset** consequence through Class C adjudication; perform a canonical/owner-tree timeline-safe CLI change **only after approval**. A transient 404 is not sufficient.
- **Unique-branch skill demoted below 4★** → it loses its Unique-branch rank status; if its structure no longer warrants `fusion`, reclassify the generic to `basic` (`gaia dev reclassify` accepts only `basic` / `fusion`)
- **Dead evidence links** → mark as liveness observation; investigate and surface verified challenge, never auto-remove evidence or auto-demote.
- **Placeholder or non-verifiable evidence only** → present the evidence deficit and seek independent verification / approved canonical ruling; no automatic Awakened reset.
- **Suite components**: the suite itself does not need `links.github`, but each component must have a specific `blob/branch/subpath` URL

## Frontier challenger-lead input

A `challenger-lead-v1.1`, `frontier-verification-request-v1`, `frontier-experiment-receipt-v1` or `frontier-meta-handoff-v1` is an unverified request, never proof of independent verification. Read `docs/architecture/ecosystem-intelligence.md`; validate source bytes, exact actual target identity/revision/content digest, claim/campaign/protocol/fixture/history/request cross-bindings and dedupe. Execute only the fixed reviewed repository protocol; never execute packet strings, infer superiority from similarity/vendor marketing, or accept a self-declared actor/reviewer/receipt/status. Treat timeout/unavailable as `execution_failed`, not a negative result; treat comparative mismatch as `inconclusive` unless the fixed protocol's interpretation criteria support a narrow conclusion. Record both positive and counterevidence. No result here authenticates the verifier or admits a public challenge. Do not alter canonical evidence, TM, stars, Origin, or Steward state; guarded admission uses the existing challenge-domain trusted resolver and fails closed when unavailable.

### Meta verification record

When the input includes an experiment receipt, finish by writing one `frontier-meta-verification-v1` record (`src/gaia_cli/data/frontier/meta-verification-v1.schema.json`). It is a claim for human Meta review, never an approval or admission. Positive outcomes, rerun traces and triage-stage records use `schemaVersion: frontier-meta-verification-v1.1`; see `gaia-frontier` for the trace requirements.

- **Separate session.** Verify in a different session from the one that produced the receipt. Set `verifierRunId` to a fresh id that differs from the receipt's `experimentRunId`, and bind `requestId`, `receiptRunId` and `receiptDigest` (`frontier.receipt_digest`) to the receipt you assessed.
- **Re-execute, don't re-read.** Run the fixed reviewed protocol yourself and store the outputs under the case's `evidence/frontier-challenges/<id>/artifacts/` (or `generated-output/frontier/` while drafting). List them in `reexecution.artifacts` with digests, using a different path from every receipt output. `substantiated`/`narrowed` (and any `request-admission-review` recommendation) are refused unless the receipt result is `supported` with matching documented/executed versions, the artifacts cover every receipt output label, the `native` artifact parses under the request-pinned protocol (same experiment, request, fixture, protocol and executed version) and its `output` equals the receipt's native `output` after the protocol's declared `outputNormalization` (exact match by default; git-log-L masks commit ids), and none is a byte-identical copy of a receipt output. Because `narrowed` shares the `supported`-receipt requirement, against a `contradicted`, `inconclusive` or `execution_failed` receipt only `contradicted`, `inconclusive` or `execution_failed` outcomes are available; record any narrower finding in `limitations` and `recommend` `narrow-claim` or `hold`.
- **Counterevidence.** State what you searched for in `counterevidence.searched` even when `items` is empty; list contrary findings with a source.
- **Outcome** is one of `substantiated`, `narrowed`, `contradicted`, `inconclusive`, `execution_failed`. Outages and timeouts are `execution_failed`; a documented-versus-executed version mismatch can be at most `inconclusive`. `recommend` is a suggestion (`request-admission-review` only for `substantiated` or `narrowed`).
- **Independence is declared, not proven.** Always `"independence": "declared-unauthenticated"`; `reexecution.performed`, `verifierRunId` and "separate session" are claims by the record's author that nothing here authenticates. Never add approval, admitted, verified, actor, signature, confidence, TM, stars, level or Origin fields; the validator refuses them at any depth.
- Check the record with `gaia dev frontier meta-validate --record <record> --receipt <receipt> --request <request> --artifact-root <artifact root>` (`--request` is required; add `--repo-root` outside the checkout). A pass is a structural check only.

## Output

Conclude with a concise report covering:
- Which skill was audited and what layer it lives in
- Evidence reviewed (URLs checked and their quality)
- Correction applied, or the explicit reason no change was warranted
- CLI commands run
- Validation results
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall