gaia-research/gaia-skill-tree / .agents/skills/ev-adversarial-audit
ev-adversarial-audit
VerifiedRun this skill for Phase 3 of the evidence verification pipeline — the adversarial audit step. Use when you need to check the evidence data lake for bad data before ingestion: dead links, wrong URL formats (tree/ vs blob/), subjective wording ("elite", "high-quality"), stale migration notes, benchmark catalog misuse/vendor-claim leakage, or skills whose star evidence conflicts with classified evidence level. Triggers on phrases like: "adversarial evidence check", "adversarial check", "ev-adversarial-audit", "check for noise in evidence", "flag bad evidence", "run the audit phase", "quality check the by-type files", or any reference to Phase 3 of the pipeline.
Adversarial Evidence Audit (ev-adversarial-audit)
Phase 3 audits the evidence lake from a Devil's Advocate perspective before URL health validation and ingestion.
Type-First Evidence Lake Contract (#1148)
The evidence lake is type-first. Audit evidence/by-type/<canonical-evidence-type>.md files. Legacy evidence/tier_*.md files may still exist as coexistence artifacts, but they are not the semantic routing key.
Audit Target
Split reviewer work across evidence/by-type/<type>.md files, not tier files. Suggested sharding:
- Repo/adoption signals:
repo-own,github-stars-own,social-signal - Technical proof:
benchmark-result,arxiv,peer-review - Governance/proxy proof:
proxy-containment,verifier-attestation - Composition/self proof:
fusion-recipe,self-attestation
Findings to Flag
- Dead or malformed URLs, including GitHub
tree/links where ablob/source is required. - Subjective/evaluative wording not supported by the source.
- Evidence type mismatches or legacy alias leakage.
- Star evidence that conflicts with live verification notes from Phase 2.
- Social/proxy/self-attestation policy violations per
docs/agents/curation-authority-map.md(#2473): do not rejectsocial-signalrows merely for being first-party (sameAsAuthor = 1.0) or published within 30 days (publishCooldownDays = 0— no 30-day hold); instead flagsocial-signalrows that fail target-skill relevance (unrelated channel uploads or generic promo), lack provenance (creatorRelation/relevanceNote), fall below the 1,000-view scoring floor (views < 1000= 0 TM), or violate(type, url)dedup / 3-row same-creator plateau. Flag anyproxy-containmentrow pointing to a first-party, canonical, orsuiteRefrepository or withexternalStars < 10000. Flag any syntheticself-attestationrow added solely to push a suite component past the 49.9 TM shared-repository baseline cap. - Benchmark catalog misuse after Phase 2B:
benchmark-resultrows citing unknown, candidate, registered, rejected, or retired sources as if they score; vendor claims presented as reproducible benchmarks; scoring provenance on non-verified catalog entries; missing reproducibility fields or missing/dubious percentile values. - Stale migration notes that still treat
tier_*.mdas the semantic working set. - Multi-target peer-review packet misuse: wrong
evidenceType, emptytargets, invalidskillId, duplicate(source.url, skillId, evidenceType)rows, or forbidden strength/scoring fields (trustNumber,grade,class,tier,level,stars,rank). - Imposter over-purging & audit taxonomy conflation (RFC #1809): ensure legitimate early-stage, self-hosted, or
installable: falseskills (Tier 2 Packaging Gap, Tier 3 Under-Evidenced Stub) are not mislabeled as Tier 1 Malicious Imposters or flagged for deletion.
Append concise findings to the source report. Do not mutate registry files.
Optional Semantic Evidence Review Sidecar
An operator may optionally run the read-only Jev advisory runner to flag semantic anomalies in evidence rows (docs/agents/jev.md):
# Offline dry-run by default ($0 spend); add --live to query TypeSafe API
python scripts/jev_advisory.py --mode evidence --input <evidence-rows.json>
This output serves as review triage hints only. It never marks evidence rows verified, does not replace deterministic HTTP health or Phase 2B benchmark verification, does not touch Trust Magnitude scoring, and emits an actionable handoff to worker-luna on uncertainty.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |