gaia-research/gaia-skill-tree / .agents/skills/ev-collection
ev-collection
VerifiedPhase 1 of the Gaia evidence verification pipeline. Use this skill whenever you need to gather, consolidate, or refresh raw evidence for named skills in the registry. Trigger phrases: "collect evidence", "populate the data lake", "gather sources", "run ev-collection", "Phase 1", "compile the evidence index", "refresh the unified lake", "aggregate skill evidence", "build the evidence database". Also invoke as the first step before running ev-star-verification, ev-adversarial-audit, or ev-link-validation — nothing downstream is meaningful without a fresh, compiled index. Covers GitHub repo links, stargazer signals, YouTube showcases, arXiv papers, peer reviews, benchmark results, blog/newsletter posts, and self-attestation artifacts.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 6.3 KB |
SKILL.md
View on GitHub---
name: ev-collection
description: >
Phase 1 of the Gaia evidence verification pipeline. Use this skill whenever you need to gather, consolidate, or refresh raw evidence for named skills in the registry. Trigger phrases: "collect evidence", "populate the data lake", "gather sources", "run ev-collection", "Phase 1", "compile the evidence index", "refresh the unified lake", "aggregate skill evidence", "build the evidence database". Also invoke as the first step before running ev-star-verification, ev-adversarial-audit, or ev-link-validation — nothing downstream is meaningful without a fresh, compiled index. Covers GitHub repo links, stargazer signals, YouTube showcases, arXiv papers, peer reviews, benchmark results, blog/newsletter posts, and self-attestation artifacts.
---
# Evidence Collection (ev-collection)
Phase 1 materializes and compiles the evidence lake. It is the foundation for star verification, adversarial audit, and link validation.
## Type-First Evidence Lake Contract (#1148)
The evidence lake is **type-first**. The primary working set is `evidence/by-type/<canonical-evidence-type>.md`. Legacy `evidence/tier_*.md` files may still be emitted as coexistence artifacts, but they are **not** the semantic routing key.
## Multi-Target Peer-Review Source Packets (#1418)
If one real peer-review source URL covers multiple named skills, materialize a temporary peer-review partition before compile instead of cloning intake rows or editing canonical evidence artifacts:
```bash
python evidence/scripts/peer_review_source_packets.py \
--manifest /path/to/manifest.json \
--by-type-dir /tmp/ev1418/by-type
```
This helper is `peer-review`-only, writes scratch `peer-review.md`, and emits one row per reviewed skill even when the same URL legitimately appears under each target. Reject strength/scoring fields anywhere in the packet (`trustNumber`, `grade`, `class`, `tier`, `level`, `stars`, `rank`). Do not commit scratch manifests or generated partitions without human approval.
## What Phase 1 does
1. Reads active source inputs, collector channels, and named-skill evidence rows.
2. Normalizes evidence row types, including legacy aliases such as `repo` → `repo-own` and `github-stars` → `github-stars-own`.
3. Writes deterministic primary partitions under `evidence/by-type/`.
4. Compiles `evidence/unified_evidence_lake.md` from by-type partitions.
5. May emit `evidence/tier_*.md` only for coexistence with older scripts.
## Commands
```bash
python evidence/scripts/generate_source_dump.py \
--output-dir evidence \
--by-type-dir evidence/by-type
python evidence/scripts/compile_data_lake.py \
--sources evidence/by-type \
--lake evidence
```
For deterministic temp-dir validation or tests, add `--skip-live-stars` and `--no-legacy-tiers` to `generate_source_dump.py`. Do not commit generated lake artifacts unless a human gate explicitly approves them.
## Pre-registry intake candidates (#1786)
A skill proposed by an intake isn't in `registry/named/` yet, so the default Phase 1 read (registry markdown + `gaia.json`) never sees its rows — there was previously no way to run Phases 1-4 on exactly the rows an L4-approved intake is meant to gate. Feed a candidate manifest in alongside the registry:
```bash
python evidence/scripts/generate_source_dump.py \
--output-dir evidence \
--by-type-dir evidence/by-type \
--candidate-manifest /tmp/intake-candidates.json
```
**Manifest shape** — a JSON (or `.jsonl`) file, one of:
- an object with a `candidates` / `entries` / `rows` list,
- a bare JSON array,
- JSONL records (one JSON object per line, `.jsonl` extension).
Each entry is a skill-shaped dict, the same shape a named skill's own frontmatter carries:
```json
{
"id": "contributor/candidate-skill",
"name": "Candidate Skill",
"contributor": "contributor",
"genericSkillRef": "generic-skill-id",
"evidence": [
{"type": "repo-own", "source": "https://github.com/contributor/tool", "date": "2026-09-01", "notes": "..."}
]
}
```
`id` is required; `genericSkillRef` is optional (only needed to inherit generic-level evidence rows). Candidate rows merge into the same by-type partitions, deduped and normalized identically to a registered skill's own evidence — they are visually indistinguishable in the output except by `id`. A candidate with no `level` field lands in the default tier bucket; that is cosmetic bucketing only and confers no rank.
To validate a candidate's URLs before or without a full Phase 1 run, `evidence/scripts/validate_sources.py --manifest <path>` accepts the identical manifest shape — see [`/ev-link-validation`](../ev-link-validation/SKILL.md).
## Canonical source URL by evidence type (#1787)
Each evidence type has one canonical URL shape; using the wrong shape breaks that type's scoring inputs or collides with same-source dedup unexpectedly. See `docs/agents/curation-guidelines.md` §5 and [`docs/agents/curation-authority-map.md`](../../../docs/agents/curation-authority-map.md) for the full contract (repo-own, github-stars-own, proxy-containment, verifier-attestation, benchmark-result, arxiv, peer-review, self-attestation, social-signal, npm-downloads, fusion-recipe). Note:
- Same-source dedup is `(type, url)`-keyed — two rows collapse only when they share BOTH the same `type` AND the same canonical URL; a `repo-own` row and a `github-stars-own` row at the identical URL are different keys and both count independently.
- Under #2473, `social-signal` admits both eligible first-party (author/maintainer) and third-party demonstrations of the target skill (`sameAsAuthor = 1.0`, `publishCooldownDays = 0` — no 30-day publication hold, 1,000-view floor, 3-row plateau, `creatorRelation` + `relevanceNote` provenance; reject unrelated uploads).
- `proxy-containment` strictly requires a qualifying external, non-first-party repository (`externalStars >= 10000`, distinct `owner/repo` outside the skill's own/canonical/suite repository).
- Never synthesize `self-attestation` rows to push a suite component past the 49.9 TM shared-repository cap.
## Completion Criteria
- `evidence/by-type/<type>.md` is current and is the primary downstream input.
- `evidence/unified_evidence_lake.md` was compiled from by-type files, not stale tier files.
- Collector rows are not re-parsed by default when by-type files exist, avoiding duplicate ingestion from coexistence dual-writes.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |