gaia-research/gaia-skill-tree / .agents/skills/ev-discovery
ev-discovery
VerifiedSkippable Phase 0 of the Gaia evidence verification pipeline. Use only on declared need for higher-quality Stage-2 evidence: a promotion candidate, or a skill that gaia-meta-sweep flagged as under-evidenced. Given a skill id, source repo/context, and needed evidence types (benchmark-result, arxiv, peer-review, social-signal), it runs Firecrawl searches, scrapes top hits to markdown, and appends discovered sources into Phase 1 inputs. Missing benchmark evidence is recorded as benchmark-source candidates first, not direct scoring rows. Trigger phrases: "discover evidence", "find new evidence", "Phase 0", "run ev-discovery", "search for benchmarks/papers", "under-evidenced", "promotion candidate needs stronger proof". This is the ONLY ev-* skill that searches the web for NEW evidence; other ev-* skills aggregate or verify known sources. Requires Firecrawl to be usable (`firecrawl --status` authenticated OR FIRECRAWL_API_KEY set); skips gracefully when neither holds.
Evidence Discovery (ev-discovery)
Different from pre-curation
/ev-scout: scouting happens before topology mapping and casts a wide contextual net, including rivals and negative evidence. This Phase 0 is a later, optional evidence-development step for selected Stage-2 sources. Scout leads are unverified and non-scoring until the canonical verification and human approval sequence completes.
Phase 0 is optional. It searches for new, higher-quality Stage-2 evidence only when a maintainer declares a need.
Type-First Evidence Lake Contract (#1148)
The evidence lake is type-first. Discovery feeds Phase 1, which materializes evidence/by-type/<canonical-evidence-type>.md. Legacy evidence/tier_*.md files may still exist as coexistence artifacts, but they are not the semantic routing key.
Discovery Scope
Prioritize new evidence rows for:
benchmark-resultarxivpeer-review- richer
social-signal(eligible first-party and third-party demonstrations of the target skill,sameAsAuthor = 1.0,publishCooldownDays = 0— no 30-day hold,views >= 1000floor,(type, url)dedup, and 3-row plateau; recordcreatorRelationandrelevanceNote, and reject unrelated uploads; seedocs/agents/curation-authority-map.mdand #2473) - qualifying external
proxy-containment(strictly external, non-first-party repositories withexternalStars >= 10000outside the skill's own/canonical/suite repository)
Discovery may also identify better repo-own or github-stars-own sources, but it must not assign authoritative rank, tier, grade, or Trust Magnitude, and must never propose synthetic self-attestation rows to clear the 49.9 TM suite-component shared-repository cap.
Benchmark candidate handoff
When discovery finds missing benchmark evidence, create a benchmark-source candidate record for Phase 2B review instead of appending a scoring benchmark-result row from a vendor page. The candidate should name the target skill, source URL, proposed benchmarkId, proposed generic applicability (appliesToGenericSkillRefs) when known, status: candidate, missing reproducibility fields, and notes. No discovered vendor page may become a scoring row until a human promotes the source in registry/benchmark-sources.json and approves /gaia-ingest-batch. A promoted registered benchmark source is still generic-applicable, non-scoring catalog metadata only until a named benchmark-result row satisfies reproducibility fields and approved provenance. Firecrawl-style vendor benchmark claims are candidate-only or registered-only until reproducibility fields, allowed provenance, and human approval exist.
Handoff to Phase 1
Append discovered raw rows to the appropriate source inputs with a canonical or normalizable evidence type. Phase 1 is responsible for normalizing aliases and materializing evidence/by-type/<type>.md. Do not hand-write generated by-type partitions as if they were source rows.
If one discovered peer-review source legitimately covers multiple named skills, hand Phase 1 a scratch multi-target packet manifest for evidence/scripts/peer_review_source_packets.py rather than cloning the same review into multiple intake mutations. The helper is peer-review-only, allows the same URL once per reviewed skill, rejects strength/scoring fields (trustNumber, grade, class, tier, level, stars, rank), and should write only temporary by-type output.
Preflight (#1788)
Firecrawl usability is not gated on the FIRECRAWL_API_KEY env var alone — the
CLI can be authenticated via stored browser/CLI credentials independent of it.
Check both before deciding to skip Phase 0:
firecrawl --status # Authenticated? credits remaining?
echo "${FIRECRAWL_API_KEY:+set}" # non-empty output if the env var is set
Proceed with discovery if either check succeeds. Skip Phase 0 gracefully
only when firecrawl --status reports unauthenticated and the env var is
unset/empty.
Commands
Quote every URL and query — unquoted ?/&/glob characters expand under zsh,
this repo's shell, and silently turn a single request into a file-glob error.
mkdir -p .firecrawl
firecrawl search "<query>" -o .firecrawl/result.json --json
firecrawl scrape '<url>' --json -o .firecrawl/scrape.json
gh api -X GET 'repos/<owner>/<repo>/git/trees/main' -f recursive=1
The gh api form above (path + -f recursive=1 instead of a literal
?recursive=1 in the URL) avoids the same zsh glob-expansion problem for
GitHub tree listings.
Retry rule
Retry is a status-code rule, not a named-host list — a source is not "unresolved" just because it wasn't on someone's retry shortlist:
- On any
401,402,403,429, or SSL/TLS failure from a direct fetch, retry once viafirecrawl scrapebefore marking the source dead. - Do not maintain or consult a list of specific domains to retry — apply the rule to every failing URL, every time.
Guardrails
- Requires Firecrawl usability per the Preflight above (
firecrawl --statusORFIRECRAWL_API_KEY); skip gracefully only when both fail. - Do not run as a broad web crawl without declared need.
- Do not mutate registry files.
- Do not append direct scoring
benchmark-resultrows from discovered vendor pages; route them through benchmark-source candidates and Phase 2B first. - Do not treat generic-applicable registered benchmark sources as Trust Magnitude evidence; they are non-scoring until a named row satisfies reproducibility and approved provenance.
- Do not commit generated collector/by-type/report artifacts without a human gate.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |