aiopshwang/data-analysis-ml-agent-skills / skills/auditing-data-and-ground-truth
auditing-data-and-ground-truth
VerifiedAudit datasets, joins, labels, and ground truth before analysis or modeling. Use when data meaning, row grain, time semantics, source-of-truth reliability, or label construction may invalidate conclusions; not for general model evaluation after the evidence base is already trusted.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 3.7 KB |
| agents/openai.yaml | config | 250 B |
| assets/data-and-ground-truth-audit.md | markdown | 2.5 KB |
| references/audit-playbook.md | markdown | 4.0 KB |
assets/data-and-ground-truth-audit.md
View on GitHub# Data and ground-truth audit
## Intended use
| Field | Definition | Confidence and evidence |
|---|---|---|
| Decision or claim | | |
| Entity and row grain | | |
| Target | | |
| Prediction or observation time | | |
| Outcome window | | |
| Population and exclusions | | |
## Source and lineage inventory
| Source | Owner | Version or extract time | Upstream filters or transformations | Expected grain and keys | Independent evidence |
|---|---|---|---|---|---|
| | | | | | |
Raw preservation location or snapshot identifiers:
## Structural evidence
| Check | Expected | Observed | Scope | Result | Evidence artifact |
|---|---|---|---|---|---|
| Row and entity counts | | | | | |
| Key uniqueness | | | | | |
| Duplicate classes | | | | | |
| Missingness semantics | | | | | |
| Value ranges and units | | | | | |
## Join evidence
| Join | Expected cardinality | Unmatched left/right | Rows before/after | Entity effect | Multiplication risk | Disposition |
|---|---|---|---|---|---|---|
| | | | | | | |
## Time evidence
| Field or event | Event time | Record time | Availability time | Backfill behavior | Risk or test result |
|---|---|---|---|---|---|
| | | | | | |
## Ground-truth evidence
- Real-world event represented:
- Label construction and version:
- Provenance and owner:
- Maturity rule:
- Missing or unresolved-label policy:
- Coverage by class, source, time, and relevant slice:
- Agreement or independent reconciliation method:
- Known ambiguity:
### Human review, when applicable
- Reviewer roles and relevant qualifications:
- Written instructions and label version:
- Blinding to predictions or prior labels:
- Individual decisions preserved at:
- Raw and chance-adjusted agreement, when appropriate:
- Disagreement categories and rate:
- Adjudicator and independence:
- Adjudication rule and output:
- Unresolved or inherently ambiguous cases:
| Sample or stratum | Sample method | Compared evidence | Agreement or mismatch | Mismatch class | Estimated affected scope |
|---|---|---|---|---|---|
| | | | | | |
## Findings and dispositions
| ID | Severity | Finding | Direct evidence | Affected scope | Consequence | Treatment | Owner or approval needed | Residual risk |
|---|---|---|---|---|---|---|---|---|
| F-01 | | | | | | | | |
## Readiness judgment
Status: **unknown / not ready / ready with conditions / ready**
Supported uses:
Unsupported uses:
Required conditions:
Evidence still missing:
Next falsifying or reconciliation check:
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |