Skip to content

aiopshwang/data-analysis-ml-agent-skills / skills/auditing-data-and-ground-truth

auditing-data-and-ground-truth

Verified

Audit datasets, joins, labels, and ground truth before analysis or modeling. Use when data meaning, row grain, time semantics, source-of-truth reliability, or label construction may invalidate conclusions; not for general model evaluation after the evidence base is already trusted.

FileKindSize
SKILL.mdskill md3.7 KB
agents/openai.yamlconfig250 B
assets/data-and-ground-truth-audit.mdmarkdown2.5 KB
references/audit-playbook.mdmarkdown4.0 KB

assets/data-and-ground-truth-audit.md

View on GitHub
# Data and ground-truth audit

## Intended use

| Field | Definition | Confidence and evidence |
|---|---|---|
| Decision or claim |  |  |
| Entity and row grain |  |  |
| Target |  |  |
| Prediction or observation time |  |  |
| Outcome window |  |  |
| Population and exclusions |  |  |

## Source and lineage inventory

| Source | Owner | Version or extract time | Upstream filters or transformations | Expected grain and keys | Independent evidence |
|---|---|---|---|---|---|
|  |  |  |  |  |  |

Raw preservation location or snapshot identifiers:

## Structural evidence

| Check | Expected | Observed | Scope | Result | Evidence artifact |
|---|---|---|---|---|---|
| Row and entity counts |  |  |  |  |  |
| Key uniqueness |  |  |  |  |  |
| Duplicate classes |  |  |  |  |  |
| Missingness semantics |  |  |  |  |  |
| Value ranges and units |  |  |  |  |  |

## Join evidence

| Join | Expected cardinality | Unmatched left/right | Rows before/after | Entity effect | Multiplication risk | Disposition |
|---|---|---|---|---|---|---|
|  |  |  |  |  |  |  |

## Time evidence

| Field or event | Event time | Record time | Availability time | Backfill behavior | Risk or test result |
|---|---|---|---|---|---|
|  |  |  |  |  |  |

## Ground-truth evidence

- Real-world event represented:
- Label construction and version:
- Provenance and owner:
- Maturity rule:
- Missing or unresolved-label policy:
- Coverage by class, source, time, and relevant slice:
- Agreement or independent reconciliation method:
- Known ambiguity:

### Human review, when applicable

- Reviewer roles and relevant qualifications:
- Written instructions and label version:
- Blinding to predictions or prior labels:
- Individual decisions preserved at:
- Raw and chance-adjusted agreement, when appropriate:
- Disagreement categories and rate:
- Adjudicator and independence:
- Adjudication rule and output:
- Unresolved or inherently ambiguous cases:

| Sample or stratum | Sample method | Compared evidence | Agreement or mismatch | Mismatch class | Estimated affected scope |
|---|---|---|---|---|---|
|  |  |  |  |  |  |

## Findings and dispositions

| ID | Severity | Finding | Direct evidence | Affected scope | Consequence | Treatment | Owner or approval needed | Residual risk |
|---|---|---|---|---|---|---|---|---|
| F-01 |  |  |  |  |  |  |  |  |

## Readiness judgment

Status: **unknown / not ready / ready with conditions / ready**

Supported uses:

Unsupported uses:

Required conditions:

Evidence still missing:

Next falsifying or reconciliation check:
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall