wanshuiyin/Anti-Autoresearch / skills/experiment-forensics
experiment-forensics
UnverifiedAudit experiment integrity against the evidence ledger. At L2 (repo + result files present) a fresh cross-model reviewer reads the eval code line-by-line for fake/derived ground truth, score self-normalization, phantom results (a paper number with no backing file/key), dead/uncalled metric code, verified-scope inflation, method-described ≠ method-evaluated drift, synthesized-looking results, placeholder/fake data still wired into a released result, code-output ≠ reported-number mismatch, and missing reproducibility artifacts (an empirical/agent/LLM paper shipping neither code nor the prompts/configs its results need) — every finding span-anchored to a ledger claim_id. At L0/L1 (PDF / source only) the same patterns are surfaced as info-level 'could-not-verify' signals where the ledger gives an anchor (observability_level_required:2) — NEVER a fraud verdict from a PDF. The reviewer PROPOSES findings; tools/adjudicate_findings.py computes the verdict. Detect-only. Triggers: "experiment forensics", "audit the results", "check the eval code", "实验诚实度".
Security report
UnverifiedStatic analysis of one exact version. It reduces risk and shows evidence; it does not certify that a skill is safe.
- Commit
014a1e7cd63f1656603daaeeadb856aed1df6869- Content hash
4e2e2d13a25f753c0050002d760bcd8ce855cf8e0b3e12bf0a4793fd029b30f9- Scanned
- 2026-11-17
- Scanner version
- skillxone-scan-1.0.0
- Policy version
- 0.1-draft
- Decision
- 1 open medium finding to review.
- Verification expires
- Not applicable
Findings (1 medium, 1 info)
The text tells the agent to keep something from the user, which hides actions taken on their behalf.
been built and this skill never invents one — tell the user to run `/evidence-ledger`Suggested fix: Remove the instruction. A skill should never ask the agent to conceal what it does from the user.
SKILL.md pre-approves unrestricted shell access through `allowed-tools`, so the agent may run commands without asking.
allowed-tools: Bash(*), Read, Write, Grep, Glob, mcp__codex__codexSuggested fix: Restrict the tool, for example `Bash(git status:*)`, or leave approval to the user.
Checks run
- prompt-injection
- dangerous-commands
- credentials
- network
- file-access
- integrity
Checks skipped
- dependencies: No dependency manifest in the skill directory.
- gitleaks: gitleaks is not installed on this host.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |