radarist/structured-analytic-skills / skills/evidence-appraisal
evidence-appraisal
VerifiedRates how much certainty a body of evidence deserves, outcome by outcome, on the GRADE scale (Grading of Recommendations Assessment, Development and Evaluation — high, moderate, low, very low), naming the reason behind every downgrade or upgrade and writing the summary-of-findings row and a calibrated certainty statement. Use when asked "how strong is the evidence for X?", "GRADE this evidence", "is this proven or just one study?", or "the studies conflict — what is actually known?". Not for one study's risk of bias (use `assess-study-bias`) or for grading a non-study source such as an article or dataset (use `rate-source-admiralty`).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 11.4 KB |
| agents/openai.yaml | config | 356 B |
| evals/evals.json | config | 5.0 KB |
| scripts/grade.pyexecutable | script | 38.9 KB |
evals/evals.json
View on GitHub{
"skill_name": "evidence-appraisal",
"evals": [
{
"id": 1,
"case_id": "grade-pos-1",
"kind": "positive",
"prompt": "GRADE the evidence for CBT-I versus sleep-hygiene education on sleep-onset latency — we have five RCTs with their risk-of-bias ratings and the pooled estimate. What certainty level does this body of evidence deserve?",
"expected_output": "A response that satisfies these checks: Rates certainty per outcome starting from High for randomized trials; Applies each of the five rate-down factors (or explicitly declines it) with a reason tied to the appraisal table or pooled statistics; Reports the level as high/moderate/low/very low with a summary-of-findings row and a GRADE-language certainty statement; Reproduces the arithmetic with scripts/grade.py or by hand and names what new evidence would change the grade.",
"files": [],
"expectations": [
"Rates certainty per outcome starting from High for randomized trials",
"Applies each of the five rate-down factors (or explicitly declines it) with a reason tied to the appraisal table or pooled statistics",
"Reports the level as high/moderate/low/very low with a summary-of-findings row and a GRADE-language certainty statement",
"Reproduces the arithmetic with scripts/grade.py or by hand and names what new evidence would change the grade"
],
"skills": [
"evidence-appraisal"
]
},
{
"id": 2,
"case_id": "grade-pos-2",
"kind": "positive",
"prompt": "How strong is the evidence that intermittent fasting improves insulin sensitivity? The studies conflict and I need to know what we actually know before writing the recommendation.",
"expected_output": "A response that satisfies these checks: Frames a PICO and rates outcomes separately rather than issuing one verdict on 'fasting'; Treats conflicting results as an inconsistency judgment with a stated reason, not as 'no evidence'; Avoids the words 'proven', 'conclusive' and 'no evidence' in the certainty statement.",
"files": [],
"expectations": [
"Frames a PICO and rates outcomes separately rather than issuing one verdict on 'fasting'",
"Treats conflicting results as an inconsistency judgment with a stated reason, not as 'no evidence'",
"Avoids the words 'proven', 'conclusive' and 'no evidence' in the certainty statement"
],
"skills": [
"evidence-appraisal"
]
},
{
"id": 3,
"case_id": "grade-pos-3",
"kind": "positive",
"prompt": "There are only observational cohort studies on this occupational exposure and cancer risk, but the relative risk is around 6 with a clear dose-response. Is this proven, or how certain should we be in the body of evidence?",
"expected_output": "A response that satisfies these checks: Starts the observational body at Low; Considers rating up for the very large effect and the dose-response gradient only after rating down, with the residual-confounding argument made explicit; States the result as a certainty level with reasons and a surveillance trigger.",
"files": [],
"expectations": [
"Starts the observational body at Low",
"Considers rating up for the very large effect and the dose-response gradient only after rating down, with the residual-confounding argument made explicit",
"States the result as a certainty level with reasons and a surveillance trigger"
],
"skills": [
"evidence-appraisal"
]
},
{
"id": 4,
"case_id": "grade-neg-1",
"kind": "negative",
"prompt": "Is this single randomized trial reliable? Walk me through its randomization, blinding, missing outcome data and selective reporting to assess bias in the study.",
"expected_output": "A response that satisfies these checks: Runs the RoB 2 per-study domains rather than a body-of-evidence certainty rating; Does not assign a GRADE level to a single study.",
"files": [],
"expectations": [
"Runs the RoB 2 per-study domains rather than a body-of-evidence certainty rating",
"Does not assign a GRADE level to a single study"
],
"skills": [
"assess-study-bias"
]
},
{
"id": 5,
"case_id": "grade-neg-2",
"kind": "negative",
"prompt": "How trustworthy is this trade-press article claiming a vendor's chip is 40% faster? Give me a source-reliability and information-credibility grade for the citation before I use it in the report.",
"expected_output": "A response that satisfies these checks: Grades the source on the Admiralty A1-F6 scale rather than applying GRADE to a non-study document; Does not construct a PICO or a summary-of-findings table.",
"files": [],
"expectations": [
"Grades the source on the Admiralty A1-F6 scale rather than applying GRADE to a non-study document",
"Does not construct a PICO or a summary-of-findings table"
],
"skills": [
"rate-source-admiralty"
]
}
]
}
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |