brycewang-stanford/Auto-Empirical-Research-Skills / skills/24-Imbad0202-academic-research-skills/deep-research
deep-research
VerifiedUniversal deep research agent team. 13-agent pipeline for rigorous academic research on any topic. 7 modes: full research, quick brief, paper review, lit-review, fact-check, Socratic guided research dialogue, and systematic review with optional meta-analysis. Covers research question formulation, Socratic mentoring, methodology design, systematic literature search, source verification, cross-source synthesis, risk of bias assessment, meta-analysis, APA 7.0 report compilation, editorial review, devil's advocate challenges, ethics review, and post-research literature monitoring. Triggers on: research, deep research, literature review, systematic review, meta-analysis, PRISMA, evidence synthesis, fact-check, guide my research, help me think through, 研究, 深度研究, 文獻回顧, 文獻探討, 系統性回顧, 後設分析, 事實查核, 引導我的研究, 幫我釐清, 幫我想想, 我不確定要研究什麼, 研究方向, 研究主題.
examples/policy_analysis.md
View on GitHub# Example: Policy Analysis (full mode)
## User Request
```
Research comparing outcomes-based funding models in higher education across OECD countries
```
## Phase 1: SCOPING
### research_question_agent Output
**Primary Research Question**:
> What are the design features, implementation strategies, and measured outcomes of performance-based funding (PBF) models in OECD higher education systems, and what contextual factors explain variation in effectiveness?
**FINER Assessment**:
| Criterion | Score | Justification |
|-----------|-------|---------------|
| Feasible | 4/5 | Substantial literature + OECD data available |
| Interesting | 4/5 | Active policy debate across multiple countries |
| Novel | 3/5 | Several reviews exist but few systematic cross-country comparisons |
| Ethical | 5/5 | Public policy analysis, no ethical concerns |
| Relevant | 5/5 | Directly informs national HE funding policy |
| **Average** | **4.2/5** | |
**Sub-questions**:
1. What PBF design features (metrics, weights, mechanisms) are used across OECD countries?
2. What outcomes (intended and unintended) have been documented?
3. What contextual factors (governance, culture, market structure) mediate PBF effectiveness?
### research_architect_agent Output
**Paradigm**: Pragmatist
**Method**: Comparative policy analysis with systematic evidence review
**Framework**: Modified Bardach's Eightfold Path + most-similar-systems design
**Data**: OECD reports, national policy documents, academic literature
**Analysis**: Cross-case comparison matrix + thematic synthesis
### devils_advocate_agent — CHECKPOINT 1
**Verdict**: PASS
- Minor: "Effectiveness" needs operational definition — suggest specifying measurable dimensions
- Observation: Country selection will significantly affect conclusions — justify selection criteria
## Phase 2: INVESTIGATION
### bibliography_agent Output
22 academic articles + 8 OECD/government reports + 3 think tank publications = 33 sources
Key sources include:
- OECD (2019). *Benchmarking Higher Education System Performance*
- Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015). Evaluating the impacts of "new" performance funding in higher education. *Educational Evaluation and Policy Analysis*, *37*(4).
- de Boer, H., et al. (2015). *Performance-based funding and performance agreements in fourteen higher education systems*. CHEPS.
### source_verification_agent Output
33 sources assessed: 28 Grade A-B, 4 Grade C, 1 Grade D (included with caveat)
## Phase 3: ANALYSIS
### synthesis_agent Output
**Comparative Matrix** (8 countries x 6 design features x 4 outcome dimensions)
Countries analyzed: US (Tennessee, Ohio), Australia, Denmark, Netherlands, Finland, UK (England), South Korea, Chile
**Key Findings**:
1. **Design convergence**: Most PBF models use graduation rates and retention as primary metrics, but weights and mechanisms vary dramatically. Tennessee allocates 85% of state funding based on outcomes (the highest proportion globally), while most European models tie less than 5% to performance. Despite this range, all models converge on a similar set of 4-6 core metrics: completion, retention, research output, and employment.
2. **Modest positive effects on targeted metrics**: Graduation rates show small positive effects (2-5 percentage points) in most contexts where PBF has been implemented for more than 3 years. Tennessee's model, the most studied, shows a 3.2 percentage point increase in 6-year graduation rates after controlling for student demographics (Hillman et al., 2015). However, several studies note difficulty in attributing causality given concurrent policy changes.
3. **Significant unintended consequences**: Grade inflation, cream-skimming, narrowing of institutional mission documented in multiple countries. In Australia, the demand-driven system (2012-2017) led to rapid enrollment growth but declining completion rates at some institutions. In the US, evidence from Indiana suggests institutions may have shifted enrollment toward better-prepared students rather than improving outcomes for existing students (Umbricht et al., 2017).
4. **Context is king**: Governance structure (centralized vs. decentralized) is the strongest moderator of PBF effectiveness. Centralized systems (Denmark, Finland, Netherlands) show more consistent implementation and fewer gaming behaviors than decentralized systems (US states, Australia). The mechanism appears to be: centralized systems can enforce compliance and adjust metrics more rapidly, while decentralized systems create more opportunities for strategic behavior.
5. **Equity concerns**: Institutions serving disadvantaged students consistently report greater difficulty meeting PBF targets. This is documented across all 8 countries studied, though the magnitude varies. Tennessee partially addresses this through "premium" weights for Pell-eligible students and adult learners, while most European models lack explicit equity adjustments.
**Major Contradiction**: US state-level studies show mixed results (Tennessee positive, Ohio neutral, Indiana negative), while European models show more consistent modest effects — likely explained by funding proportion (US: 5-25% of budget; Europe: often < 5%).
### devils_advocate_agent — CHECKPOINT 2
**Verdict**: PASS
- Major (downgraded from Critical): Selection of 8 countries needs stronger justification — why not Germany, Japan, Canada?
- Minor: "Context is king" is an assertion — specify which contextual variables and their mechanisms
## Phase 4: COMPOSITION
### report_compiler_agent Output
**Title**: Performance-Based Funding in OECD Higher Education: A Comparative Analysis of Design, Outcomes, and Contextual Mediators
**Word Count**: 6,800 words
**Structure**:
1. **Introduction** (600 words): PBF as global policy trend; rationale for comparative analysis
2. **Literature Review** (1,200 words): Evolution of PBF models from 1970s to present; theoretical foundations (principal-agent theory, new public management, institutional theory)
3. **Methodology** (800 words): Comparative policy analysis framework; country selection criteria (most-similar-systems with variation on key variables); data sources and analytical approach
4. **Findings**:
- 4.1 Design Feature Comparison (1,200 words): Cross-case matrix of 8 countries across 6 design dimensions (metrics, weights, mechanism type, funding proportion, adjustment frequency, equity provisions)
- 4.2 Outcome Evidence (800 words): Intended effects on graduation, retention, research output
- 4.3 Unintended Consequences (600 words): Gaming, cream-skimming, mission drift, administrative burden
- 4.4 Contextual Mediators (600 words): Governance, market structure, institutional culture, data infrastructure
5. **Discussion** (600 words): Implications for policy design; transferability considerations
6. **Recommendations** (400 words): 5 evidence-based design principles
- R1: Keep performance-linked proportion below 10% initially
- R2: Include explicit equity adjustments from the start
- R3: Use improvement metrics alongside absolute metrics
- R4: Build in 3-year review cycles with stakeholder input
- R5: Invest in institutional data capacity before implementation
7. **Limitations & Future Research** (300 words)
8. **References** (33 sources in APA 7.0)
## Phase 5: REVIEW
### editor_in_chief_agent Output
**Verdict**: Accept with Minor Revision (Score: 4.1/5.0)
- Strength: Well-structured comparative framework; balanced treatment of evidence
- Minor: Table 2 (design features) needs a notes column to explain country-specific nuances
- Minor: Recommendation #5 (data capacity) needs supporting evidence — currently assertion-based
### ethics_review_agent Output
**Verdict**: CLEARED
- AI disclosure: Present and accurate
- References spot-checked: 7/33 (21%) verified — all confirmed
- Dual-use risk: Low (public policy analysis)
- Fair representation: Adequate — noted absence of non-OECD perspectives as limitation
- Potential bias: Report slightly favors European models over US models; flagged for author awareness
### devils_advocate_agent — CHECKPOINT 3
**Verdict**: PASS
- Observation: Conclusion recommendation #3 ("use improvement metrics alongside absolute metrics") is the strongest recommendation but has the weakest evidence base — only 2 countries have implemented this
- "So what?" test: Passed — clear implications for policymakers designing or reforming PBF systems
- Counterfactual check: What if PBF effects are entirely explained by the Hawthorne effect (attention to metrics, not funding mechanism)? This alternative explanation is not adequately addressed
## Phase 6: REVISION
### report_compiler_agent (Revision 1)
**Changes**:
1. Table 2 expanded with notes column explaining country-specific nuances (Minor, Editor)
2. Recommendation #5 supported with evidence from Finnish data infrastructure investment (Minor, Editor)
3. Discussion section expanded to address Hawthorne effect alternative explanation (Observation, Devil's Advocate)
4. Country selection justification strengthened in Methodology — added explicit criteria and acknowledged Germany, Japan, Canada as excluded cases with rationale (Major, Devil's Advocate from Checkpoint 2)
**Final Word Count**: 6,800 words
**Revision Loops Used**: 1 of 2
---
## Final Output Summary
- Full APA 7.0 report: 6,800 words
- 33 cited sources (Levels I-VII)
- 8-country comparative matrix
- 5 key findings
- 5 evidence-based policy recommendations
- Ethics cleared
- 1 revision loop completed
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |