Skip to content

DAAF-Contribution-Community/daaf / .claude/skills/daaf-orchestrator

daaf-orchestrator

Verified

Operational framework for the DAAF orchestrator. Defines engagement modes, confirmation protocol, subagent dispatch, context budget, and reference-loading. Loaded exclusively by the orchestrator — not for subagents or user questions.

references/full-pipeline-mode.md

View on GitHub
# Full Pipeline Mode

This reference is loaded after the orchestrator classifies a request as Full Pipeline mode and the user confirms. It contains the complete 5-phase, 12-stage workflow, subagent coordination patterns, quality framework, and session management protocol.

**Path variables** (defined in core SKILL.md):
- **`{BASE_DIR}`** = project root (where `CLAUDE.md` resides)
- **`{SKILL_REFS}`** = `{BASE_DIR}/.claude/skills/daaf-orchestrator/references`

> **Domain Extensibility:** This workflow is domain-agnostic. Skill names referenced below (e.g., `education-data-explorer`, `education-data-query`, `education-data-context`) are the demonstration domain defaults. The orchestrator resolves actual skill names from the confirmed pre-flight scope during Stages 2-3 (before a Plan exists) and from Plan.md's Domain Configuration section afterwards, and provides them in Agent prompts. New domains can be added by authoring domain-specific Skills and registering them in Plan.md's Domain Configuration.

> **Invocation Pattern Authority:** Two layers:
> 1. **`WORKFLOW_PHASE*.md`** — stage-specific invocation templates with full context fields, thoroughness directives, gate criteria, verification checklists, and PSU content. **Start here** when constructing a subagent prompt.
> 2. **This file (`full-pipeline-mode.md`)** — cross-phase templates (code-reviewer invocation, revision requests), generic prompt structure, context inlining protocol, and QA enforcement.
>
> **`.claude/agents/README.md`** provides agent behavioral specs and input/output contracts — consult when understanding an agent's capabilities, not for constructing invocation prompts.

> **Parallel Dispatch Guideline:** Follow the default limit in `SKILL.md` § Subagent Coordination > "Parallel Dispatch Guideline": at most **5 subagents in flight at once** unless the user explicitly approves a larger batch. In this pipeline it governs wave-based task dispatch, Stage 3 source-researcher dispatch, code-reviewer invocations, and any other parallel invocation: when more than 5 independent tasks are ready, sub-batch into groups of ≤5. Parallel dispatch is achieved by making multiple Agent tool calls in a **single response message**. See `SKILL.md` § Subagent Coordination > Wave Barrier Discipline (Async Dispatch) for how results arrive. Wait for every subagent in a sub-batch to return before dispatching the next, and for the whole set before acting on the results, treating mid-wave notifications as status-only (no gate decisions, plan changes, or synthesis until the whole wave is in). See "Wave Barrier Discipline" below.

---

## Scope Check

**Tier:** Hard (blocking). **Where:** the Pre-Flight Checklist below is Full Pipeline's Scope Check; it embeds the standard block from `SKILL.md` § Scope Check and keeps its own deliverables and proposed-scope fields. **Read first:** everything the user referenced (papers, prior analyses, documents, URLs, data descriptions), in the turn after the mode is confirmed and before the pre-flight is presented, following `SKILL.md` § Scope Check > Read First (the rules for staging, PDFs and data files). No project folder exists yet, so fetches, PDF ingests and any digest are staged; they move into the project folder when the user confirms the pre-flight (§ Pre-Flight Checklist > On Confirmation (Gate G1)). **Nothing referenced:** the pre-flight's estimates are labelled preliminary. **On resume:** see "Session recovery" under the Pre-Flight Checklist.

---

## Pre-Flight Checklist

> **Template note:** This section is Full Pipeline's **Scope Check** (hard tier; `SKILL.md` § Scope Check) and also holds the mode's user orientation, consolidated into a single pre-flight message. Other modes define `## User Orientation` and `## Scope Check` as separate sections per `MODE_TEMPLATE.md`. The orientation overview follows the same on-request rule as every mode's User Orientation, so by default the pre-flight carries only the scope check.

After mode confirmation, read what the user referenced, then present the pre-flight checklist before beginning Phase 1 work. The user's confirmation of the pre-flight is **Gate G1**: the pre-flight Scope Check is confirmed (which includes mode confirmation). The mode confirmation previewed the process; the pre-flight confirms the deliverables and a scope grounded in those materials, so a document that reshapes the question does so before any discovery work runs on a guess.

**Read the referenced materials first,** following `SKILL.md` § Scope Check > Read First, the single home of the staging, PDF and data-file rules: local documents and code in full; web pages with `web_fetch.sh` (`web-retrieval` skill); every referenced PDF, and any large or many materials, through **one** `search-agent` digest dispatch, then wait for it to return; a referenced data file only from what the user said and its listing. Do not create the project folder in this turn: it is created when the user confirms the pre-flight (§ On Confirmation (Gate G1) below). Until then the fetches, PDF ingests and the digest record are staged in the session scratchpad's `scope_staging/` folder, and nothing durable cites a staged path. Keep the list of materials for the pre-flight's "What I read" line.

**Orientation overview** — The mode confirmation already gave the user this preview, so do not repeat it after confirmation. Present these points only when the user signals they are new to DAAF or asks for more detail.
- 5 phases: Discovery → Planning → Data Acquisition → Analysis → Synthesis
- Checkpoint after each of the first 4 phases (data sources? plan? data quality? results?)
- Planning checkpoint is the most important review — approves methodology before code runs
- Nothing moves forward without user go-ahead
- Session progress is saved if conversation gets long

**Scope Check** — present:

```
**Full Pipeline Analysis: Pre-Flight Scope Check**

**What I read:** [each material you pointed to, one line on what in it shapes the scope] (or "Nothing beyond your request — the estimates below are preliminary")

This analysis will create:
- [ ] Research Plan documents (Plan.md + Plan_Tasks.md) summarizing all key goals, considerations, decisions, risks, interpretations, work stage summaries, and final work review notes
- [ ] STATE.md session state file (for progress tracking and session recovery)
- [ ] Comprehensive analytic scripts (.py for Python, .R for R) covering data fetch, clean, join, transformation and analysis, plus the quality review of each script
- [ ] Validated datasets (raw + processed)
- [ ] Marimo notebook (Python) or Quarto notebook (R) "walkthrough" of successfully completed analysis scripts and their execution runtime logs for inspection
- [ ] Illustrative key data visualizations
- [ ] Summary stakeholder report synthesizing key findings and interpreting key data visualizations
- [ ] LEARNINGS.md, notes on lessons learned that can help improve DAAF for future projects

**Proposed scope**[ (preliminary) — when nothing beyond your request was read]:
- Research question: [one line]
- Data sources: [candidate sources]
- Years: [year range]
- Approximate records: [estimate]
- Geographic scope: [geography]

**What changed from your original request:** [differences the materials introduced — e.g., "the paper you sent defines the outcome as 4-year completion, not enrollment" — or "Nothing — your request and the materials agree"]

**Decisions for you:** [each independent decision on its own line, with its default — or "None"]

**Please confirm whether you'd like me to begin with this approach, or let me know if you have any changes you'd like to make.**
```

Scope adjustments the user made at mode confirmation ("yes, but just 2015-2020") go into "Proposed scope" (`SKILL.md` § Mode Confirmation Gate).

**Session recovery:** the record of a confirmed pre-flight is the project folder's `output/preliminary_notes/{date}_preflight_scope-check.md`, written when the user confirms (§ On Confirmation (Gate G1), step 3). When it exists, do not re-present the pre-flight: with a STATE.md (created at Stage 4), resume at its recorded position (`session-recovery.md`); without one, read the record for the confirmed scope and materials and resume Phase 1 at the first stage whose notes are missing. No project folder, or a folder without that record (a confirmation interrupted before step 3, or the `/tmp` fallback in `SKILL.md` § Scope Check > Read First, where the folder is created before the pre-flight), means the pre-flight was not confirmed: re-present it, without the orientation overview, reusing anything already moved into the folder. The earlier session's staged materials may survive in its scratchpad, but nothing durable recorded them, so re-read or re-fetch them as needed.

**For more detail:** Consult `{BASE_DIR}/user_reference/02_understanding_daaf.md`.

**User may:**
- Confirm → § On Confirmation (Gate G1), then Phase 1 (Stage 2: Data Exploration)
- Request scope adjustment → Clarify and reconfirm
- Decline → Switch to Data Discovery or Data Lookup mode

#### Pre-Flight Turn Boundary Rule

The pre-flight comes in the turn after the user confirms the mode. Before you present it, that turn may:
- read this mode file (`full-pipeline-mode.md`)
- read the local materials the user referenced (`SKILL.md` § Scope Check > Read First)
- stage `web_fetch.sh` fetches of the web pages they referenced in the scratchpad's `scope_staging/` folder
- copy a referenced PDF that sits outside `/daaf` into `scope_staging/user_pdfs/` for the digest (one `cp` per call)
- dispatch **one** `search-agent` to digest the materials (always for a PDF; also when they are large or many), writing to the staging folder and citing staged artifacts by paths relative to the staging root (`SKILL.md` § Scope Check > Read First), and wait for it to return

Until the user confirms the pre-flight:
- Do NOT create the project folder (§ On Confirmation (Gate G1) creates it; the one exception is the `/tmp` fallback in `SKILL.md` § Scope Check > Read First)
- Do NOT load phase-specific workflow files (no `Read` of `WORKFLOW_PHASE1_DISCOVERY.md`, etc.)
- Do NOT dispatch any other subagent (no Stage 2 or Stage 3 dispatches)
- Do NOT begin Stage 2 exploration work
- Do NOT download datasets or write analysis scripts
- Do NOT read any other reference files

Why the reads are allowed: the scope the user approves has to rest on what they pointed to. Why the rest is not: discovery stays behind the user's approval, so a wrong scope costs one read rather than a discovery wave, and a declined scope leaves no project folder behind.

The pre-flight message ends the turn: nothing follows it in that response. It is a STOPPING POINT; wait for user confirmation before proceeding. Phase-specific workflow files are loaded *after* the user confirms the pre-flight, in a subsequent turn.

#### Pre-Flight Self-Check

Before sending your pre-flight response, verify:
- [ ] The materials the user referenced were read (or digested by one search-agent), or none were referenced
- [ ] "What I read" line present (or "Nothing beyond your request — the estimates below are preliminary")
- [ ] Deliverables list presented
- [ ] Proposed scope filled in, marked preliminary when nothing beyond the request was read
- [ ] "What changed from your original request" line present
- [ ] "Decisions for you" listed, each with its default (or "None")
- [ ] Orientation overview omitted, unless the user signaled they are new to DAAF or asked for more detail
- [ ] Message ends with an explicit question to the user
- [ ] No phase workflow files loaded, no Stage 2 or Stage 3 work begun, no datasets downloaded
- [ ] At most one subagent dispatched (the materials digest)
- [ ] No project folder created; fetches, PDF ingests and any digest are in the staging folder, and the message names materials by title or URL, never by staged path
- [ ] Nothing follows the pre-flight message in this turn

#### On Confirmation (Gate G1)

When the user confirms the pre-flight, Gate G1 is met. In the next turn, before loading `WORKFLOW_PHASE1_DISCOVERY.md` or dispatching Stage 2:

1. **Create the project folder** `research/YYYY-MM-DD_[Title]/` with this layout (one `mkdir -p` per call). Phase 1 writes its notes into it, and Stage 4 reuses it:
   ```
   research/YYYY-MM-DD_[Title]/
   ├── data/
   │   ├── raw/
   │   └── processed/
   └── output/
       ├── analysis/
       ├── figures/
       └── preliminary_notes/
   ```
2. **Move the staged materials in** (one `mv` per call, whole folders, before anything cites them; `SKILL.md` § Scope Check > Read First > "Graduation on confirmation"): `{scratchpad}/scope_staging/web_fetches/` → `{PROJECT_DIR}/web_fetches/`; `papers/` → `{PROJECT_DIR}/papers/` (and `user_pdfs/` → `{PROJECT_DIR}/user_pdfs/`); the digest `{date}_{slug}_materials-digest.md` → `{PROJECT_DIR}/output/preliminary_notes/`. Manifest rows written before the move (e.g. `web_fetches/MANIFEST.jsonl`) are historical records: never rewrite them (`agent_reference/SCRIPT_EXECUTION_REFERENCE.md` § Scratch Graduation Check, step 3). Then re-ingest each staged paper whose source PDF moved with it, with `--reingest`, as that section's graduation step describes.
3. **Record the confirmed pre-flight** at `{PROJECT_DIR}/output/preliminary_notes/{date}_preflight_scope-check.md`: the Scope Check as the user confirmed it (with their adjustments and answers to the decisions), the materials list with their paths in the project, and one dated old → new line listing the moves (or "nothing was staged"). STATE.md does not exist until Stage 4; when it is created, this record is copied into its Pre-Flight Scope Check section (`agent_reference/STATE_TEMPLATE.md`).
4. **Hand it on:** the materials' project paths fill the Stage 2 and Stage 3 `USER-REFERENCED MATERIALS:` fields, and the confirmed scope and materials fill the Stage 4 data-planner's `CONFIRMED PRE-FLIGHT SCOPE:` and `USER-REFERENCED MATERIALS:` fields (`agent_reference/WORKFLOW_PHASE2_PLANNING.md`).

Why the folder waits for this moment: before it, the user has approved nothing that warrants a project, and a scope they decline leaves nothing in `research/`. It also means the pre-flight record in the folder marks a confirmed pre-flight, which session recovery relies on.

---

## Critical Warning: Custom Planning Workflow

> **DO NOT use Claude Code's built-in `EnterPlanMode` tool for this workflow.**
>
> This research system has its own planning protocol that is DIFFERENT from Claude Code's native Plan Mode:
>
> | Aspect | Claude Code Plan Mode | This Workflow |
> |--------|----------------------|---------------|
> | **Plan Creation** | `EnterPlanMode` tool | Stage 4 + `data-planner` agent + `PLAN_TEMPLATE.md` |
> | **Validation** | User clicks "approve" | Stage 4.5 + `plan-checker` agent (automated) |
> | **Gate** | `ExitPlanMode` | Gate G4.5 (plan-checker PASSED + user confirmed PSU2) |
>
> **Why this matters:** The built-in Plan Mode has different semantics and will bypass the plan-checker validation gate (G4.5). Always use the custom workflow defined in this file.

---

## Stage Overview

| Stage | Phase | Name | Primary Skill/Agent | Dispatched agent (`subagent_type`) |
|-------|-------|------|---------------------|----------|
| 1 | 1 | Initial Intake | — | Orchestrator |
| 2 | 1 | Data Exploration | Domain explorer skill (e.g., `education-data-explorer`) | search-agent |
| 3 | 1 | Source Deep-Dive | `*-data-source-*` | source-researcher |
| **3.5** | 1 | Findings Synthesis | `research-synthesizer` agent | research-synthesizer |
| 4 | 2 | Plan Creation | `data-planner` agent | data-planner |
| **4.5** | 2 | Plan Validation | `plan-checker` agent | plan-checker |
| 5 | 3 | Data Retrieval | Domain query skill (e.g., `education-data-query`) | research-executor, then code-reviewer per script |
| 6 | 3 | Context Application | Domain context skill (e.g., `education-data-context`) | research-executor, then code-reviewer per script |
| 7 | 4 | EDA & Transformation | `data-scientist`, `polars` (Python) or `tidyverse` (R) | research-executor, then code-reviewer per script |
| 8 | 4 | Analysis & Visualization | `data-scientist`, `polars`/`tidyverse`, modeling library (Python: `statsmodels`/`pyfixest`/`linearmodels`/`svy`/`scikit-learn`/`igraph`; R: `r-stats`/`fixest`/`plm`/`survey-r`/`tidymodels`/`igraph-r` per Plan), `plotnine`/`plotly` (Python) or `ggplot2`/`plotly-r` (R), `geopandas` (Python) or `sf-terra` (R) if spatial, `igraph` (Python) or `igraph-r` (R) if network | research-executor, then code-reviewer per script |
| 9 | 4 | Notebook Assembly | `marimo` (Python) or `quarto` (R) | notebook-assembler, then integration-checker |
| 10 | 4 | QA Aggregation | — (orchestrator) | — |
| 11 | 5 | Report Generation | `report-writer` agent | report-writer, then integration-checker |
| 12 | 5 | Final Review | `data-verifier` agent (adversarial verification with cross-artifact coherence) | integration-checker (full scope), then data-verifier |

> Quick reference for prompt construction. See "Core Workflow Overview" below for execution details and gate criteria.

---

## Core Workflow Overview

The Full Pipeline workflow consists of **5 Phases** and **12 Stages**.

```
┌─────────────────────────────────────────────────────────────────────────────┐
│ PHASE 1: DISCOVERY & SCOPING                                                │
├─────────────────────────────────────────────────────────────────────────────┤
│  Stage 1: Initial Intake                                                    │
│      ├─ Classify engagement mode                                            │
│      ├─ Ask clarifying questions (if needed)                                │
│      └─ Output: Research question + mode confirmed                          │
│                          ↓                                                  │
│  ┌─ PRE-FLIGHT CHECKLIST (see § Pre-Flight Checklist above) ────────────┐   │
│  │  □ Read the materials the user referenced (staged; one digest)       │   │
│  │  □ Scope Check: what I read, deliverables, scope, what changed       │   │
│  │  □ Estimates preliminary when nothing beyond the request was read    │   │
│  │  □ User confirms or adjusts before proceeding                        │   │
│  │  ★ STOP — Pre-Flight Turn Boundary Rule applies                      │   │
│  └──────────────────────────────────────────────────────────────────────┘   │
│      Gate G1: pre-flight Scope Check confirmed (includes mode confirmation) │
│      On confirmation: create project folder, move staged materials in       │
│                          ↓                                                  │
│  Stage 2: Data Exploration ←── domain explorer skill                        │
│      ├─ Identify candidate datasets and variables                          │
│      ├─ Report findings to user (adaptive)                                  │
│      └─ Gate G2: ≥1 dataset identified, key variables flagged               │
│                          ↓                                                  │
│  Stage 3: Source Deep-Dive ←── domain source skills                         │
│      ├─ Understand limitations, caveats, suppression patterns               │
│      ├─ Document source-specific gotchas                                    │
│      └─ Gate G3: Coded values documented, suppression patterns identified   │
│                          ↓                                                  │
│  Stage 3.5: Findings Synthesis ←── research-synthesizer agent               │
│      ├─ Consolidate parallel Stage 2-3 findings                             │
│      ├─ Resolve cross-source conflicts                                      │
│      └─ Gate G3.5: Synthesis complete, unified guidance for Plan.md and Plan_Tasks.md│
└─────────────────────────────────────────────────────────────────────────────┘
                          ↓
            ┌─────────────────────────────────┐
            │  ★ PSU1: Phase Status Update 1  │
            │  Present findings, await user   │
            │  confirmation before planning   │
            └─────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ PHASE 2: PLANNING                                                           │
├─────────────────────────────────────────────────────────────────────────────┤
│  Stage 4: Plan Document Creation                                            │
│      ├─ Synthesize Phase 1 findings                                         │
│      ├─ Document methodology decisions                                      │
│      ├─ Create Plan.md + Plan_Tasks.md (project folder exists since G1)     │
│      ├─ **CRITICAL:** Complete Transformation Sequence table                │
│      ├─ Create STATE.md with Plan Validation section (initially NOT_RUN)    │
│      ├─ Create LEARNINGS.md skeleton (project metadata + empty sections)    │
│      ├─ **WARNING:** DO NOT use Claude Code's EnterPlanMode tool here!      │
│      │   Use data-planner agent + PLAN_TEMPLATE.md instead.                 │
│      ├─ Tell the user: "The plan is drafted; a specialist is now checking   │
│      │   it for gaps before you review it."                                 │
│      └─ Gate G4: Plan.md + Plan_Tasks.md + STATE.md + LEARNINGS.md created  │
│                          ↓                                                  │
│  Stage 4.5: Plan Validation ←── plan-checker agent                          │
│      ├─ Automated 6-dimension plan validation                               │
│      └─ Gate G4.5: plan-checker passes, then user confirms PSU2             │
│                                                                             │
│  **Transformation Sequence:** This table (in Plan.md) with task blocks     │
│  (in Plan_Tasks.md) serves as the contract between orchestrator and        │
│  subagents during Stages 5-8. Each task becomes a separate subagent        │
│  invocation. Incomplete sequences lead to incomplete validation and        │
│  unreliable results.                                                       │
└─────────────────────────────────────────────────────────────────────────────┘
                          ↓
            ┌─────────────────────────────────┐
            │  ★ PSU2: Phase Status Update 2  │
            │  Present Plan.md SUMMARY for    │
            │  review; user reads full Plan.md│
            │  before confirming              │
            └─────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ PHASE 3: DATA ACQUISITION & PREPARATION                                     │
├─────────────────────────────────────────────────────────────────────────────┤
│  ┌─ STAGE 5 PRE-FLIGHT CHECK (MANDATORY) ─────────────────────────────────┐ │
│  │  Before executing ANY Stage 5 task, verify:                            │ │
│  │  □ Plan.md exists at expected path                                      │ │
│  │  □ Plan_Tasks.md exists at expected path                               │ │
│  │  □ STATE.md exists                                                     │ │
│  │  □ STATE.md "Plan Validation" section shows:                           │ │
│  │    - Plan-Checker Status: PASSED or PASSED_WITH_WARNINGS               │ │
│  │    - Gate G4.5 Status: SATISFIED                                       │ │
│  │  □ If Plan-Checker Status is NOT_RUN → STOP, invoke plan-checker first │ │
│  └────────────────────────────────────────────────────────────────────────┘ │
│                                                                             │
│  Stage 5: Data Retrieval ←── domain query skill                             │
│      ├─ Download from configured mirrors (per mirrors.yaml)                 │
│      ├─ Auto-validate: shape, columns, years, missingness (CP1)             │
│      ├─ STOP if: unexpected empty results, data access errors               │
│      ├─ [Per-script QA loop — see Composite Execution Pattern below]        │
│      └─ Gate G5: CP1 PASSED, QA1 PASSED/WARNING, data in data/raw/         │
│                          ↓                                                  │
│  Stage 6: Context Application ←── domain context skill                      │
│      ├─ Assess missingness and coded value presence                         │
│      ├─ Calculate suppression rates (CP2)                                   │
│      ├─ STOP if: >50% suppression (CP2), Plan governance rule broken        │
│      ├─ [Per-script QA loop — see Composite Execution Pattern below]        │
│      └─ Gate G6: CP2 PASSED, QA2 PASSED/WARNING, data in processed/        │
└─────────────────────────────────────────────────────────────────────────────┘
                          ↓
            ┌─────────────────────────────────┐
            │  ★ PSU3: Phase Status Update 3  │
            │  Present data quality metrics,  │
            │  await confirmation before      │
            │  analysis                       │
            └─────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ PHASE 4: ANALYSIS & NOTEBOOK DEVELOPMENT                                    │
├─────────────────────────────────────────────────────────────────────────────┤
│  Stage 7: EDA & Transformation ←── data-scientist + polars/tidyverse skills │
│      ├─ Initial data profiling (auto-execute)                               │
│      ├─ Report key findings to user (adaptive)                              │
│      ├─ Transformations with validation (CP3 per transformation)            │
│      ├─ [Per-script QA loop — see Composite Execution Pattern below]        │
│      └─ Gate G7: All CP3 PASSED, all QA3 PASSED/WARNING                     │
│                          ↓                                                  │
│  Stage 8: Analysis & Visualization ←── modeling + viz skills                │
│      ├─ 8.1: Run statistical analyses (save to output/analysis/)            │
│      │   ├─ Load modeling skill per Plan (see Modeling library selection)   │
│      │   └─ [Per-script QA loop (QA4a) — see Composite Execution Pattern]   │
│      ├─ 8.2: Generate exploratory and final plots (save to output/figures/) │
│      │   ├─ Load viz skill: plotnine/plotly (Python) or ggplot2/plotly-r (R),│
│      │   │   or geopandas (Python) / sf-terra (R) for maps                  │
│      │   └─ [Per-script QA loop (QA4b) — see Composite Execution Pattern]   │
│      └─ Gate G8: Analyses + viz complete, CP4 PASSED per script,            │
│         all QA4a AND QA4b PASSED/WARNING                                    │
│                          ↓                                                  │
│  Stage 9: Script Compilation ←── notebook-assembler agent                   │
│      ├─ LITERALLY COPY scripts into canonical three-cell                    │
│      │   header/source/log archive bundles (Python), with optional display, │
│      │   or canonical Quarto archive sections (R)                           │
│      ├─ VERBATIM execution logs in format-specific collapsed containers     │
│      ├─ NO new analysis; only optional format-specific output inspection    │
│      ├─ Quarto render validates assembly/previews, not archived re-execution│
│      ├─ NO dashboards, NO widgets, NO filters, NO aggregations              │
│      └─ Gate G9: `marimo check` gate + `marimo export html` + 6c            │
│         fidelity check (Python) or `quarto render` (R) passes; all          │
│         scripts represented; integration-checker (Stage 9 scope)            │
│         CONNECTED or WARNING-only                                           │
│                          ↓                                                  │
│  Stage 10: QA Aggregation                                                   │
│      ├─ **Aggregate QA findings from Stages 5-8 (WARNINGs reviewed)**       │
│      ├─ Review accumulated WARNINGs, confirm no unresolved BLOCKERs         │
│      ├─ STOP if: unresolved BLOCKERs or systemic WARNING patterns           │
│      └─ Gate G10: QA aggregated, BLOCKERs resolved, WARNINGs documented     │
└─────────────────────────────────────────────────────────────────────────────┘
                          ↓
            ┌─────────────────────────────────┐
            │  ★ PSU4: Phase Status Update 4  │
            │  Present analysis results and   │
            │  QA aggregation, await user     │
            │  confirmation before synthesis  │
            └─────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────────────────────────┐
│ PHASE 5: SYNTHESIS & DELIVERY                                               │
├─────────────────────────────────────────────────────────────────────────────┤
│  Pre-Report: Collect session logs into project                              │
│      └─ bash {BASE_DIR}/scripts/collect_session_logs.sh {PROJECT_DIR}       │
│                          ↓                                                  │
│  Stage 11: Report Generation ←── report-writer agent                        │
│      ├─ Synthesize Plan.md, Notebook, STATE.md, LEARNINGS.md, QA            │
│      ├─ Follow Section-Source Mapping for each REPORT_TEMPLATE.md section   │
│      ├─ Cross-check Research Outcomes against Key Findings                  │
│      └─ Gate G11: Report complete with all sections + figure references     │
│         + integration-checker (Stage 11 scope) CONNECTED or WARNING-only    │
│                          ↓                                                  │
│  Stage 12: Final Review                                                     │
│      ├─ Verify alignment with original request                              │
│      ├─ Check all Plan.md commitments fulfilled                             │
│      ├─ Document any deviations                                             │
│      ├─ Update STATE.md with Final Review Log                               │
│      ├─ **Consolidate LEARNINGS.md (review incremental entries, fill gaps)**│
│      ├─ **Generate System Update Action Plan section in LEARNINGS.md**      │
│      └─ Gate G12: Final review passed, all commitments fulfilled            │
│         + integration-checker (full scope, before data-verifier)            │
│         CONNECTED or WARNING-only                                           │
│                          ↓                                                  │
│  DELIVERY: Summary to user with file paths                                  │
│      + Learnings summary (key insights + action plan item count)            │
│      + If action plan has items: suggest Framework Development mode          │
└─────────────────────────────────────────────────────────────────────────────┘
```

> **AUTHORITATIVE EXECUTION LOOP:** The per-script QA loop referenced in each stage above is defined in full detail in the **Stage 5-8 Composite Execution Pattern** section below. That pattern is the MANDATORY atomic unit for all Stage 5-8 work. The workflow diagram above is a visual summary; the Composite Pattern is the binding specification.

---

## Stage 5-8 Per-Script Execution & QA Loop

**Every stage from 5 through 8 is executed as MULTIPLE subagent calls with interleaved QA, NOT as a single invocation per stage.** Each script in Plan.md's Transformation Sequence table is executed by research-executor, then **immediately and separately** reviewed by code-reviewer, before any script that depends on it begins. Same-wave scripts run in parallel (at most 5 in flight), each with its own review (§ Wave-Based Parallel Execution). This applies equally to Stage 5 (fetch scripts), Stage 6 (clean scripts), Stage 7 (transformation scripts), and Stage 8 (analysis and visualization scripts). Any Stage writing net new code must adhere to this. QA scripts are saved to `scripts/cr/stage{N}_{step}_cr{1..5}.py` for Stages 5-7; **Stage 8 uses the split-QA suffix** `scripts/cr/stage8_{step}_cra{1..5}.py` (QA4a, statistical validity) and `scripts/cr/stage8_{step}_crb{1..5}.py` (QA4b, visualization quality), per `agent_reference/QA_CHECKPOINTS.md`. The **Stage 5-8 Composite Execution Pattern** below defines the authoritative execution flow — it is the MANDATORY atomic unit for all Stage 5-8 work. See `.claude/agents/code-reviewer.md` for the complete QA protocol and `agent_reference/QA_CHECKPOINTS.md` for checkpoint definitions.

**Why this matters:**
- The core principle "Every transformation has a validation" requires separate execution cycles
- Each subagent call captures pre-state, executes ONE script, validates post-state
- QA must run immediately after each script so findings can inform whether to proceed, revise, or stop
- Batching QA to stage end means errors in script 1 propagate silently through every script that builds on it — compounding data corruption
- The Transformation Sequence table (Plan.md) and task blocks (Plan_Tasks.md) are the contract for these invocations

**See:** the appropriate `agent_reference/WORKFLOW_PHASE*.md` file for detailed per-script execution guidance.

### QA Invocation Responsibility

**Expectation for QA depth:** The code-reviewer is not a rubber-stamp. The reviewer should reason adversarially about the script, not merely run templated checks. A high-quality QA report includes reasoning about *why* the code is correct, not just confirmation that checks passed. If a code-reviewer returns PASSED with only template-level checks and no script-specific observations, consider whether the review was thorough enough.

The complete QA invocation workflow is defined in the **Stage 5-8 Composite Execution Pattern** below. See `.claude/agents/code-reviewer.md` for the QA protocol and `agent_reference/QA_CHECKPOINTS.md` for checkpoint definitions.

### Stage 5-8 Composite Execution Pattern (MANDATORY)

For EACH task in Stages 5-8, follow this complete loop. **Do NOT skip any step.**

```
┌─────────────────────────────────────────────────────────────────────────────┐
│  STEP 1: INVOKE research-executor                                           │
│      ├─ Use stage-specific template from the appropriate WORKFLOW_PHASE*.md  │
│      ├─ Capture from result: script_path, output_files, CP_status           │
│      └─ If CP_status == FAILED → Handle error, do not proceed to QA         │
├─────────────────────────────────────────────────────────────────────────────┤
│  STEP 2: INVOKE code-reviewer (MANDATORY - DO NOT SKIP)                     │
│      │                                                                      │
│      │   Use the code-reviewer invocation template from the                  │
│      │   "Code-Reviewer Invocation" section below, with stage-specific      │
│      │   values.                                                            │
│      │                                                                      │
│      │   **Review Expectation:** code-reviewer should perform adversarial   │
│      │   analysis, not just template validation. Expect the QA report to    │
│      │   include script-specific checks and reasoning about WHY the code    │
│      │   is correct, not merely confirmation that checks passed.            │
│      │                                                                      │
│      └─ WAIT for code-reviewer to return before proceeding                  │
├─────────────────────────────────────────────────────────────────────────────┤
│  STEP 3: EVALUATE QA severity                                               │
│      ├─ PASSED → Log to STATE.md, proceed to next task                      │
│      ├─ WARNING → Log to STATE.md (for Stage 10 review), proceed            │
│      ├─ CODE-REVIEWER STOP, Issue Category `Methodology Conflict`           │
│      │   (Plan.md's method looks wrong for the data, or Plan.md does not    │
│      │   settle the choice) → escalate to the user immediately (RULE 4;     │
│      │   no revision)                                                       │
│      ├─ CODE-REVIEWER STOP, any other Issue Category → route by category    │
│      │   (§ Gate Status Translation)                                        │
│      └─ Any other BLOCKER (the fix restores what Plan.md specifies, or      │
│          corrects a defect without changing the method) → Go to STEP 4      │
├─────────────────────────────────────────────────────────────────────────────┤
│  STEP 4: REVISION FLOW (BLOCKER that is not a Methodology Conflict)         │
│      ├─ Invoke research-executor to create revised script (_a.py/_a.R)      │
│      ├─ Re-invoke code-reviewer on revised script                           │
│      ├─ If still BLOCKER → Create _b.py/_b.R revision, re-invoke reviewer   │
│      └─ If still BLOCKER after 2 revisions → STOP and escalate to user      │
├─────────────────────────────────────────────────────────────────────────────┤
│  STEP 5: UPDATE STATE.md                                                    │
│      ├─ Update Transformation Progress table (per-script QA status)         │
│      ├─ Append QA findings to QA Findings Summary incrementally             │
│      │   (BLOCKERs Resolved, WARNINGs Logged — don't defer to Stage 10)    │
│      ├─ If analysis addresses a Plan.md hypothesis, update Hypothesis       │
│      │   Assessment Progress                                                │
│      └─ Task complete; the next wave waits for every task in this wave      │
├─────────────────────────────────────────────────────────────────────────────┤
│  STEP 0 (before each cycle): READ STATE.md                                  │
│      ├─ Verify current position and prior task statuses                     │
│      ├─ Check Error Budget — confirm remaining budget > 0                   │
│      └─ Confirm no unresolved BLOCKERs from prior tasks                    │
└─────────────────────────────────────────────────────────────────────────────┘
```

**CRITICAL:** Steps 0-5 form an atomic unit for each task. Step 0 runs before each new task cycle. Same-wave tasks run their loops in parallel, at most 5 in flight (§ Wave-Based Parallel Execution). NEVER start a task until every task it depends on has completed all steps. NEVER review several scripts in one code-reviewer call, and never defer reviews to the end of a stage.

How a BLOCKER is classed: A BLOCKER whose fix brings the code back to what the Plan already specifies (a MISALIGNED row under Methodology Alignment that a revision can correct, such as a LEFT join where the Plan says INNER) goes through revision (max 2). Any other BLOCKER (a bug, a stub, a validation gap) corrects a defect without changing the method and also goes through revision; a code-reviewer STOP of another Issue Category routes as § Gate Status Translation says. A methodology BLOCKER — the Plan's own methodology looks wrong for the data, or the Plan does not settle the choice — is a code-reviewer STOP with Issue Category `Methodology Conflict` and escalates to the user immediately. It skips revision because fixing it means changing the method, which only the user can approve (RULE 4; `agent_reference/ERROR_RECOVERY.md` § QA BLOCKER Recovery).

---

## Phase-to-Protocol Mapping

| Phase | Primary Protocol | Also Applies | PSU at Boundary | Reference |
|-------|------------------|--------------|-----------------|-----------|
| Phase 1 | Data Discovery | — | PSU1 (after Phase 1) | `agent_reference/WORKFLOW_PHASE1_DISCOVERY.md` |
| Phase 2 | Plan Management | — | PSU2 (after Phase 2) | `agent_reference/WORKFLOW_PHASE2_PLANNING.md` |
| Phase 3 | Data Acquisition | Validation (CP1-CP2) | PSU3 (after Phase 3) | `agent_reference/WORKFLOW_PHASE3_ACQUISITION.md` |
| Phase 4 | Validation Checkpoints | STATE.md Management (runtime updates) | PSU4 (after Phase 4) | `agent_reference/WORKFLOW_PHASE4_ANALYSIS.md` |
| Phase 5 | Synthesis & Delivery | Stages 11-12 | Final Review | `agent_reference/WORKFLOW_PHASE5_SYNTHESIS.md` |

Read each phase's file when that phase begins, not at mode start; do not load them all at once.

**Note:** Session Recovery is documented in `{SKILL_REFS}/session-recovery.md`.

**Cross-Phase Concerns:**
- **Validation Checkpoints** apply across Phases 3-5 with different checkpoints:
  - Phase 3: CP1 (after fetch), CP2 (after cleaning)
  - Phase 4: CP3 (after transformation), CP4 (after analysis & visualization, Stage 8)
  - Phase 5: Pre-Output Verification (Stage 12) — an agent checklist owned by data-verifier, not a script-embedded checkpoint
- **QA Checkpoints (QA1-QA4b)** run in parallel as secondary validation during Phases 3-4
- **Plan Document Maintenance** — Plan.md + Plan_Tasks.md are created and frozen in Phase 2; runtime state is tracked in STATE.md throughout Phases 3-5

## Skill-to-Stage Mapping

| Stage | Primary Skill(s) | Subagent Type | Invocation Pattern |
|-------|------------------|---------------|-------------------|
| 2 | `data-scientist`, domain explorer skill | search-agent | Subagent invokes skill |
| 3 | `data-scientist`, domain source skill(s) | source-researcher | Subagent invokes skill(s) |
| 3.5 | `data-scientist` | research-synthesizer | `research-synthesizer` agent |
| 4 | `data-scientist` | data-planner | `data-planner` agent |
| 4.5 | `data-scientist` | plan-checker | `plan-checker` agent |
| 5 | `data-scientist`, domain query skill | research-executor | Subagent invokes skill |
| **5-QA** | `data-scientist` | code-reviewer | `code-reviewer` agent (after each Stage 5 script) |
| 6 | `data-scientist`, domain context skill | research-executor | Subagent invokes skill |
| **6-QA** | `data-scientist` | code-reviewer | `code-reviewer` agent (after each Stage 6 script) |
| 7 | `data-scientist`, `polars` (Python) or `tidyverse` (R), `geopandas`/`sf-terra` (if spatial data) | research-executor | Subagent invokes skills |
| **7-QA** | `data-scientist` | code-reviewer | `code-reviewer` agent (after each Stage 7 script) |
| 8.1 | `data-scientist`, `polars`/`tidyverse`, modeling library per Plan (Python: `statsmodels` / `pyfixest` / `linearmodels` / `svy` / `scikit-learn` / `geopandas` / `igraph`; R: `r-stats` / `fixest` / `plm` / `survey-r` / `tidymodels` / `sf-terra` / `igraph-r`), `great-tables` (Python) or `gt` (R) (if formatted tables needed) | research-executor | Subagent invokes skills |
| 8.2 | `data-scientist`, `plotnine` or `plotly` (Python) or `ggplot2` or `plotly-r` (R), `geopandas` (Python) or `sf-terra` (R) if map visualization, `igraph` (Python) or `igraph-r` (R) if network visualization, `great-tables` (Python) or `gt` (R) (if formatted summary tables needed) | research-executor | Subagent invokes skills |
| **8-QA** | `data-scientist` | code-reviewer | `code-reviewer` agent (after each Stage 8 script) |
| 9 | `marimo` (Python) or `quarto` (R) | notebook-assembler | `notebook-assembler` agent (COMPILES scripts — NO new code, NO dashboards) |
| 10 | — | — | Orchestrator aggregates QA findings (no subagent) |
| 11 | `data-scientist`, `science-communication` (if non-technical audience) | report-writer | `report-writer` agent |
| 12 | `data-scientist` | data-verifier | `data-verifier` agent |

**Notes:**
- **Stage 4 responsibility split:** The `data-planner` agent creates Plan.md and Plan_Tasks.md. The **orchestrator** is responsible for creating STATE.md (from `agent_reference/STATE_TEMPLATE.md`) and the LEARNINGS.md skeleton (from `agent_reference/WORKFLOW_PHASE5_SYNTHESIS.md`) after the data-planner returns. Gate G4 requires all four files. **When creating STATE.md, populate the Session Metadata section:** run `git rev-parse --short HEAD` to capture the DAAF version, and record the Session Model ID, the Subagent Model Tiers in effect, the Provider Route and Model Settings, and the session start date; also copy in the confirmed pre-flight scope and materials — see the authoritative creation checklist under **Creation Trigger** in the STATE.md section below. These feed into the AI Use Disclosure section of the final report.
- **Stage 9 R/Quarto authority:** `.claude/skills/quarto/references/daaf-notebook.md` is the canonical DAAF assembly contract. It governs DAAF archive-shaped Quarto notebooks and their Reproducibility Verification anchors, not arbitrary Quarto documents. `quarto render` validates assembly and explicitly enabled previews; it does not re-run archived Stage 5-8 analysis scripts.
- **Stage 10** has no dedicated agent — the orchestrator performs QA aggregation directly by reviewing accumulated code-reviewer findings from Stages 5-8.

**Stage 10 Protocol:** Read STATE.md's Transformation Progress table as the input for Stages 5-8, plus the Stage 9 row of § Final Review Log > Integration Checks for any WARNING carried from Stage 9 (add it to the WARNING summary). For each script: (1) Check its QA Status: `PASSED`, `WARNING` and `REVISED` mean it was reviewed; a `NOT_RUN` or `PENDING` row is a missing review, so STOP and run code-reviewer for that script before aggregating (G10 requires a code-reviewer review of every Stage 5-8 script), (2) Aggregate WARNING items into a summary, (3) Verify no unresolved BLOCKERs exist, (4) Compose QA Aggregation Summary for PSU4. Do NOT re-read individual QA scripts — STATE.md already tracks all QA outcomes.
- **QA substages** (5-QA through 8-QA) run code-reviewer after each script execution in the parent stage.
- The `search-agent` type writes only its own records (a report at a dispatch-named path, fetch artifacts) and never data files or existing project files; Stage 2 exploration findings are written by the agent at the dispatch-named preliminary notes path, and the orchestrator confirms that file without overwriting it (SKILL.md § Subagent Return Processing, step 2 carve-out), persisting the notes itself only when they came back inline.
- All Stages 5-8 scripts must follow IAT documentation standards (`agent_reference/INLINE_AUDIT_TRAIL.md`).

**Note:** Stages 2, 3, 5, and 6 use domain-specific skills resolved by the orchestrator: from the confirmed pre-flight scope at Stages 2-3 (before a Plan exists), and from the active domain configuration in Plan.md at Stages 5-6.

**Skill loading mechanism:** Every agent this pipeline dispatches except search-agent preloads `data-scientist` via its frontmatter `skills:` field (full content injected at startup). search-agent has no `skills:` field, so its prompt includes `Call the skill tool with name 'data-scientist'` when the task needs it; Stage 2 does (`agent_reference/WORKFLOW_PHASE1_DISCOVERY.md`). Otherwise the orchestrator's Agent prompts include `Call the skill tool` instructions only for skills the agent does not preload (domain skills, Python: `polars`, `plotnine`, `plotly`, `statsmodels`, `pyfixest`, `linearmodels`, `svy`, `scikit-learn`, `geopandas`, `igraph`, `great-tables`; R: `tidyverse`, `ggplot2`, `plotly-r`, `r-stats`, `fixest`, `plm`, `survey-r`, `tidymodels`, `sf-terra`, `igraph-r`, `gt`; either: `science-communication`). Route by execution language preference in CLAUDE.md User Preferences. Stage 2 (`search-agent`) and Stage 3 (`source-researcher`) also need explicit skill tool calls for their domain skills (explorer, source).

**Cross-language annotation preference:** When the user's primary analysis language background differs from the execution language and cross-language code annotations are enabled in CLAUDE.md User Preferences, add the appropriate translation directive to all Stage 5-8 agent prompts. Select the directive based on execution language and background (see orchestrator SKILL.md § User Language Preference Propagation for the full 4-way table): Python execution with R/Stata background loads `r-python-translation`/`stata-python-translation`; R execution with Python/Stata background loads `python-r-translation`/`stata-r-translation`. This propagates to research-executor (code annotation), code-reviewer (annotation verification), debugger (framed error explanations), and data-ingest (profiling script annotation during Data Onboarding). The translation skills are loaded on demand via the Skill tool — they are NOT preloaded in any agent's frontmatter.

**Modeling library selection for Stage 8.1:** The Plan_Tasks.md `<skill>` element specifies which modeling library to load. The orchestrator passes this to the research-executor. The `data-scientist` skill's routing tree provides the canonical decision logic, routed by execution language (per CLAUDE.md User Preferences):

**Python:**
- Standard regression (OLS, GLM, logit/probit) → `statsmodels`
- Time series modeling (ARIMA/SARIMAX, VAR, forecasting, stationarity tests) → `statsmodels`
- Fixed effects, IV with FE, or DiD → `pyfixest`
- Random effects, between estimation, first difference, Fama-MacBeth, LIML / non-panel GMM-IV, SUR/3SLS → `linearmodels`
- Dynamic-panel GMM (Arellano-Bond, Blundell-Bond) → no Python skill: escalate
- Survey-weighted analysis (complex survey design) → `svy`
- Spatial analysis → `geopandas`
- Supervised ML (classification, prediction, risk scoring) → `scikit-learn`
- Unsupervised analysis (clustering, PCA, dimensionality reduction) → `scikit-learn`
- Network / graph analysis (centrality, community detection, bipartite) → `igraph`

**R:**
- Standard regression (OLS, GLM, logit/probit) → `r-stats`
- Time series modeling (ARIMA, forecasting, stationarity tests) → `r-stats`; VAR → escalate (no installed R VAR package; see the data-scientist skill's § Related Skills - When to Load > Load for Specific Needs, Statistical modeling branch)
- Fixed effects, IV with FE, or DiD → `fixest`
- Random effects, between estimation, first difference, Fama-MacBeth → `plm`
- Dynamic-panel GMM (Arellano-Bond, Blundell-Bond) → `plm` (`pgmm`)
- LIML, non-panel GMM-IV, SUR/3SLS → no installed R package: escalate (plain 2SLS: `fixest`)
- Survey-weighted analysis (complex survey design) → `survey-r`
- Spatial analysis → `sf-terra`
- Supervised/Unsupervised ML (classification, prediction, clustering) → `tidymodels`
- Network / graph analysis (centrality, community detection, bipartite) → `igraph-r`

**Methodology verification:** When the Plan specifies statistical methods or modeling approaches, verify that the chosen library supports the intended technique with its current API. Library skills encode syntax at a point in time — if a method call produces unexpected errors during execution, use WebSearch to check the library's latest documentation before assuming the code is wrong. This is especially important for rapidly-evolving libraries.

**Parsing the `<skill>` element:** When constructing the Stage 8.1 Agent prompt, extract the modeling library from the Plan_Tasks.md task block's `<skill>` element. The format is `<skill>data-scientist, {library}</skill>` — the second comma-separated value is the modeling library to substitute for `{modeling_library}` in the invocation template. For unsupervised analysis tasks, the library may be `scikit-learn`.

**Audience type for Stage 11:** The Plan.md "Target Audience" field determines whether the report-writer loads `science-communication`. When constructing the Stage 11 Agent prompt, include `**TARGET AUDIENCE:** {audience}` from the Plan. If the audience is non-technical (policy, executive, public, media), add: "Call the skill tool with name 'science-communication'."

---

## Orchestrator Responsibilities

See the Orchestrator Context Budget in SKILL.md for the general framework (what stays in main context vs. what gets delegated to subagents).

**Pipeline-specific delegation:** In addition to the standard delegation list, Full Pipeline mode delegates **QA code review** — the code-reviewer agent is invoked to closely inspect each individual Stage 5-8 script immediately after execution.

### Task Types

See the "Task Types" section below for the complete taxonomy, behavioral descriptions, and the Task Specificity Test.

### Context Completeness Checklist (MANDATORY)

> For context-field-to-source mapping tables, see the "Context Passing Requirements by Stage Transition" section below.

**Before invoking ANY Stage 5-8 subagent, verify context is complete.** Incomplete context causes subagent confusion, wasted tokens, and incorrect results.

**Stage 5 (Fetch) Checklist:**
- [ ] Script target path specified (absolute, following naming convention)
- [ ] Methodology context from Plan.md and task spec from Plan_Tasks.md inlined
- [ ] Research question inlined
- [ ] Years specified (exact range, not "recent years")
- [ ] Geographic scope specified (state, national, etc.)
- [ ] Filters specified (exact conditions)
- [ ] Expected row count range and critical columns specified
- [ ] Output file paths specified (not placeholder)
- [ ] Missingness and coded value expectations mentioned
- [ ] Risk Register items for fetch included (from Plan.md)
- [ ] Domain query skill specified
- [ ] Script follows IAT documentation standards

**Stage 6 (Clean) Checklist:**
- [ ] Script target path specified (absolute, following naming convention)
- [ ] Methodology context from Plan.md and task spec from Plan_Tasks.md inlined
- [ ] Research question inlined
- [ ] Raw data location specified (exact path from Stage 5 output)
- [ ] Source caveats from Stage 3 inlined (not just referenced)
- [ ] Coded value handling specification provided
- [ ] Suppression tolerance thresholds specified
- [ ] Expected row count range and critical columns identified (from Plan.md Research Outcomes)
- [ ] Output file paths specified (not placeholder)
- [ ] Risk Register items for cleaning included
- [ ] Domain context skill specified (or N/A)
- [ ] Script follows IAT documentation standards

**Stage 7 (Transform) Checklist:**
- [ ] Script target path specified (absolute, following naming convention)
- [ ] Methodology context from Plan.md and task spec from Plan_Tasks.md inlined
- [ ] Research question inlined
- [ ] Input file paths specified (absolute, from prior stage outputs)
- [ ] Output file paths specified (absolute, per Plan_Tasks.md)
- [ ] Prior transformation context inlined (EDA findings, prior transform results)
- [ ] Invariants to maintain listed (from prior transformations)
- [ ] Transformation specification complete (exact columns, exact conditions)
- [ ] Expected outcome specified (row count, shape)
- [ ] Join cardinality specified (if join task)
- [ ] Risk Register items included
- [ ] Research Outcome contribution stated
- [ ] Skill(s) to load specified
- [ ] Script follows IAT documentation standards

**Code-Reviewer (QA) Checklist:**
- [ ] Script path specified (exact path)
- [ ] Plan.md + Plan_Tasks.md expectations INLINED (not just path) — row counts, tolerances, critical columns
- [ ] QA tolerance thresholds specified (BLOCKER if, WARNING if)
- [ ] Risk Register items included
- [ ] Research Outcome contribution stated
- [ ] Prior QA findings accumulated (if any WARNING items from prior scripts)
- [ ] Coded values from Plan.md inlined
- [ ] IAT compliance expectations stated

**Stage 8.1 (Analysis) Checklist:**
- [ ] Script target path specified (absolute, following naming convention)
- [ ] Methodology context from Plan.md and task spec from Plan_Tasks.md inlined
- [ ] Research question inlined
- [ ] Analysis dataset path specified (exact path from Stage 7 output)
- [ ] Statistical method specified (regression, summary stats, comparison, etc.)
- [ ] Dependent and independent variables identified
- [ ] Grouping/stratification variables specified (if applicable)
- [ ] Expected output format specified (summary table, model results, etc.)
- [ ] Output file path specified (`output/analysis/[date]_[description].parquet`)
- [ ] Significance thresholds or interpretation guidelines provided
- [ ] Research Outcome contribution stated
- [ ] Risk Register items included
- [ ] Modeling library skill specified (Python: `statsmodels` / `pyfixest` / `linearmodels` / `svy` / `scikit-learn` / `geopandas` / `igraph`; R: `r-stats` / `fixest` / `plm` / `survey-r` / `tidymodels` / `sf-terra` / `igraph-r` per Plan methodology and execution language; see "Modeling library selection" above)
- [ ] If spatial analysis: `geopandas` (Python) or `sf-terra` (R) skill specified
- [ ] If network analysis: `igraph` (Python) or `igraph-r` (R) skill specified
- [ ] Script follows IAT documentation standards

**Stage 8.2 (Visualization) Checklist:**
- [ ] Script target path specified (absolute, following naming convention)
- [ ] Methodology context from Plan.md and task spec from Plan_Tasks.md inlined
- [ ] Research question inlined
- [ ] Analysis dataset and/or analysis results paths specified
- [ ] Figure specification provided (chart type, variables, grouping)
- [ ] Output file path specified (`output/figures/[date]_[description].png`)
- [ ] Labeling requirements stated (title, axes, legend, source note)
- [ ] Accessibility considerations noted (colorblind-safe palette, etc.)
- [ ] Research Outcome contribution stated
- [ ] Risk Register items included
- [ ] Visualization skill specified (Python: `plotnine` for static, `plotly` for interactive, `geopandas` for maps, `igraph` for network diagrams; R: `ggplot2` for static, `plotly-r` for interactive, `sf-terra` for maps, `igraph-r`/ggraph for network diagrams)
- [ ] Script follows IAT documentation standards

**Revision Request Checklist (when re-invoking research-executor after a QA BLOCKER that is not a `Methodology Conflict` STOP):**
- [ ] Original script path specified (absolute)
- [ ] Current final version path specified (absolute)
- [ ] QA report with BLOCKER details inlined
- [ ] Suggested fix from code-reviewer included
- [ ] Methodology context from Plan.md and task spec from Plan_Tasks.md inlined

**If any checklist item is unchecked:** Add the missing context before invoking. Incomplete context = subagent asks clarifying questions = wasted round-trip.

These checklists correspond to the input requirements defined in each agent's definition file (e.g., `.claude/agents/research-executor.md` > Inputs).

### Progress Reporting Protocol

Report to the user **adaptively** at these trigger points:

| Trigger | Report Content |
|---------|----------------|
| Phase completion | Summary of phase outcomes, any issues encountered |
| Phase boundary | **Phase Status Update (PSU) — MANDATORY. Present comprehensive PSU and WAIT for user confirmation. See Phase Status Updates section.** |
| Notable finding | Surprising data insight, limitation discovered |
| Decision point | Methodology choice with rationale |
| Error/blocker | Issue description, attempted resolution, escalation if needed |
| STOP condition hit | Clear explanation of why execution paused |

**Report Format:**
```
**Progress Update: [the activity, in plain words — e.g., cleaning the data]**
- Completed: [What was done]
- Key Findings: [Notable insights or issues]
- Next Steps: [What happens next]
- [If applicable] Action Needed: [What user input is required]
```

**Note:** Progress reports during a phase are one-way informational updates. Phase Status Updates at phase boundaries are BLOCKING — the orchestrator must wait for user confirmation before proceeding.

**Turn-end briefing.** PSUs are the heavyweight, blocking, phase-boundary instances of DAAF's general Turn-End Briefing standard, and the lightweight Progress Update above is its in-phase, non-blocking form. Every *other* substantive turn that returns control to the user — an error-recovery pause, a mid-phase return after a wave of dispatches, an answer to a user question during execution — still closes with a briefing of what happened since the user's last message, taken altogether. See the master statement in `SKILL.md` § User-Facing Communication Standards > "Turn-End Briefing."

### Phase Status Updates (Mandatory)

**Phase Status Updates (PSU) are enforced pause points at every phase boundary.** After completing a phase, the orchestrator MUST present a comprehensive Phase Status Update to the user and WAIT for explicit confirmation before proceeding to the next phase.

**Cardinal Rule:** No phase transition occurs without user approval. The orchestrator presents, the user decides.

#### PSU Design Principles

1. **Blocking:** The orchestrator MUST wait for an explicit user response. Do NOT proceed automatically.
2. **Comprehensive:** Each PSU includes everything the user needs to make an informed go/no-go decision.
3. **Actionable:** Each PSU ends with explicit approval request and clear options for the user.
4. **Cumulative:** Later PSUs reference earlier ones, building a coherent narrative of the analysis.

#### PSU Schedule

| ID | Transition | After Stage | Before Stage | What User Reviews |
|---|---|---|---|---|
| PSU1 | Phase 1 → Phase 2 | 3.5 (Synthesis) | 4 (Plan Creation) | Discovery findings, data availability, source caveats, feasibility, recommended approach |
| PSU2 | Phase 2 → Phase 3 | 4.5 (Plan Validation) | 5 (Data Retrieval) | Plan.md summary (methodology, scope, task sequence, research outcomes, hypotheses) — user directed to read full Plan.md before approving |
| PSU3 | Phase 3 → Phase 4 | 6 (Context Application) | 7 (EDA & Transformation) | Data quality metrics, suppression rates, datasets acquired, QA1/QA2 summaries |
| PSU4 | Phase 4 → Phase 5 | 10 (QA Aggregation) | 11 (Report Generation) | Statistical results, key visualizations, QA aggregation, deviations from Plan.md |

#### PSU Template and Content Requirements

Every Phase Status Update MUST follow this format, written in **plain language** per the Plain-Language Rule table in `SKILL.md` § User-Facing Communication Standards, the single glossary of internal terms and their user-facing wording. The header names the work just finished: Discovery (PSU1), Planning (PSU2), Data Preparation (PSU3), Analysis (PSU4).

```
**Checkpoint: {Activity} Complete**

[Progress bar showing how much of the analysis is done, with no phase numbers]
━━━━━━━━━━━━░░░░░░░░░░░░░░░░░░ (or similar proportional marker)

**Why this checkpoint:** [1 sentence — see checkpoint purpose text below]

**Summary:**
[2-3 sentence overview of what was accomplished in this phase]

**Key Findings:**
- [Finding 1]
- [Finding 2]
- [Finding 3]

**Decisions Made:**
| Decision | Rationale | Impact |
|----------|-----------|--------|
| [decision] | [why] | [what it affects] |

**Warnings & Issues:**
| Item | Severity | Status | Details |
|------|----------|--------|---------|
| [item] | Needs resolution / For your awareness | Resolved/Open | [details] |

[If no warnings: "No warnings or issues encountered in this phase."]

**Artifacts Produced:**
- [File path 1]: [description]
- [File path 2]: [description]

**What Comes Next:**
[Phase transition narrative — see bridge text below. 2-3 sentences connecting
what just happened to what comes next, written as a coherent story.]

**What's Most Useful From You Here:**
[Checkpoint-specific feedback guidance — see per-PSU content below]

**Your Options:**
- **Approve** — proceed to the next phase
- **Ask questions** — I'll explain anything that's unclear, then check back in again
- **Request changes** — tell me what to adjust, and I'll revise within this phase
- **Change scope** — if you'd like to expand or narrow the analysis

**Ready to move forward, or would you like to discuss anything first?**
```

#### PSU-Specific Content

The checkpoint purpose text, phase transition bridge text, feedback guidance, and content requirements for each PSU live in the respective WORKFLOW_PHASE file alongside the PSU definition:

| PSU | Location |
|-----|----------|
| PSU1 | `WORKFLOW_PHASE1_DISCOVERY.md` > Phase Status Update 1 |
| PSU2 | `WORKFLOW_PHASE2_PLANNING.md` > Phase Status Update 2 |
| PSU3 | `WORKFLOW_PHASE3_ACQUISITION.md` > Phase Status Update 3 |
| PSU4 | `WORKFLOW_PHASE4_ANALYSIS.md` > Phase Status Update 4 |

#### User Response Handling

The checkpoint template includes an explicit "Your Options" block so the user always knows what they can do. Internally, handle responses as follows:

- **Approve** ("proceed", "looks good", "continue", etc.) → Proceed to next phase. If this is the user's first Full Pipeline session (based on conversation history), briefly preview what comes next: *"Great — moving on to [next activity]. I'll check back in when [next checkpoint condition]."*
- **Request revision** ("redo X", "fix Y", "I'm concerned about Z") → Orchestrator addresses within current phase, then re-presents the checkpoint
- **Request scope change** ("can we also look at...", "let's narrow to...") → Triggers scope change protocol (Ask First), then revises as needed
- **Ask questions** ("what does X mean?", "why did you choose Y?") → Orchestrator answers, then re-presents the approval request

**CRITICAL:** After answering questions or providing clarification, the orchestrator MUST re-present the approval request. Do not assume that a question implies approval.

### Plan Document Maintenance

The Plan documents (Plan.md + Plan_Tasks.md) are your **persistent memory** for methodology and task specification, and the most important documents for auditability, replicability, and rigor. They are **frozen after Stage 4.5** — all runtime state updates go to STATE.md.

1. **Create** during Phase 2 (Stage 4) — data-planner produces Plan.md (strategic specification) + Plan_Tasks.md (executable task sequence)
2. **Freeze** after Stage 4.5 (Plan Validation) — no further modifications to Plan.md or Plan_Tasks.md
3. **Reference** when delegating to subagents — methodology context from Plan.md, task-specific context from Plan_Tasks.md (orchestrator searches for `### Task {step}:` headers to find specific task blocks)
4. **Track runtime state** in STATE.md — all decisions, findings, risks, QA results, and progress go here

See `agent_reference/PLAN_TEMPLATE.md` for the complete template.

The Plan documents are the **single source of truth** for methodology and task specification. They:
- Capture all pre-execution decisions and their rationale
- Provide context for subagent invocations
- Enable session continuity (return to work later)
- Support version control for revisions

STATE.md is the **single source of truth** for runtime state. It tracks:
- All runtime decisions and their rationale (Key Decisions Made)
- Runtime risks discovered during execution (Runtime Risks)
- QA findings across all stages (QA Findings Summary)
- Final review results (Final Review Log)
- Transformation progress and checkpoint statuses

**Completeness Standard:** The Plan documents must be comprehensive enough that any subagent can execute its stage with ONLY Plan.md + Plan_Tasks.md as context (plus skill knowledge).

### Runtime State Update Events

Update STATE.md as the analysis progresses:

| Event | STATE.md Section to Update |
|-------|---------------------------|
| Decision made | Key Decisions Made |
| Limitation discovered | Context Snapshot or Runtime Risks |
| Deviation from plan | Key Decisions Made (with rationale) |
| Checkpoint passed | Checkpoint Status table |
| Error encountered | Blockers section |
| Phase completed | Session History |
| Risk identified | Runtime Risks (see below) |
| QA finding | QA Findings Summary |
| Final review result | Final Review Log |

### Runtime Risk Tracking

Runtime risks discovered during execution are tracked in STATE.md's **Runtime Risks** section.

**Update Triggers:**

| Trigger Event | Risk Type | When to Add |
|---------------|-----------|-------------|
| Stage 3 discovers source-specific limitations | Data Quality | When caveats affect analysis validity or completeness |
| Stage 5 fetch returns unexpected shape | Data Availability | When row count deviates significantly from expected |
| Stage 6 suppression rate is 30-50% | Data Quality | Even if below STOP threshold (50%), document the risk |
| Stage 5 data lag detected (CP1 year-freshness check) | Data Quality | When latest year available is older than requested |
| Stage 7 transformation has unexpected row loss | Methodological | When row count drops >20% unexpectedly |
| Stage 7 join cardinality violation | Methodological | When actual cardinality differs from expected |
| Any stage encounters data definition changes | Data Quality | When variable definitions changed between years |

**Update Format:**

Add row to Runtime Risks section of STATE.md with:
- **Risk:** Clear description of the issue
- **Likelihood:** Low/Medium/High
- **Impact:** Low/Medium/High (on analysis validity/completeness)
- **Mitigation:** What was done or will be done to address it
- **Owner/Stage:** Which stage discovered and owns the risk

**Example Update:**
```markdown
| Suppression Rate Elevated | Medium | Medium | Aggregate to district level if exceeds 40% | Stage 6 |
```

---

## Pipeline-Specific Subagent Patterns

These patterns supplement the Universal Prompt Requirements in `SKILL.md` and the invocation templates in the `agent_reference/WORKFLOW_PHASE*.md` files.

### Wave-Based Parallel Execution

Tasks in Plan_Tasks.md have wave assignments:

```
Wave 1: [fetch-ccd, fetch-meps]     ← Run in parallel
Wave 2: [clean-ccd, clean-meps]     ← Depends on Wave 1
Wave 3: [join-data]                 ← Depends on Wave 2
```

**Execution Rules:**
- Same-wave tasks dispatch simultaneously as multiple Agent tool calls in a **single response message**. See `SKILL.md` § Subagent Coordination > Wave Barrier Discipline (Async Dispatch) for how results arrive. Wait for every task in the wave to return before proceeding.
- **Stages 5-8:** each same-wave task runs its own Composite Execution Pattern loop. Dispatch the wave's research-executor calls together; once all have returned, dispatch one code-reviewer per script together (at most 5 in flight each time). Every task in the wave has its review recorded in STATE.md, and any BLOCKER handled, before the next wave begins, because the next wave holds the tasks that depend on it.
- If any parallel dispatch stage contains more than 5 tasks (e.g., Stage 3 source-researcher dispatch, any ad-hoc parallel exploration, and code-reviewer invocations), sub-batch into groups of ≤5 and wait for each sub-batch to complete before dispatching the next. This is the default parallel dispatch limit (`SKILL.md` § Subagent Coordination > "Parallel Dispatch Guideline"); dispatch more than 5 at once only with the user's explicit approval.
- Each subagent starts with a fresh context window (no accumulated degradation)
- Later waves wait for ALL prior waves to complete
- Dependencies in `depends_on` must be satisfied

**Wave Barrier Discipline (async dispatch).** See `SKILL.md` § Subagent Coordination > Wave Barrier Discipline (Async Dispatch) for how results arrive. The barrier applies however they arrive. Treat every mid-wave notification as **status-only**: do not make gate decisions, revise the plan, finalize STATE.md conclusions, present a PSU/checkpoint, or begin QA aggregation or synthesis until EVERY task in the wave has returned. Synthesize once, over the complete wave — never incrementally per return, so a later return can genuinely change the decision rather than being anchored out by the first. An early return under context pressure, or a failed/BLOCKED task, still counts as that task's completion — handle its redelegation as part of whole-wave synthesis, not as an immediate mid-wave reaction. Narrating progress to the user ("two of three fetches are back") is fine; acting on partial results is not. This is the pipeline application of the master statement in `SKILL.md` § Subagent Coordination > "Wave Barrier Discipline (Async Dispatch)."

See `agent_reference/PLAN_TEMPLATE.md` for wave-based task table format.

### Plan_Tasks.md Extraction Protocol

Plan_Tasks.md can be 1000+ lines. **Never read the full file into orchestrator context.** Use targeted extraction:

1. **Read the Task Index** — The `## Task Index` table is near the top of the file (~first 30-40 lines after frontmatter). It provides a compact lookup of all tasks: step, name, wave, stage, script path, dependencies.
2. **Identify the target task** — Match by step number (e.g., `1.1`) or task name (e.g., `fetch-ccd-schools`).
3. **Search for the task header** — Use the pattern `### Task {step}: {task-name}` to find the specific block (e.g., `### Task 1.1: fetch-ccd-schools [Stage 5]`).
4. **Read the task block** — From the `### Task` header through the closing `</task>` tag. Each block is typically 20-40 lines.
5. **Inline the extracted block** — Paste the `<task>` XML into the subagent prompt under `## TASK SPECIFICATION`.

**What the orchestrator keeps in its own context:**
- The Task Index table (compact — ~10-20 rows)
- The current wave number and which tasks remain

**What gets extracted on demand per subagent dispatch:**
- The specific `<task>` XML block for that dispatch
- Relevant methodology context from Plan.md (separate extraction)

### Thoroughness Directives by Stage

See the Thoroughness Directive section in the appropriate `agent_reference/WORKFLOW_PHASE*.md` file for stage-specific requirements.

### Handoff Specifications

Each stage has explicit input/output contracts and gate criteria:

| Stage | Input From | Output To | Gate Criteria |
|-------|------------|-----------|---------------|
| 1 | User request + the materials the user referenced | Stage 2 | G1: Pre-flight Scope Check confirmed (includes mode confirmation) |
| 2 | Stage 1 + pre-flight (mode, confirmed scope, user-referenced materials) | Stage 3 | G2: ≥1 dataset identified, key variables flagged |
| 3 | Stage 2 datasets | Stage 3.5 | G3: All flagged variables investigated, coded values documented, suppression patterns identified |
| 3.5 | Stages 2, 3 | PSU1 to user, then Stage 4 | G3.5: Synthesis complete, conflicts resolved, user confirmed PSU1 |
| 4 | Phase 1 findings + confirmed pre-flight scope and user-referenced materials | Stage 4.5 | G4: Plan.md + Plan_Tasks.md created, STATE.md created, LEARNINGS.md skeleton created |
| 4.5 | Stage 4 (Plan) | PSU2 to user, then Stage 5 | G4.5: Plan validation PASSED or PASSED_WITH_WARNINGS, user confirmed PSU2 |
| 5 | Plan.md + Plan_Tasks.md (query spec) | Stage 6 | G5: CP1 PASSED per script, code-reviewer separately invoked per script immediately after completion, all QA1 ∈ {PASSED, WARNING}, data saved to data/raw/ |
| 6 | Stage 5 (raw data) | PSU3 to user, then Stage 7 | G6: CP2 PASSED per script, code-reviewer separately invoked per script immediately after completion, all QA2 ∈ {PASSED, WARNING}, suppression <50%, data saved to data/processed/, user confirmed PSU3 |
| 7 | Stage 6 (clean data) | Stage 8, 9 | G7: All transformations validated (CP3) per script, code-reviewer separately invoked per script immediately after completion, all QA3 ∈ {PASSED, WARNING}, analysis dataset saved to `data/processed/[date]_analysis.parquet` (at Stage 7.3) |
| 8 | Stage 7 (analysis data) | Stage 9, 11 | G8: CP4 PASSED per script, statistical results saved to output/analysis/, visualizations saved to output/figures/, code-reviewer separately invoked per 8.1 script (QA4a, plus QA4b when the analysis script also draws figures) and per 8.2 script (QA4b) immediately after each completes, all QA4a and QA4b ∈ {PASSED, WARNING} |
| 9 | Stages 7, 8 | Stage 10 | G9: Python passes the `marimo check --select MB` structural gate, the `marimo export html` execution check and the Step 6c verbatim fidelity check (all four conditions: clean decompile, bundle count and decompiled paths equal to the final-version inventory, only expected numstat lines), or R `quarto render` validates assembly and enabled previews; all scripts represented with code + execution logs; Quarto render does not re-run archived Stage 5-8 analysis scripts; integration-checker (Stage 9 scope) CONNECTED or WARNING-only |
| 10 | Stage 9 (notebook) | PSU4 to user, then Stage 11 | G10: QA findings aggregated, all BLOCKERs resolved, all WARNINGs documented, user confirmed PSU4 |
| 11 | Stages 9, 10, Plan.md + Plan_Tasks.md, STATE.md, LEARNINGS.md | Stage 12 | G11: report-writer returned COMPLETE or COMPLETE_WITH_GAPS, all REPORT_TEMPLATE.md sections populated, figure references verified, integration-checker (Stage 11 scope) CONNECTED or WARNING-only |
| 12 | All prior stages | Delivery | G12: integration-checker (full scope) CONNECTED or WARNING-only before data-verifier, Final Review PASSED, all commitments fulfilled, LEARNINGS.md consolidated with System Update Action Plan, cross-artifact coherence verified |

**QA Gate Notes:**
- **PASSED or WARNING:** QA may log WARNINGs that don't block execution (documented for Stage 10 aggregation)
- **QA BLOCKER:** A methodology BLOCKER (a code-reviewer STOP with Issue Category `Methodology Conflict`) escalates to the user immediately (RULE 4); any other BLOCKER requires revision before handoff (max 2 attempts, then escalate)
- **QA findings aggregated:** Stage 10 consolidates all WARNINGs from Stages 5-8 for final review

### Subagent Output Verification Protocol

**CRITICAL:** Before integrating subagent findings into STATE.md (or into Plan documents during Phase 2) or proceeding to the next stage, verify that subagent output meets orchestrator expectations.

**Verification Checklist:**

| Check | What to Verify | Action if Failed |
|-------|----------------|------------------|
| **Size** | Output is under 2000 words | Extract only structured summary fields; discard verbose sections, raw logs, and data samples before integrating into context. If chronic, re-invoke with emphasis on the 2000-word hard cap. |
| **Completeness** | All required output sections present (for an agent-written notes file, checked on the file, not on the summary it returns) | Re-invoke with clarification |
| **Format** | Output matches specified OUTPUT FORMAT (agent-written notes file: checked on the file) | Re-invoke with format emphasis |
| **Confidence** | No LOW confidence items without resolution | Request resolution or escalate |
| **Substantive** | Real findings, not template placeholders | Re-invoke with thoroughness emphasis |
| **Persistence** | Preliminary notes file on disk (notes-producing agents only: search-agent Stage 2 and source-researcher Stage 3, written by the agent at the dispatch-named path, confirmed but never overwritten by the orchestrator, and written by the orchestrator only if they came back inline; research-synthesizer Stage 3.5, written by the orchestrator) | Retry write; if retry fails, stop and report to user |

**Verification Procedure:**

1. **After subagent returns findings:**
   - Review output against the OUTPUT FORMAT specification provided in the prompt
   - Check that all required sections contain substantive content
   - Verify any confidence assessments (HIGH/MEDIUM/LOW)
   - Confirm no placeholder text remains (e.g., "[add more]", "[description]")

2. **If verification fails (first time):**
   - Re-invoke subagent with clarification about what's missing
   - Provide more specific context or examples
   - Emphasize the missing elements

3. **If verification fails (second time):**
   - Re-invoke with simplified task scope
   - Break complex tasks into smaller subtasks
   - Consider if task is feasible with available skills

4. **If verification fails (third time):**
   - STOP execution
   - Escalate to user with explanation of what couldn't be completed
   - Propose alternative approaches

**Example Verification:**

```markdown
**Subagent Output Review: Stage 2 (Data Exploration)**

Checklist:
- [x] Recommended Data Level specified
- [x] Candidate Datasets table complete (3 mirror paths found)
- [x] Key Variables table complete (8 variables identified)
- [x] Variables Flagged for Deep-Dive (2 flagged with reasons)
- [x] Limitations Encountered documented
- [x] Completeness Assessment all items checked
- [x] Confidence: HIGH (multiple sources confirm)

Status: VERIFIED - Proceeding to Stage 3
```

**Code-Reviewer Output Verification (Additional):**

When verifying code-reviewer QA reports specifically, also check:
- [ ] cr1 includes at least 5 script-specific checks (one per Skeptical Lens) and 5 spot-checks
- [ ] cr1 includes data profiling section; if multiple iterations, each has documented trigger
- [ ] Report includes reasoning (WHY correct, not just WHAT was checked)
- [ ] Adversarial analysis section has substantive content (not boilerplate)
- [ ] If PASSED: report articulates basis for confidence, not just absence of failures
- [ ] Report includes Investigation Narrative synthesizing across all iterations
- [ ] If capped at 5 iterations: "Additional Strands of Inquiry" section present

If the QA report reads like a template with values filled in and no script-specific reasoning, it has not met the review depth expectation. Consider re-invoking with emphasis on adversarial analysis.

### Learning Signal Extraction

For the complete Learning Signal Protocol (format, categories, accumulation flow, flush triggers), see the "Learning Signal Protocol" section below.

After verifying subagent output, extract any Learning Signal:

1. Check if subagent output contains a `**Learning Signal:**` field
2. If value is "None" → skip
3. If value is present → append to STATE.md "Pending Learning Signals" buffer:
   ```
   - [Stage N.step] [Category] — [Signal text]
   ```
4. Do NOT write to LEARNINGS.md on every signal — wait for flush triggers

---

## Invocation Principles

### When to Delegate

Delegate to subagents to:
- Preserve orchestrator context
- Leverage specialized skill knowledge
- Execute focused tasks

### Skill Loading Confirmation

After invoking a skill, confirm it loaded successfully:

**Indicators of successful loading:**
1. The skill's core guidance is now available in your working context
2. Reference files can be accessed from the skill's `./references/` directory
3. The skill's decision trees and workflows are clear

**If skill loading fails:**
1. Report: "Unable to load [skill-name] skill"
2. Attempt to proceed with base knowledge (if possible)
3. Flag reduced confidence in output due to missing specialized guidance
4. Escalate to user if skill is critical for the task

**Example confirmation check:**
```markdown
After calling skill tool with name 'education-data-explorer':
- ✓ Core guidance loaded: Understand data levels (schools, districts, colleges)
- ✓ Reference files accessible: schools-datasets.md, districts-datasets.md, colleges-datasets.md
- ✓ Decision trees clear: "What data level (and mirror file) do I need?" flow available

Status: Skill loaded successfully, proceeding to route the question to mirror datasets
```

### Subagent Type Selection

See daaf-orchestrator SKILL.md "Subagent Type Selection" for capabilities by type (`search-agent` and `source-researcher` = search and information gathering, write only their own records; `Plan` and the agents without Write or Edit tools (plan-checker, data-verifier, integration-checker) = read-only; `general-purpose` = full capabilities including file writes).

## Standard Agent Prompt Structure

**REQUIRED:** Every subagent invocation MUST use this standardized format to ensure consistent handoffs and verifiable outputs.

### File-First Execution Rule

All code execution in Stages 5-8 MUST follow the file-first pattern (write → execute via wrapper → version on failure). See `CLAUDE.md` > "Execution Philosophy" and `agent_reference/SCRIPT_EXECUTION_REFERENCE.md` for the complete file-first protocol.

### Model Selection

Every dispatch uses the target agent's `model:` frontmatter default (DAAF's two-tier `opus`/`sonnet` routing). The orchestrator MAY override the tier per dispatch via the Agent tool's `model` parameter — escalate to `opus` for complex methodology, retries of BLOCKED/failed work, or high-stakes outputs; downgrade to `sonnet` for small mechanical tasks. Never exceed the session-model ceiling (enforced by `enforce-model-ceiling.sh`). See daaf-orchestrator SKILL.md > "Model Selection for Subagent Dispatch" for the routing table and full guidance — that section is the single source of truth; it is not restated here.

### Template

```python
Agent({
    description: "[3-5 word summary]",
    prompt: """**BASE_DIR:** {BASE_DIR}
All relative paths in referenced files resolve from BASE_DIR.

## SKILL LOADING
[Only include skill tool calls for skills NOT preloaded via agent frontmatter.
Agents whose frontmatter lists `data-scientist` already have it — do not re-load it;
search-agent does not preload it.
Call the skill tool only for additional skills — route by execution language
(per CLAUDE.md User Preferences):
  Python: polars, plotnine, plotly, statsmodels, pyfixest, linearmodels, svy,
          geopandas, scikit-learn, igraph, great-tables
  R: tidyverse, ggplot2, plotly-r, r-stats, fixest, plm, survey-r, tidymodels,
     sf-terra, igraph-r, gt
  Either: science-communication, domain skills]

## CONTEXT FROM PLAN
[Paste relevant Plan.md methodology sections and Plan_Tasks.md task blocks - Context Completeness Checklist always takes priority over brevity]

Original Request: [verbatim user request — required for Stage 4; include for other stages when methodology alignment matters]
Research Question: [from Plan.md]
Data Source: [from Plan.md]
Current Stage: [N]
Wave: [N] (if applicable)

## TASK SPECIFICATION
<task name="[task-name]" type="[auto|checkpoint:human-verify|checkpoint:decision]" wave="[N]">
  <depends_on>[task-ids or "none"]</depends_on>
  <skill>[skill-name]</skill>
  <agent>[agent-name]</agent>
  <files>
    <input>[input file path]</input>
    <output>[output file path]</output>
  </files>
  <action>
    1. [Specific step 1]
    2. [Specific step 2]
    3. [Specific step 3]
  </action>
  <verify>
    - [Verification criterion 1]
    - [Verification criterion 2]
    - [Verification criterion 3]
  </verify>
  <done>[Measurable completion condition]</done>
</task>

## FILE-FIRST RULE (Stages 5-8)
Write code to a script file FIRST (.py for Python, .R for R). Do NOT execute interactively.
Execute ONLY via single Bash call: `bash {BASE_DIR}/scripts/run_with_capture.sh {PROJECT_DIR}/scripts/.../script.py` (or `script.R` for R) — do NOT run `python script.py` or `Rscript script.R` directly, chain commands with `&&`/`;`, or prefix with `cd`.
Follow the IAT documentation standard (`{BASE_DIR}/agent_reference/INLINE_AUDIT_TRAIL.md`).
Closely read `{BASE_DIR}/agent_reference/SCRIPT_EXECUTION_REFERENCE.md` for the mandatory file-first execution protocol covering complete code file writing, output capture, and file versioning rules.

## OUTPUT FORMAT

**Hard cap: 2000 words maximum.** The orchestrator has limited context — every word you return consumes shared capacity across the entire pipeline. Your Agent output is a *signal to the orchestrator*, not an archive. The script files on disk are the archive.

**Do NOT include in your output:**
- Raw execution logs or captured stdout/stderr (already appended to the script file by `run_with_capture.sh`)
- Data samples, row-level examples, or Polars/pandas table displays
- Full checkpoint output (summarize as PASSED/WARNING/FAILED + the checkpoint's count line + 1-line reason)
- QA script code or contents (reference by file path instead)

Return findings in this EXACT structure:

### [Task Name] Results

**Status:** [PASSED | FAILED | WARNING]
**Task Type:** [auto | checkpoint:human-verify | checkpoint:decision]

> **Log line ⇔ reported status.** The checkpoint verdict printed in the execution log uses three strings — `PASSED`, `PASSED WITH WARNINGS`, `FAILED`. Map them to the reported status above: `PASSED` ⇒ `PASSED`, **`PASSED WITH WARNINGS` ⇒ `WARNING`**, `FAILED` ⇒ `FAILED`. Never report `PASSED` for a log line that said `PASSED WITH WARNINGS`. The count line immediately above the verdict (`Checks: N passed, N warnings, N failed, N not verified, N skipped`) is part of the summary: if `not verified > 0`, the CP verdict is `FAILED` by construction, and a QA `RESULT` is at least `WARNING`.

**Summary:**
[2-3 sentence summary of what was done]

**Pre-State:** [For transformations: shape, sample]
**Post-State:** [For transformations: shape, sample]
**Row Change:** [+/-X% or N/A]

**Findings:**
- [Key finding 1]
- [Key finding 2]
- [Key finding 3]

**Verification:**
| Criterion | Result | Notes |
|-----------|--------|-------|
| [From verify block] | PASS/FAIL | [Details] |

**Files Created/Modified:**
- `[path]`: [description]

**Issues Encountered:**
- [Issue + resolution, or "None"]

**Confidence:** [HIGH | MEDIUM | LOW]
**If LOW:** [What needs resolution before proceeding]

**Deviations Applied:** [List per RULE 1-3 from `{BASE_DIR}/agent_reference/BOUNDARIES.md`, or "None"]

**User-Facing Summary:** [For phase-ending tasks only: 5-8 sentence summary for inclusion in the Phase Status Update. Write for a research professional audience. Omit this field for mid-phase tasks.]

**Commit:** [If task completed, suggested commit message]
""",
    subagent_type: "[agent-name]"
})
```

---

## Task Types

| Type | When to Use | Human Involvement |
|------|-------------|-------------------|
| `auto` | Fully automatable (90% of tasks) | None unless STOP condition |
| `checkpoint:human-verify` | Needs visual confirmation | Report results, await "proceed" |
| `checkpoint:decision` | Multiple valid approaches | Present options, await selection |
| `checkpoint:human-action` | User must perform action themselves | Report instructions, await completion |

**Note:** `checkpoint:human-action` is used when Claude cannot automate a step (e.g., external authentication, restricted data downloads). See `agent_reference/VALIDATION_CHECKPOINTS.md` § Checkpoint Classification System for full classification details.

### checkpoint:auto (Default)

Use for:
- Data downloads (mirror-based, API-based, etc.)
- Data cleaning
- Transformations
- Aggregations
- Visualization generation
- Test execution

**Behavior:** Execute, validate, report status. Proceed if PASSED.

### checkpoint:human-verify

Use for:
- Unusual suppression patterns (30-50%)
- Data lag warnings (≥3 years)
- COVID-19 data quality flags
- Final Report before delivery
- Results that differ from expectations

**Behavior:** Execute, report results with context, ask "Should I proceed?"

### checkpoint:decision

Use for:
- Multiple valid data sources
- Methodology alternatives
- Scope adjustments when data is limited
- How to handle edge cases

**Behavior:** Present options with pros/cons, await selection, then execute

### Task Specificity Test

Before sending any task to a subagent, verify it passes this test:

**Test:** Could a fresh Claude instance with ONLY this task description + skill access complete it without asking clarifying questions?

**Checklist:**
- [ ] **Unambiguous Scope:** Clear what files/data this touches
- [ ] **Concrete Actions:** Steps specific enough to execute without interpretation
- [ ] **Verifiable Completion:** "done" condition is objectively measurable
- [ ] **No Hidden Dependencies:** All prerequisites explicitly stated
- [ ] **Size Appropriate:** One focused task; if its context runs consistently large, split it into smaller tasks

If any checkbox is unchecked → Add specificity until all pass.

---

## Context Inlining Protocol

### Principle: Inline Critical Context Directly

**Rule:** When dispatching an Agent, inline critical context directly in the prompt. Don't rely on subagent file reads for essential information.

**Why This Matters:**
- Eliminates round-trip file lookups
- Ensures subagent has complete context immediately
- Produces reproducible results across sessions
- Reduces failure points (no "file not found" errors)

### What to Inline

| Content Type | Inline? | Rationale |
|--------------|---------|-----------|
| Relevant Plan.md sections | YES | Methodology decisions needed |
| Relevant Plan_Tasks.md task blocks | YES | Task specifications needed |
| Prior stage findings | YES | Dependencies must be clear |
| Decision context | YES | Rationale affects execution |
| Expected values | YES | Validation needs targets |
| File paths | YES | Must be explicit |
| Full skill content | NO | Subagent loads via skill tool |
| Raw data samples | NO | Only shapes/summaries |
| Complete code files | NO | Only relevant sections |

### Inlining Template

```python
Agent({
    description: "Stage [N]: [Name]",
    prompt: """**BASE_DIR:** {BASE_DIR}
All relative paths in referenced files resolve from BASE_DIR.

## SKILL LOADING
Call the skill tool with name '[skill-name]'.

## INLINED CONTEXT

### From Plan.md (Methodology):
{paste_relevant_methodology_section}

### From Plan_Tasks.md (Task Specification):
{paste_relevant_task_block}

### From Stage [N-1] (Prior Findings):
- Key finding 1: [value]
- Key finding 2: [value]
- Files created: [paths]
- CP Status: [PASSED/FAILED]

### Decision Context:
- [Decision 1]: [what was decided and why]
- [Decision 2]: [what was decided and why]

## TASK SPECIFICATION
<task name="[task-name]" ...>
...
</task>
""",
    subagent_type: "[agent-name]"
})
```

### What NOT to Inline

- **Full Plan.md or Plan_Tasks.md:** Too large. Inline only relevant sections/task blocks.
- **Complete skill content:** Subagent loads via skill tool (5K-20K tokens saved).
- **Downloaded raw data:** Summarize to shapes and key values.
- **Complete notebooks:** Reference by path, inline only specific cells if needed.
- **Full error tracebacks:** Summarize to error type and key message.

### Size Limits for Inlined Content

There is no fixed word target: the Context Completeness Checklist (above) decides what goes in, and completeness beats brevity. An incomplete prompt costs more (a confused subagent, a re-dispatch, wasted output) than a thorough one.

**If context needs consistently run large:** Consider breaking the task into smaller subtasks, each with more focused scope.

---

## Code-Reviewer Invocation (Cross-Phase QA)

### code-reviewer (QA Agent)

**Purpose:** Secondary QA review of executed scripts
**Stage:** 5-QA, 6-QA, 7-QA, 8-QA (after each script execution; Stage 8 uses QA4a for analysis, QA4b for visualization, and both for an analysis script that also draws figures)
**Subagent:** `code-reviewer`

**Invocation Timing:** After research-executor completes each script (after CP validation passes); for a wave, once every research-executor in it has returned (§ Wave-Based Parallel Execution).

```python
Agent({
    description: "QA Review: Stage {N} Step {step} - {task_name}",
    prompt: """**BASE_DIR:** {BASE_DIR}
**PROJECT_DIR:** {PROJECT_DIR}
All relative paths in referenced files resolve from BASE_DIR.

**SCRIPT TO REVIEW:**
Path: {PROJECT_DIR}/scripts/stage{N}_{type}/{step}_{task-name}.py (or .R for R)

**PLAN LOCATIONS:**
Plan.md: {plan_path}
Plan_Tasks.md: {plan_tasks_path}

**OUTPUT FILES (absolute paths):**
{list_of_output_files}

**CONTEXT:**
- Checkpoint: {QA1 | QA2 | QA3 | QA4a | QA4b}
- Stage: {N}
- Step: {step}
- Wave: {wave}
- Task: {task_name}
- Research Question: {research_question}

## PLAN EXPECTATIONS FOR THIS TASK (REQUIRED - Inline, not just path)

| Aspect | Expected | Source |
|--------|----------|--------|
| Output rows | {min_rows} - {max_rows} | Plan.md Transformation Sequence row {step} |
| Row change | ±{tolerance}% | Plan_Tasks.md expected outcome |
| Critical columns | {column_list} | Plan.md Research Outcomes |
| Max acceptable loss | {loss_pct}% | Plan.md Risk Register |
| Join cardinality | {cardinality_or_NA} | Plan_Tasks.md (if join task) |

## RISK REGISTER ITEMS FOR THIS STAGE

| Risk | Mitigation | Watch For |
|------|------------|-----------|
| {risk_1} | {mitigation_1} | {symptom_1} |

## RESEARCH OUTCOME CONTRIBUTION

This task contributes to: "{research_outcome_text}"
Addressed when: {verification_condition}

## QA TOLERANCE FOR THIS ANALYSIS

- Acceptable row change: ±{tolerance}%
- Acceptable new nulls: {null_tolerance}%
- BLOCKER if: {blocker_condition}
- WARNING if: {warning_condition}

**TASK:**
1. Review the executed script for correctness and methodology alignment
2. Review the execution log for outcome verification
3. Create iterative QA scripts at: {PROJECT_DIR}/scripts/cr/stage{N}_{step}_cr1.py (+ cr2..cr5 as warranted)
   [For Stage 8, use the split-QA suffix instead: {PROJECT_DIR}/scripts/cr/stage8_{step}_cra1.py for QA4a (statistical validity) / {PROJECT_DIR}/scripts/cr/stage8_{step}_crb1.py for QA4b (visualization quality)]
4. Execute QA scripts and synthesize findings across iterations
5. Return QA report with severity classification

**PRIOR QA FINDINGS (if any):**
{accumulated_warnings_from_prior_scripts}

**OUTPUT FORMAT:**
Return findings using the Code Reviewer Output Format
(see your agent protocol, § Output Format).

**Emphasis for this invocation:**
- Code correctness and execution success
- Methodology alignment with Plan specifications
- IAT documentation completeness
""",
    subagent_type: "code-reviewer"
})
```

### Revision Request (After QA BLOCKER)

When code-reviewer returns a BLOCKER whose fix brings the code back to what Plan.md specifies, or corrects a defect (a bug, a stub, a validation gap) without changing the method, orchestrator sends revision request to research-executor. A `Stub Detection` or `Validation Gap` STOP comes here too; other STOP categories route as § Gate Status Translation says. A methodology BLOCKER (a code-reviewer STOP with Issue Category `Methodology Conflict`) is not revised: STOP and escalate to the user (§ Stage 5-8 Composite Execution Pattern).

```python
Agent({
    description: "Revision: Stage {N} Step {step} - {task_name}",
    prompt: """**BASE_DIR:** {BASE_DIR}
**PROJECT_DIR:** {PROJECT_DIR} (absolute project path; build every data and output path in the script from it)
All relative paths in referenced files resolve from BASE_DIR.

**REVISION REQUEST**

**Original Script:** {PROJECT_DIR}/scripts/stage{N}_{type}/{step}_{task-name}.py (or .R)
**Current Final Version:** {PROJECT_DIR}/scripts/stage{N}_{type}/{step}_{task-name}_{suffix}.py (or .R)

**METHODOLOGY CONTEXT (from Plan.md):**
{relevant methodology decisions, coded values, research outcomes for this task}

**TASK SPECIFICATION (from Plan_Tasks.md):**
{the <task> XML block for this specific task, including <action>, <verify>, <done>}

**QA BLOCKER Issue:**
- **Type:** {issue_type}
- **Description:** {issue_description}
- **Location:** {location_in_code}
- **Suggested Fix:** {suggested_fix_from_code_reviewer}

**Instructions:**
1. Create new versioned script: {step}_{task-name}_{next_suffix}.py (or .R)
2. Apply fix for the BLOCKER issue while maintaining alignment with Plan.md methodology
3. Execute with full validation per the task's <verify> block
4. Append execution log
5. Return execution report

**Do NOT modify prior script versions** — they serve as audit trail.

**OUTPUT FORMAT:**
[Standard research-executor output format]
""",
    subagent_type: "research-executor"
})
```

---

## Error Handling in Invocations

> For retry limits and the escalation decision table, see the "Error Recovery" section below.

### Retry Pattern

```python
# If subagent returns error, retry with clarification
Agent({
    description: "Stage [N] - Retry",
    prompt: """Previous attempt encountered: {error_description}

**CORRECTIVE CONTEXT:**
{what_went_wrong}
{how_to_fix}

Please retry the task with this correction.

[Original task specification]""",
    subagent_type: "[agent-name]"
})
```

### Escalation Pattern

After 2 failed attempts, stop and tell the user, using `agent_reference/ERROR_RECOVERY.md` § Escalation Format > Standard Escalation Message: what happened, what was tried, the options with their pros and cons, and a recommendation. Write it in plain language per the Plain-Language Rule table in `SKILL.md` § User-Facing Communication Standards: describe the step by its activity (e.g., "cleaning the enrollment data"), not its stage number.

---

## Context Passing Requirements by Stage Transition

**Principle:** Each stage transition requires explicit context handoff. The orchestrator MUST pass accumulated findings, not just file paths.

### Stage 2 → Stage 3 Context

| Context Item | Source | Required In Stage 3 Prompt |
|--------------|--------|---------------------------|
| Candidate datasets identified | Stage 2 output | YES — canonical dataset identifiers (e.g., mirror paths for education data) |
| Variables flagged for deep-dive | Stage 2 output | YES — with reasons for flagging |
| Research question | Stage 1 | YES — verbatim |
| Years needed | Pre-flight Scope Check / Stage 2 | YES — exact range |
| Geographic scope | Pre-flight Scope Check | YES — states or national |
| User-referenced materials | Pre-flight Scope Check | YES — paths (and any digest path) bearing on this source, or "none" |

### Stage 3 → Stage 4 Context

| Context Item | Source | Required In Stage 4 Prompt |
|--------------|--------|---------------------------|
| Source caveats | Stage 3 output (per source) | YES — all caveats, not summary |
| Coded value mappings | Stage 3 output | YES — complete table |
| Suppression patterns | Stage 3 output | YES — typical rates |
| Cross-state comparability | Stage 3 output | YES — assessment |
| Confidence levels | Stage 3 output | YES — LOW items especially |
| Confirmed pre-flight scope | Pre-flight Scope Check (`output/preliminary_notes/{date}_preflight_scope-check.md`) | YES — `CONFIRMED PRE-FLIGHT SCOPE:` field: scope as confirmed, what changed, decisions with the user's answers |
| User-referenced materials | Pre-flight Scope Check | YES — `USER-REFERENCED MATERIALS:` field: paths in the project folder (and the digest path), or "none" |

### Stage 4 → Stage 5 Context

| Context Item | Source | Required In Stage 5 Prompt |
|--------------|--------|---------------------------|
| Query specifications | Plan.md + Plan_Tasks.md | YES — exact dataset identifier, years, filters |
| Expected row counts | Plan_Tasks.md | YES — ranges |
| Risk Register items for fetch | Plan.md | YES — relevant risks |
| Output file paths | Plan_Tasks.md | YES — explicit paths |

### Stage 5 → Stage 6 Context

**Note:** Stage 5 may produce multiple scripts (one per fetch task). All outputs must be passed forward.

| Context Item | Source | Required In Stage 6 Prompt |
|--------------|--------|---------------------------|
| All raw data file paths (one per fetch script) | Stage 5 output | YES — exact paths for every file produced |
| CP1 validation results for ALL Stage 5 scripts | Stage 5 output | YES — what passed/failed per script |
| QA1 status for EACH Stage 5 script separately | Stage 5 QA | YES — per-script QA outcomes |
| Source caveats | Stage 3 → Plan.md | YES — inlined, not just referenced |
| Coded value handling rules | Plan.md | YES — complete specification |
| Suppression tolerance | Plan.md | YES — BLOCKER/WARNING thresholds |

### Stage 6 → Stage 7 Context

**Note:** Stage 6 may produce multiple scripts (one per clean task). All outputs must be passed forward.

| Context Item | Source | Required In Stage 7 Prompt |
|--------------|--------|---------------------------|
| All processed data file paths (one per clean script) | Stage 6 output | YES — exact paths for every file produced |
| CP2 validation results for ALL Stage 6 scripts | Stage 6 output | YES — suppression rates per script |
| QA2 status for EACH Stage 6 script separately | Stage 6 QA | YES — per-script QA outcomes |
| EDA findings (for 7.2+) | Stage 7.1 output | YES — distributions, quality issues |
| Prior transformation results (for 7.N) | Stage 7.(N-1) output | YES — row counts, changes, findings |
| Invariants to maintain | Prior transformations | YES — accumulated constraints |

### Code-Reviewer Context (All QA Invocations)

| Context Item | Source | Required In QA Prompt |
|--------------|--------|----------------------|
| Script path | research-executor | YES — exact path |
| Plan expectations | Plan.md + Plan_Tasks.md (inlined) | YES — row counts, tolerances |
| Research Outcome contribution | Plan.md | YES — what this task enables |
| Risk Register items | Plan.md | YES — relevant mitigations |
| QA tolerance thresholds | Plan.md (QA Tolerance Decisions section) | YES — BLOCKER/WARNING criteria |
| Prior QA findings | Accumulated from prior scripts | YES — WARNING items to track |

---

## Confidence Level Defaults

### Standard Confidence Assignments

| Source Type | Default Confidence | Upgrade Path | Downgrade Path |
|-------------|-------------------|--------------|----------------|
| Official NCES documentation | HIGH | — | Contradicted by actual data |
| Skill reference content | HIGH | — | Outdated info discovered |
| Data exploration results | MEDIUM | Multiple mirrors confirm | Single dataset identifier, unclear docs |
| Inferred from data patterns | LOW | Documentation confirms | Contradicted by test |
| User-provided information | HIGH | — | Conflicts with official sources |

### Confidence Requirements

**HIGH confidence findings:** Proceed normally.

**MEDIUM confidence findings:** Document the uncertainty and proceed with caution.

**LOW confidence findings:** MUST have resolution path before proceeding:
1. Re-run discovery with refined parameters
2. Escalate to user for guidance
3. Document risk acceptance explicitly in Plan.md (if during planning) or STATE.md Runtime Risks (if during execution)

**LOW confidence items cannot be silently ignored.**

### Reporting Confidence

Every subagent return MUST include confidence assessment:

```markdown
**Confidence Assessment:**
| Finding | Confidence | Rationale |
|---------|------------|-----------|
| Mirror file exists | HIGH | Direct download successful |
| Variable meaning | MEDIUM | Skill reference, not NCES docs |
| Suppression threshold | LOW | Inferred from patterns |

**Overall Confidence:** [MEDIUM]
**LOW Confidence Items Requiring Resolution:**
- Suppression threshold: Need to verify with source documentation
```

---

## Learning Signal Protocol

### Purpose

Every analysis produces learning opportunities beyond its immediate findings:
- Data download behaviors not documented elsewhere
- Data quality patterns
- Methodology insights
- Performance optimizations
- Common pitfalls to avoid

Capturing these lessons prevents repeated mistakes and accelerates future analyses. **Equally important:** incorporating and formalizing prior learnings ensures we don't repeat past mistakes.

### Learning Signal Format

Every agent this pipeline dispatches includes a lightweight Learning Signal field in its output:

```
**Learning Signal:** [Category: Access|Data|Method|Perf|Process] — [One-line insight] | or "None"
```

**Examples:**
- `**Learning Signal:** CCD enrollment value codes were not as expected in the codebook; codes needed to be explicitly examined for progress to continue`
- `**Learning Signal:** Data — MEPS poverty rates have 15% suppression in rural counties (higher than Plan.md estimate of 5%)`
- `**Learning Signal:** None`

### Accumulation Flow

```
Agent returns with Learning Signal
     ↓
Orchestrator extracts signal (if not "None")
     ↓
Orchestrator appends to STATE.md "Pending Learning Signals" buffer
     ↓
At next flush trigger → orchestrator appends buffered signals to LEARNINGS.md
     ↓
Clear STATE.md buffer
```

### Flush Triggers

The orchestrator writes buffered signals to LEARNINGS.md at these points:

1. **Phase boundary** — end of Phase 1 (after Stage 3/3.5), Phase 2 (after Stage 4.5), Phase 3 (after Stage 6-QA), Phase 4 (after Stage 10 — QA Aggregation)
2. **After blocker resolution** — a resolved BLOCKER often yields the richest learnings
3. **After debugging session** — debugger agent's Prevention section feeds learnings directly
4. **At utilization gates** (ELEVATED, HIGH) — ensures learnings are persisted before potential session end

*Not* at every stage transition or every subagent return — that would be too frequent and disruptive.

### Flush Operation

What the orchestrator does at each flush:

1. Read pending signals from STATE.md buffer
2. Categorize each signal into the appropriate LEARNINGS.md section (Access/Data Gotchas, What Worked Well, Surprises, etc.)
3. Append as a quick-capture entry with stage number and timestamp
4. Clear the STATE.md buffer
5. This should take ~1 minute of orchestrator time, not a subagent invocation

Stage-specific creation triggers (Stage 4 skeleton creation) and consolidation steps (Stage 12) live in their respective phase files (`WORKFLOW_PHASE2_PLANNING.md` and `WORKFLOW_PHASE5_SYNTHESIS.md`). See `WORKFLOW_PHASE5_SYNTHESIS.md` > "Lessons Learned Consolidation" for the LEARNINGS.md template and Stage 12 consolidation procedure.

---

## Quality & Validation Framework

This section consolidates all quality standards, validation checkpoints, enforcement gates, and stage-specific verification checklists.

### Confidence Levels

See the "Confidence Level Defaults" section above for the complete three-tier system (HIGH/MEDIUM/LOW), source-type defaults, and upgrade/downgrade paths.

**Critical rule:** LOW confidence items cannot be silently ignored — they require resolution before proceeding.

### Truth Hierarchy for Data Interpretation

When interpreting data values and resolving discrepancies between sources, apply this priority:

| Priority | Source | Rationale | Example |
|----------|--------|-----------|---------|
| 1 (highest) | **Actual data file** (parquet) | What you observe IS the truth | Column has values 1-7, not 1-5 as documented |
| 2 | **Live codebook/metadata** (.xls in mirror) | Authoritative documentation; may lag behind data | Codebook says "1=Regular, 2=Special Ed" (e.g., in education) |
| 3 (lowest) | **Archived skill docs** (e.g., variable-definitions.md) | Summarized; convenient but may drift | Skill says "values 1-5" but codebook says "1-7" |

**Application Rules:**
- When skill docs contradict observed data → trust the data, flag the discrepancy
- When codebook contradicts observed data → trust the data, but investigate (codebook may describe a different year)
- When skill docs contradict codebook → trust the codebook, and record the discrepancy as a Learning Signal (Data) so the Stage 12 System Update Action Plan in LEARNINGS.md carries the skill fix to a Framework Development session; Full Pipeline does not edit skills
- For education domain: Codebook URLs are cataloged in `datasets-reference.md` (codebook column); use `get_codebook_url()` in `fetch-patterns.md` to construct download URLs. Other domains will use analogous structures in their domain query skill.
- See also: `.claude/agents/data-ingest.md` Data Primacy table for the same hierarchy applied during data onboarding

### Validation Checkpoints

| Checkpoint | When | Validates | STOP Condition |
|------------|------|-----------|----------------|
| **CP1** | After data fetch | Shape, expected rows, required columns, year coverage, missingness, data lag, flagged years | Empty data, a required column absent, >90% missing in a required column |
| **CP2** | After cleaning | Row loss, suppression rate, coded values handled | Suppression above the Plan threshold (50% default), >90% row loss |
| **CP3** | After transformation | Row counts, join validation, no data loss | >90% row loss, unexpected NAs |
| **CP4** | After analysis & visualization (Stage 8) | Analysis outputs valid, critical nulls, figures, correct data source | Non-finite estimates, missing or empty figures, script reading `data/raw/` |

**CP4 Detail:** CP4 is **Stage 8 Post-Analysis & Visualization Validation**, embedded in the analysis and visualization scripts (`checkpoint:auto`). It validates:
- **CP4.1:** Analysis outputs valid — results object exists, estimates finite, convergence flag `TRUE` when the estimator reports one, n adequate against the Plan's minimum, expected sign only where the Plan pre-registers it
- **CP4.2:** Critical nulls — no nulls in the analysis frame's outcome/key columns
- **CP4.3:** Figures — every figure the task promised exists, is non-empty, and has been visually inspected via the **Read tool**
- **CP4.4:** Correct data source — the script loads `data/processed/` (or the Plan's named input), not `data/raw/`

**CP4 STOP Conditions:** Non-finite estimates, a convergence flag reporting failure, nulls in the analysis frame's outcome/key columns, a promised figure missing or empty, or the script reading from `data/raw/`.

**Report completeness and Plan consistency are NOT CP4.** Report sections, Plan-commitment consistency, Research Outcome coverage, Hypothesis assessment, and notebook render success belong to **Pre-Output Verification (Stage 12)** in `agent_reference/VALIDATION_CHECKPOINTS.md` — an agent checklist owned by data-verifier and classified `human-verify`, not script-embedded checkpoint code.

**Pre-Output Verification STOP Conditions:** Missing Executive Summary, missing Key Findings, any Research Outcome not addressed, major deviation from Plan.md methodology.

When a CP result needs interpreting (a reported FAILED or WARNING status; in the execution log, a FAILED or PASSED WITH WARNINGS verdict), or when you review the data-verifier's Pre-Output Verification (Stage 12), read these sections of `agent_reference/VALIDATION_CHECKPOINTS.md`, each in full: § Per-Check Outcome Vocabulary, § Count Line and Verdict Line, and that checkpoint's own section (§ CP1: Post-Fetch Validation through § CP4: Post-Analysis & Visualization Validation, § Join-Specific Validation, or § Pre-Output Verification (Stage 12)). The trigger loads those sections, not the file: `VALIDATION_CHECKPOINTS.md` is a catalogue of per-checkpoint sections holding every checkpoint's Python and R code templates, which the executing agents write their checkpoint code from; the orchestrator does not.

### QA Checkpoints (Secondary Validation)

In addition to CP checkpoints (embedded in code), **QA checkpoints** provide independent secondary validation after each script execution in Stages 5-8.

| Checkpoint | Stage | Validates | BLOCKER Threshold |
|------------|-------|-----------|-------------------|
| **QA1** | After fetch (5) | Schema correctness, ID uniqueness, distributions | Data integrity compromised |
| **QA2** | After clean (6) | Coded value handling, filtering logic, methodology | Cleaning logic invalid |
| **QA3** | After transform (7) | Join cardinality, aggregation logic, derived columns | Transformation produces wrong results |
| **QA4a** | After analysis (8.1) | Statistical validity, assumption checks, result interpretation | Analysis methodology invalid or results unreliable |
| **QA4b** | After viz (8.2), and after an 8.1 analysis script that also draws figures | Figure existence, data source accuracy, labeling, visual inspection via **Read tool** | Visualization misleading or incorrect |

**Key Difference:** CP checkpoints catch **operational failures** (empty data, missing required columns, excess missingness or row loss). QA checkpoints catch **logical errors** (wrong methodology, misinterpretation).

**Severity Levels:**
- **BLOCKER:** Revision required (max 2 attempts, then escalate); a methodology BLOCKER (a code-reviewer STOP with Issue Category `Methodology Conflict`) escalates to the user immediately, with no revision (RULE 4)
- **WARNING:** Log for Stage 10 aggregation, proceed
- **INFO:** Log only, proceed

See `agent_reference/QA_CHECKPOINTS.md` for complete definitions and `.claude/agents/code-reviewer.md` for the QA agent protocol.

### Stage Gates (Cannot Proceed Without)

Forcing functions are mandatory design interventions that **prevent** poor practices as the main enforcement mechanism for core design principles and values. The following gates CANNOT be bypassed.

| Gate | Transition | Requires | Enforcement |
|------|------------|----------|-------------|
| G1 | 1 → 2 | Pre-flight Scope Check confirmed (includes mode confirmation) | Cannot invoke Stage 2 subagent |
| G2 | 2 → 3 | ≥1 dataset identified, key variables flagged | Cannot invoke source deep-dive |
| G3 | 3 → 3.5 | All flagged variables investigated, coded values documented, suppression patterns identified | Cannot invoke research-synthesizer |
| G3.5 | 3.5 → 4 | Synthesis complete, cross-source conflicts resolved, **User confirmed PSU1** | Cannot create Plan documents without user PSU1 confirmation |
| **G4** | **4 → 4.5** | **Plan.md + Plan_Tasks.md created AND STATE.md created AND LEARNINGS.md skeleton created** | **Cannot invoke plan-checker** |
| **G4.5** | **4.5 → 5** | **plan-checker returned PASSED or PASSED_WITH_WARNINGS, User confirmed PSU2** | **Cannot begin data acquisition without user PSU2 confirmation** |
| G5 | 5 → 6 | CP1 PASSED per script, data saved to data/raw/, code-reviewer separately invoked per script immediately after completion, every QA1 ∈ {PASSED, WARNING} | Cannot proceed to cleaning |
| G6 | 6 → 7 | CP2 PASSED per script, suppression <50%, data saved to data/processed/, code-reviewer separately invoked per script immediately after completion, every QA2 ∈ {PASSED, WARNING}, **User confirmed PSU3** | Cannot proceed to transformation without user PSU3 confirmation |
| G7 | 7 → 8 | All transformations CP3 PASSED per script, code-reviewer separately invoked per script immediately after completion, every QA3 ∈ {PASSED, WARNING} | Cannot proceed to analysis and visualization |
| G8 | 8 → 9 | Analyses and visualizations complete, CP4 PASSED per script, code-reviewer separately invoked per 8.1 script (QA4a, plus QA4b when the analysis script also draws figures) and per 8.2 script (QA4b) immediately after each completes, every QA4a and QA4b ∈ {PASSED, WARNING} | Cannot assemble notebook |
| G9 | 9 → 10 | Python: the `marimo check --select MB` structural gate passes, `marimo export html` exits 0, and the Step 6c verbatim fidelity check passes all four conditions, each with quoted evidence: clean decompile, bundle count and decompiled paths equal to the Step 1 final-version inventory, every numstat line an ` => dev/null}` line or the single `MANIFEST.md` line (notebook-assembler Step 6). R: `quarto render` validates assembly and explicitly enabled previews without re-running archived Stage 5-8 scripts. All scripts are represented with code + execution logs. integration-checker (Stage 9 scope) returned CONNECTED or WARNING-only | Cannot run QA aggregation |
| G10 | 10 → 11 | QA findings aggregated, all BLOCKERs resolved, all WARNINGs documented, **User confirmed PSU4** | Cannot generate report without user PSU4 confirmation |
| G11 | 11 → 12 | Report complete with all sections and figure references; integration-checker (Stage 11 scope) returned CONNECTED or WARNING-only | Cannot run final review |
| G12 | 12 → Delivery | integration-checker (full scope) returned CONNECTED or WARNING-only before data-verifier ran, Final Review verification PASSED, all commitments fulfilled, LEARNINGS.md consolidated with System Update Action Plan, cross-artifact coherence verified | Cannot deliver |

**Gate G4 Enforcement:** Plan-checker (Stage 4.5) CANNOT be invoked without all four files: Plan.md, Plan_Tasks.md, STATE.md (`agent_reference/STATE_TEMPLATE.md`), and LEARNINGS.md (`agent_reference/WORKFLOW_PHASE5_SYNTHESIS.md`). If any are missing, create before proceeding. After plan-checker returns, the orchestrator MUST present PSU2 to the user and wait for confirmation before proceeding to Stage 5.

**Gate G4.5 Enforcement:** plan-checker MUST be invoked and return PASSED or PASSED_WITH_WARNINGS, and the user MUST confirm PSU2. If ISSUES_FOUND, revise Plan documents (max 2 attempts) then escalate. Update STATE.md "Plan Validation" section with the result before proceeding. See Stage 4.5 in `agent_reference/WORKFLOW_PHASE2_PLANNING.md` for the invocation pattern.

**Plan-revision STATE.md synchronization (blocking, within G4.5):** After ANY accepted plan revision (data-planner returns REVISION_COMPLETE and plan-checker subsequently passes), and BEFORE proceeding to Stage 5, the orchestrator MUST resynchronize STATE.md to the revised Plan_Tasks.md — STATE.md's Transformation Progress table, Current Position paths, and Next Actions were first populated at Stage 4 from the *original* Plan_Tasks.md and go stale when a revision changes task count, task names, script paths, or wave structure. The orchestrator MUST: (1) read the revised Plan_Tasks.md frontmatter (`total_tasks`, `total_waves`, `version`) and full Task Index; (2) rebuild the Transformation Progress table from the revised Task Index (carry a completed row over ONLY when the task still exists with an identical script path — otherwise re-derive its status honestly); (3) update the Plan.md / Plan_Tasks.md paths, Current Stage, and Next Actions; (4) validate mechanically — Transformation Progress row count == `total_tasks`, every script path appears in both the table and the revised Task Index (and vice versa), and no STATE.md content references a superseded plan-version filename. If any check fails, STOP — fix STATE.md and re-validate before any further dispatch. This is a blocking condition inside Gate G4.5, not a new gate. See the STATE.md Update Gates table row "Plan revised (accepted revision)" for field-level detail.

**CRITICAL:** Gate G4.5 requires POSITIVE confirmation that plan-checker was invoked and returned PASSED or PASSED_WITH_WARNINGS. If plan-checker was never invoked, the gate condition is NOT satisfied. Update STATE.md "Plan Validation" section with the result before proceeding to Stage 5. Additionally, after plan-checker returns PASSED or PASSED_WITH_WARNINGS, the orchestrator MUST present PSU2 (Phase Status Update) to the user including the plan-checker result, a Plan.md summary, and the exact filepath to Plan.md for the user's deeper inspection. Stage 5 CANNOT begin until the user confirms PSU2.

**Gate G5-G8 Enforcement (Per-Script QA Invocation):** Gates G5-G8 require POSITIVE confirmation that code-reviewer was **separately invoked to review each individual script immediately after that script completed execution** — not batched at stage end. "Immediately" means: before any script that depends on it begins, so before the next wave starts (same-wave scripts run in parallel, each reviewed on its own; § Wave-Based Parallel Execution). "Separately" means: one code-reviewer invocation per script, not one invocation reviewing multiple scripts. Starting a later-wave script before every script it depends on has its own recorded review, or reviewing several scripts in one code-reviewer call, does **NOT** satisfy these gates — the QA must be interleaved with execution so that each script's QA findings can inform whether to proceed, revise, or stop before any script that builds on it runs. If code-reviewer was never invoked for a given script, that script's QA status is NOT_RUN and the gate is NOT satisfied. For Gate G8, QA4a (statistical analysis) and QA4b (visualization) are independent, separate invocations: each 8.1 analysis script gets QA4a, plus QA4b when it also draws figures, and each 8.2 visualization script gets QA4b. See the **Stage 5-8 Composite Execution Pattern** for the complete flow.

### Per-Script QA Enforcement Protocol

**To prevent batching, the orchestrator MUST maintain a QA invocation discipline throughout Stages 5-8.**

**Rule: One script in, one QA out, before any script that depends on it begins.** Same-wave scripts run in parallel (at most 5 in flight), each with its own review; the next wave starts only when every script in the current wave has its review recorded.

Concretely:
1. **Before dispatching research-executor for a task**, verify that every task it depends on (every task in earlier waves) has a completed code-reviewer QA entry in STATE.md (with QA status ∈ {PASSED, WARNING} or BLOCKER resolved via revision). If any of those entries is missing or incomplete, STOP and invoke code-reviewer for that script first.
2. **STATE.md Transformation Progress table** must have one row per script, and each row must include: script path, CP status, QA status, and QA script path. A row with QA status = `NOT_RUN` blocks the next wave.
3. **Self-check before every research-executor dispatch**: *"Which earlier scripts does this task depend on? Did I invoke code-reviewer separately for each? Is each QA status recorded in STATE.md? If any answer is no → invoke code-reviewer NOW, do not invoke research-executor."*

**Why this matters:** Batching QA to the end of a stage means errors in script 1 propagate silently through every script that builds on it — producing compounding data corruption that is far harder to diagnose and fix. Per-script QA catches errors at the source, before they cascade.

### Gate Status Translation

Agents use domain-specific status vocabularies. The orchestrator translates these to gate vocabulary:

| Agent | Agent Output | Gate Interpretation |
|-------|-------------|-------------------|
| **research-executor** | PASSED | Proceed to QA |
| | WARNING | Proceed to QA (log warning) |
| | FAILED | Attempt versioned fix or STOP |
| **code-reviewer** (read the `QA{n} RESULT:` line; `Status` says only whether the review itself ran to completion) | `QA{n} RESULT: PASSED` | QA = PASSED |
| | `QA{n} RESULT: WARNING` | QA = WARNING |
| | `QA{n} RESULT: BLOCKER` whose fix would change Plan.md's method, make a methodology choice Plan.md does not settle, or restore a Plan.md choice that itself looks wrong for the data (the reviewer should have raised a STOP) | Treat as `Methodology Conflict`: STOP, escalate to user immediately (RULE 4) |
| | `QA{n} RESULT: BLOCKER`, any other (the fix brings the code back to what Plan.md specifies, and Plan.md's choice does not itself look wrong for the data; or it corrects a defect such as a bug, a stub or a validation gap without changing the method) | QA = BLOCKER (revision flow, max 2) |
| | `CODE-REVIEWER STOP`, Issue Category `Methodology Conflict` | STOP, escalate to user immediately (RULE 4) |
| | `CODE-REVIEWER STOP`, any other Issue Category | Route by category: `Data Corruption` → debugger (code-reviewer recommends it); `Stub Detection` or `Validation Gap` → revision flow, max 2; `Missing Output` → confirm with research-executor that the script ran and saved its outputs, then re-dispatch the review |
| | Neither a `QA{n} RESULT:` line nor a `CODE-REVIEWER STOP` block (the review did not finish) | Re-dispatch the review; the script's QA Status stays `PENDING` |
| **data-planner** | COMPLETE | Proceed to Stage 4.5 |
| | CONTINUATION | Read partial Plan.md on disk, invoke fresh data-planner in continuation mode |
| | REVISION_COMPLETE | Re-invoke plan-checker; after it passes, resynchronize STATE.md to the revised Plan_Tasks.md (rebuild Transformation Progress + Current Position + Next Actions + plan paths) and run the parity validation before Stage 5 (see Gate G4.5 Enforcement) |
| | BLOCKED | Escalate to user |
| **plan-checker** | PASSED | Plan-checker condition met; G4.5 = SATISFIED once the user confirms PSU2 |
| | PASSED_WITH_WARNINGS | Plan-checker condition met (log warnings); G4.5 = SATISFIED once the user confirms PSU2 |
| | ISSUES_FOUND | G4.5 = NOT SATISFIED (revision needed) |
| **data-verifier** | PASSED | G12 = SATISFIED |
| | ISSUES_FOUND (severity: WARNING) | Log, proceed with caveats |
| | ISSUES_FOUND (severity: BLOCKER) | G12 = NOT SATISFIED |
| **source-researcher** | COMPLETE | Proceed to next source or Stage 3.5 |
| | COMPLETE_WITH_WARNINGS | Log warnings; proceed |
| | BLOCKED | Escalate |
| **notebook-assembler** | PASSED | G9 = SATISFIED |
| | WARNING | Log; proceed |
| | BLOCKER | Revision needed |
| **report-writer** | COMPLETE | G11 = SATISFIED |
| | COMPLETE_WITH_GAPS | G11 = SATISFIED (log gaps) |
| | BLOCKED | G11 = NOT SATISFIED |
| **research-synthesizer** | PASSED | G3.5 = SATISFIED, proceed to Stage 4 |
| | WARNING | G3.5 = SATISFIED (log warnings for Plan documents) |
| | BLOCKER | G3.5 = NOT SATISFIED (resolve or escalate) |
| **debugger** | RESOLVED (Bug fix) | Apply fix, re-run task via research-executor |
| | RESOLVED (Data quality) | Document limitation, adjust scope |
| | RESOLVED (Transient) | Retry operation |
| | RESOLVED (Methodology) | STOP, escalate to user (RULE 4) |
| | UNRESOLVED | Escalate to user with hypothesis log |
| | PARTIAL | Escalate with findings; user decides |
| **integration-checker** | CONNECTED | Wiring checkbox satisfied (G9, G11, or G12 depending on stage; templates in WORKFLOW_PHASE4 § Stage 9 and WORKFLOW_PHASE5 § Stages 11-12) |
| | ISSUES FOUND (severity: WARNING) | Log; proceed with caveats |
| | ISSUES FOUND (severity: BLOCKER) | Gate NOT SATISFIED; revision needed |
| **data-ingest** | COMPLETE | Present integration guidance; offer skill registration |
| | COMPLETE_WITH_WARNINGS | Present discrepancies; user review required |
| | BLOCKED | Present STOP condition; await user resolution |

### STATE.md Update Gates

| Event | Required STATE.md Field Updates |
|-------|--------------------------------|
| Stage N starts | Current Stage → N |
| Plan revised (accepted revision — data-planner REVISION_COMPLETE and plan-checker passed; also applies to Revision and Extension mode new-version creation) | Rebuild Transformation Progress table from the revised Plan_Tasks.md Task Index (carry a completed row over ONLY when the task still exists with an identical script path; otherwise re-derive status honestly) + Current Position (Plan.md / Plan_Tasks.md paths, Current Stage) + Next Actions. Then validate: row count == Plan_Tasks `total_tasks`; every script path present in both the table and the Task Index; no reference to a superseded plan-version filename. Blocking within Gate G4.5 — STOP and fix if any check fails |
| Checkpoint passes | Checkpoint Status table |
| QA completes | Transformation Progress (the script's QA Status column) + Checkpoint Status > Secondary Validation (QA1-QA4b) |
| Blocker encountered | Blockers section + Next Actions |
| Key decision made | Key Decisions Made table |
| Risk identified during execution | Runtime Risks table |
| QA finding recorded | QA Findings Summary (incremental — append per code-reviewer return) |
| Analysis result addresses hypothesis | Hypothesis Assessment Progress table |
| notebook-assembler Stage 9 return | Final Review Log > Stage 9 Check Evidence (results quoted from the return, with run timestamps) |
| integration-checker return (Stages 9, 11, 12) | Final Review Log > Integration Checks (one row per return; Stage 12 also persisted in full at `output/preliminary_notes/{date}_stage12_integration-check.md`) |
| Final review completed | Final Review Log |
| Context Utilization reaches ELEVATED (See CLAUDE.md § Context & Session Health > Context Quality Curve for the trigger points of each threshold profile) | Context Snapshot section |
| User checkpoint (PSU) presented | Current Position > Status → Awaiting Confirmation + Next Actions > Pending User Input (names the checkpoint). PSU1 is presented before STATE.md exists, so it has no row here |
| Phase boundary reached (PSU confirmed) | Current Position > Status → In Progress + Key Decisions Made (one row per checkpoint: the user's answer and any changes they asked for) + Current Position (Current Phase). PSU1 is confirmed before STATE.md exists: add its row when STATE.md is created at Stage 4 |
| Phase completes | Session History (if multi-session) |

### Automatic STOP Conditions

These conditions trigger an immediate STOP with escalation to user. See `agent_reference/BOUNDARIES.md` for complete specifications.

| Condition | Stage | Action |
|-----------|-------|--------|
| Data acquisition returns empty data (mirror or API) | Stage 5 | STOP, report to user, await guidance |
| Suppression rate >50% | Stage 6 | STOP, report issue, propose alternatives |
| Domain governance rule violation (e.g., cross-state assessment comparison in education) | Stage 6 | BLOCK with explanation (never valid) |
| Row count drops >90% after transformation | Stage 7 | STOP, verify transformation logic |
| **QA BLOCKER after 2 revisions** | 5-QA to 8-QA | STOP, escalate to user |
| **QA methodology conflict** (code-reviewer STOP, Issue Category `Methodology Conflict`) | 5-QA to 8-QA | STOP, escalate immediately |
| Python `marimo check` gate or `marimo export html` failure, or R `quarto render` assembly/enabled-preview error, after 2 fix attempts | Stage 9 | STOP, report format-specific error details; do not treat either check as archived analysis re-execution |
| Stage 9 display input (Parquet or figure, Marimo or Quarto) is missing and its file name appears in the project's `*_Plan.md`, `*_Plan_Tasks.md` or `STATE.md` (a declared Stage 5-8 output) | Stage 9 | STOP, report as an upstream failure; never drop the display to hide it |
| Marimo execution log cannot be archived verbatim (contains both `'''` and `"""`; 6a MB001 inside a log accordion whose delimiter is already the one notebook-assembler's Grep rule picks; a log character the bundle or decompiler cannot carry, such as a NUL byte, invalid UTF-8 or a carriage return inside a line; or the 6c fidelity check fails: a script missing, an earlier or renamed version archived, or an archived script differing from its original) | Stage 9 | STOP, report script path and the offending line, missing or extra paths, or differing lines; never edit archived code or log text |
| Data unavailable in configured data source | Stage 2-3 | STOP, escalate immediately |
| LOW confidence finding unresolved | Any | Cannot proceed |

**STOP/Escalation Format:** See `agent_reference/ERROR_RECOVERY.md` § Escalation Format for the detailed format. At minimum, include: what happened, what was tried, options with pros/cons, and a recommendation.

### Verification Checklists by Stage

Each stage has a verification checklist for subagent output. These checklists live in the appropriate `agent_reference/WORKFLOW_PHASE*.md` file alongside the stage they verify:

| Stages | Checklist Location |
|--------|--------------------|
| 2, 3, 3.5 | `WORKFLOW_PHASE1_DISCOVERY.md` > Verification Checklists |
| 4 | `WORKFLOW_PHASE2_PLANNING.md` > Verification Checklists |
| 5, 6 | `WORKFLOW_PHASE3_ACQUISITION.md` > Verification Checklists |
| 7, 8.1, 8.2 | `WORKFLOW_PHASE4_ANALYSIS.md` > Verification Checklists |
| 12 | `WORKFLOW_PHASE5_SYNTHESIS.md` > Verification Checklists |

**When to check:** After each subagent returns, apply the relevant stage's checklist before proceeding.

---

## Pipeline-Specific Behavioral Boundaries

These supplement the universal boundaries in `CLAUDE.md` (Boundaries & Safety) and `agent_reference/BOUNDARIES.md`. See `BOUNDARIES.md` > Full Pipeline Mode for complete specifications.

**Always Do:**
- Validate data at every checkpoint (CP1-CP4)
- Create Plan.md + Plan_Tasks.md before data acquisition
- Complete Final Review (Final Review) before delivery
- Generate all three deliverables (Plan.md + Plan_Tasks.md, Marimo/Quarto Notebook, Report)
- Follow the Inline Audit Trail (IAT) protocol for all scripts (`agent_reference/INLINE_AUDIT_TRAIL.md`)
- Include validation assertions in every script (the notebook archives them verbatim)
- Update STATE.md with all runtime decisions, deviations, and findings
- Surface online verification as an option at Phase 1 (Discovery) when skill-sourced data source details may have changed, at Phase 2 (Planning) when methodology choices draw on general knowledge beyond loaded skills, and at checkpoints when presenting findings that rest on skill-derived assumptions

**Never Do:**
- Skip any protocol or checkpoint
- Deliver without all three deliverables
- Proceed without resolving LOW confidence findings
- Proceed after STOP condition without user guidance

**QA-Specific Boundaries (Stages 5-8):**
- Invoke code-reviewer after EVERY script execution
- Create QA scripts in `scripts/cr/` for every reviewed script
- Address BLOCKER issues via revision before proceeding; escalate a methodology BLOCKER to the user immediately instead
- Never skip QA for "simple" scripts
- Never modify scripts after QA review (create new version instead)
- Never allow code-reviewer to directly modify execution scripts

See `agent_reference/BOUNDARIES.md` > QA-Specific Boundaries for complete specifications.

### Autonomous Deviation Rules (Quick Reference)

When executing Plan_Tasks.md tasks, the agent MAY deviate **without asking** for these categories:

| Rule | Category | Action |
|------|----------|--------|
| RULE 1 | Bug fixes (syntax, types, imports) | Fix immediately, document |
| RULE 2 | Critical functionality (validation, error handling) | Add silently, document |
| RULE 3 | Blocking issues (missing deps, wrong paths) | Fix immediately, document |
| RULE 4 | Methodological changes | STOP, escalate to user |
| RULE 5 | QA-triggered revisions (non-methodology BLOCKER) | Fix via versioned revision, re-QA |

**Always Requires Approval:** Scope expansion, methodology changes, removing validation, skipping checkpoints.

See `agent_reference/BOUNDARIES.md` for complete boundary specifications and deviation decision tree.

---

## Context Utilization Management

The orchestrator receives actual context utilization via the `context-reporter` hook. See `CLAUDE.md` > "Context & Session Health" for the complete threshold table and required actions.

**Cardinal Principle: Quality Is the Invariant, Session Restart Is the Pressure Valve.** Never sacrifice subagent prompt completeness, skip checklist items, or reduce inlined context to "save space." Use session restart (see `{SKILL_REFS}/session-recovery.md`) as the relief mechanism instead.

---

## Session State Management

### STATE.md (MANDATORY for Full Pipeline)

**STATE.md is REQUIRED for all Full Pipeline analyses.** This is not optional.

**Why Mandatory:**
- Enables session recovery if context is exhausted
- Provides checkpoint history for debugging
- Creates audit trail of progress and decisions
- Allows handoff between sessions
- Prevents context exhaustion without recovery path

**Creation Trigger:**
- **Create:** At Stage 4 (Plan creation) — IMMEDIATELY after Plan.md file is written
- **Gate:** Stage 5 CANNOT begin until STATE.md exists alongside Plan.md + Plan_Tasks.md (see Gate G4) and Gate G4.5 is satisfied: Plan-Checker Status is PASSED or PASSED_WITH_WARNINGS and the user confirmed PSU2 (see § Stage Gates).
- **Required Sections:** STATE.md must include skeleton sections for Runtime Risks, QA Findings Summary, and Final Review Log at creation time.
- **Pre-Flight Scope Check (required at creation):** copy the confirmed scope, the materials list with their project paths, and the dated old → new move line from `output/preliminary_notes/{date}_preflight_scope-check.md` (§ Pre-Flight Checklist > On Confirmation (Gate G1)) into STATE.md's Pre-Flight Scope Check section.
- **Session Metadata (required at creation):** Populate the Session Metadata section immediately when creating STATE.md:
  - **DAAF Version (at setup):** Run `git rev-parse --short HEAD` in the DAAF repository root and record the result. This is captured once at project setup and is NOT refreshed to current HEAD on later sessions — a session resuming against a newer DAAF checkout should read this as the setup-time commit, not current HEAD
  - **Session Model ID:** Record the model identifier actually in use for the session (e.g., "claude-opus-4-8[1m]")
  - **Subagent Model Tiers:** Record the specialist model tiers/IDs in use — agent frontmatter defaults (`model: opus` / `model: sonnet`) plus any per-dispatch overrides applied; resolved IDs where known, tier alias + session date otherwise (see STATE_TEMPLATE.md)
  - **Provider Route:** Record how this installation reaches its models, using one of the values STATE_TEMPLATE.md lists. The report's AI Use Disclosure takes the provider and access method from this field. To find the route, see `AI_DISCLOSURE_REFERENCE.md` § Determining the Provider Route. If the route is still not evident, leave the field empty rather than guess: the disclosure then asks the researcher to state it
  - **Model Settings:** Record the reasoning-effort settings for that route as STATE_TEMPLATE.md describes (the same section of `AI_DISCLOSURE_REFERENCE.md` says where each route sets them); the disclosure's Parameters line reads this field
  - **Session Date(s):** Record today's date; update if the project spans multiple sessions
  - **Session Transcript(s):** Leave as the default value — project-local logs are collected at completion

**Update Triggers:** See the **STATE.md Update Gates** table above for the complete list of mandatory update events and which fields to update.

See `agent_reference/STATE_TEMPLATE.md` for the complete template.

### Session Transcript Archiving

On session end, the `archive-session.sh` hook automatically archives the full session transcript (JSONL + readable Markdown) to `.claude/logs/sessions/`. This provides a complete audit trail independent of STATE.md, useful for debugging cross-session issues or reviewing past decisions.

### Session Log Collection (Pre-Report)

**Trigger:** Between Stage 10 (QA Aggregation) and Stage 11 (Report Generation), after all execution is complete.

**Action:** The orchestrator runs:
```
bash {BASE_DIR}/scripts/collect_session_logs.sh {PROJECT_DIR}
```

This searches all archived session transcripts for references to the project folder and copies matching JSONL + MD pairs into `{PROJECT_DIR}/logs/`. The report-writer can then reference these project-local logs in the Reproducibility section.

After collection, update STATE.md:
- **Session Metadata → Session Transcript(s):** Confirm `logs/` contains the collected files
- **Session History → Archive column:** Fill in the archive filenames for each session row

---

## Error Recovery

> For retry/escalation prompt templates, see the "Error Handling in Invocations" section above.

See `agent_reference/ERROR_RECOVERY.md` for complete decision trees and recovery procedures.

### Quick Reference: Error Types & Responses

| Error Type | Max Retries | Escalation Trigger | Reference |
|------------|-------------|-------------------|-----------|
| Data unavailable | 0 | Immediate | `ERROR_RECOVERY.md` § Data Availability |
| Access/network error | 3 | After 3 failures | `ERROR_RECOVERY.md` § Access/Network |
| Code execution error | 2 | After 2 failures | `ERROR_RECOVERY.md` § Code Execution |
| Validation failure (STOP condition) | 0 | Immediate | `ERROR_RECOVERY.md` § Validation |
| Validation failure (warning) | N/A | Document and proceed | `ERROR_RECOVERY.md` § Validation |

### Re-run Guidance

For the complete re-run guidance table (situations, stages to re-run, and refresh/additive modes), see `{SKILL_REFS}/revision-and-extension-mode.md` > Re-run Guidance.

See `agent_reference/ERROR_RECOVERY.md` "Re-run Procedures" for complete re-run decision trees.

---

## Literature Review Handoffs

Each direction needs the user's explicit confirmation (`SKILL.md` § Mode Escalation Paths; `literature-review-mode.md` § Escalation Triggers):
- **Before planning:** when the user wants the published research reviewed before the plan is written (most often at the Phase 1 checkpoint), propose Literature Review mode and pass the Phase 1 notes on.
- **After delivery:** when a surprising finding needs grounding in published research, offer a Literature Review, passing the report path and the findings.
- **Arriving from a review:** a Literature Review that finds a gap the user wants analysed hands over `output/LitReview_Handoff_v{N}.md` in its project folder: the question, the final deliverable's path, the gaps marked answerable with data, the reference list and the manifest pointer. Read it in the pre-flight as a user-referenced material (§ Scope Check). Any fact about the literature comes only from the files `agent_reference/LIT_REVIEW_OUTPUT_CONTRACT.md` § Files Consumers May Read lists; a fact not there is a request for another review round.

reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall