Skip to content

dimagi-internal/ace / skills/demo-data-setup-qa

demo-data-setup-qa

Verified

Internal /ace:run step — Pass/fail check that a demo dataset handoff is live.

FileKindSize
SKILL.mdskill md80.4 KB
checks.tsscript69.3 KB
---
name: demo-data-setup-qa
description: >
  Internal /ace:run step — Pass/fail check that a demo dataset handoff is live.
user-invocable: false
disable-model-invocation: false
---

# Demo Data Setup QA

Structural correctness checks on the `demo-data-setup` handoff — the
`7-synthetic/realized.json` map and the `run_state` `source` block. A demo whose
`par_url` renders the run picker (not the dashboard), or whose opp isn't
labs-only, or whose timeline slides, is a broken funder demo — this gate catches
that before `demo-narrative` builds scenes on top of it.

See `skills/_qa-template.md` for the shared QA contract (verdict YAML format,
auto-fix protocol, static-vs-LLM rules).

## Inputs

Pipeline-only: checks a `demo-data-setup` handoff, which exists only inside a demo or Phase 7 run.

| Source | Artifact | Used for |
|---|---|---|
| `demo-data-setup` | `<demo-run>/7-synthetic/realized.json` | the handoff under check |
| `semantic-registry-author` | `<demo-run>/7-synthetic/semantic-registry-author_registry.json` (or `semantic_registry_get`) | check 28: the registry text the cascade renders (ace-run only) |
| `demo-data-setup` | `run_state.yaml…products.synthetic.source` | provider + labs opp id + deliver units |
| `demo-data-setup` | `<demo-run>/7-synthetic/demo-data-setup_manifest.yaml` | check 29 (ace-run: the per-partner manifest index exists and covers every partner); timeline pin + flagged-worker check; `timeline.end_date` feeds check 11's `opts.timelineEndDate` |
| live labs | `pipeline_get` + `pipeline_preview` per authored pipeline | check 12: the DECLARED field list, and a fresh extraction to judge it against |
| live labs | `workflow_get` per authored workflow, scoped to its OWNING opp/program | check 10: the definition, plus the `pipeline_sources[].{name, schema_summary}` that says whether each declared id resolves in that scope (ace#1894) |
| `demo-data-setup` | `<demo-run>/7-synthetic/branch-scrub_report.yaml` | check 9: the spec derivation (incl. `unparsed[]`), the branch-scrub ledger, the post-scrub audit, and `declared_omissions[]` |
| `demo-data-setup` | `run_state.yaml…products.synthetic.source.interactive_live_load` | checks 8 + 18: the producer's step-4c probe of the interactive run's live pipeline stream, and — on a failure — the control run + upstream ref that evidence the escape (ace#2430) |
| `demo-narrative` | `<demo-run>/7-synthetic/<demo-slug>.yaml` | check 14: the authored scenes' `actions[]`, read for the framing of every judged still |
| `demo-data-setup` | `<demo-run>/7-synthetic/dashboard-terms.yaml` | check 29 (ace-run: it exists, one block per report); check 15: the coined column/row labels each dashboard renders, and the definition affordance each one carries (written by the producer's step 3d) |
| `demo-narrative` | `<demo-run>/7-synthetic/<demo-slug>.yaml` -> `setup` | check 18: the authored spec's `setup.command` + `setup.rerun`. Omit when QA runs BEFORE the narrative is authored — that half is then reported not-judged, never silently passed |

## Products

- `<demo-run>/7-synthetic/demo-data-setup-qa_result.yaml` — QA result per `lib/qa-types.ts`, written by `scripts/demo-data-setup-qa.ts` — **always**, on pass, fail or incomplete (§ Writing the result)

## Checks

| # | id | type | description | auto-fix on fail |
|---|---|---|---|---|
| 1 | `realized_json_parses` | static | `realized.json` exists and parses as a **flat** JSON object (no nested values — DDD substitutes `${var}` verbatim) | re-run `demo-data-setup` step 5 — the handoff was never written / was nested |
| 2 | `every_par_url_is_run_deeplink` | static | `primary_par_url` is present, and every `<key>_par_url` is a run deep-link `/labs/workflow/<id>/run/?run_id=<id>&(opportunity_id|program_id)=<id>` **whose scope param matches the dashboard's OWNERSHIP** — `program_admin_report` / `audit_par` are program-owned and MUST carry `&program_id=`; every opp-owned dashboard MUST carry `&opportunity_id=`. Importable: `checkParUrlScope` in `checks.ts`. Every `*worker_review_url` (ace-run cascade) must also be a run deep-link carrying `&program_id=` — `owning_program_id` alone renders "Workflow definition <id> not found under opportunity <n>" when opened cold (ace#2521). Importable: `checkWorkerReviewUrlScope(realized)`. | fix the scope, not just the shape. Verified live (workflow 5040 / run 5048 / program 10037): `&program_id=` renders the SOP grid, `&opportunity_id=` returns **200** with body "Workflow definition 5040 not found" — so a 200 is not evidence. Pre-#1037 this row demanded `&opportunity_id=` on everything, which failed a correctly-built program rollup and could only be satisfied by emitting the verified-broken URL. Other verified-live traps: `/run/?opportunity_id=` with no `run_id` → workflow LIST; `/workflow/<def>/?opportunity_id=` (no `/run/`) → DEFINITION page |
| 2b | `dashboards_match_realized` | static | `source.dashboards[]` is non-empty; each has `key` + `template` + `role` + `shape` (`run`\|`action`); and every `<key>` has a matching `<key>_par_url` in `realized.json` (plan ↔ handoff agree) | ensure every planned dashboard was built and its `${key}_par_url` written; drop any dashboard that failed to seed |
| 3 | `opp_is_labs_only` | static | `source.labs_synthetic_opp_id` is an integer **≥ 10000** | regenerate with a labs-only opp id ≥ 10000 — a real HQ-backed opp can't be driven by the generator |
| 4 | `timeline_pinned` | static | the manifest `timeline.start_date` is a fixed ISO Monday (not a relative/sliding expression) and **equals every minted run's `instance.period_start`**, read from the `par_url` payloads check 7 already fetched (`#workflow-data`). Importable: `checkTimelinePinned(start_date, [{key, payload}])`. With no payload, or none declaring `period_start`, the second clause is **`anchor-not-judged`** and the check fails: it is never reported as a full pass. This clause used to say "equals the `par_url` opportunity's env anchor". No atom reads that: `synthetic_env_get` is keyed by an env TEMPLATE, not an opportunity, so gates evaluated half the check and passed it (ace#2246). Not run under `ace-run`, whose window is judged by the cascade checks. | set `--pin-monday` to a fixed Monday; a sliding window breaks idempotency. A `period_start` that disagrees with the pin means the run was minted on a different window: re-mint it, or correct the pin |
| 5 | `flagged_worker_not_pre_seeded` | static | the current-week flagged worker has NO pre-seeded audit/task in the manifest (created on camera) | remove the flagged current-week worker's audit/task from the manifest |
| 6 | `deliver_units_present` | static | `source.deliver_units` is a non-empty array | re-capture `deliver_units` from the `synthetic_generate_from_manifest` response |
| 7 | `par_url_payload_is_populated` | static | **Fetches each `par_url` with the labs session**, parses the embedded `#workflow-data` script, and fails when (a) `definition.pipeline_sources` is non-empty while the run's pipeline rows are empty, or (b) any bound field is null/zero for EVERY row. `instance.snapshot.pipelines` arrives as a **dict keyed by alias** (what labs writes) or as an array of `{alias, rows}`; both are read (ace#1701). labs' own built-in row columns (`id`, `status`, `flagged`, `visit_date`, `total_visits`, the `*_visits` counters, `first/last_visit_date`) are **excluded** from (b) — whether they are filled is decided by `terminal_stage`, not by a schema path, so they fire on every entity-stage and visit-level pipeline and evidence nothing. For the **interactive** dashboard, whose run check 8 requires to stay `in_progress`, there is no snapshot by design and the server-rendered payload carries `pipeline_data: {}` (the page fills it over SSE after mount) — fetch `GET /labs/workflow/api/<definition_id>/pipeline-data/?opportunity_id=<opp>` and pass it as `livePipelines`; not doing so is a reported `live-pipelines-unavailable` finding, never a silent pass. Judgement is pure — `checkParUrlPayloadPopulated` in `checks.ts` takes the parsed payload, so only the fetch is the skill's job. | re-point the pipeline schema at the REAL form paths the generator writes (not the stock template's `form.meta.*`), and declare `snapshot_inputs.pipelines` for every alias in `pipeline_sources` before completing the run (#1160). A field zero for SOME rows is data, not a dead binding, and is not flagged |

| 8 | `interactive_run_is_live` | static | **The interactive dashboard's run state matches its role, AND the `in_progress` mandate has been PROVEN (ace#2430).** Using the payload check 7 already fetched plus the producer's `source.interactive_live_load`: the dashboard whose `role` is interactive (`review-action` / `review` / `decision`) MUST have `instance.status != completed`, every OTHER dashboard MUST have `instance.status == completed`, and the interactive one's live load must be RECORDED. **Fails** when an interactive run is `in_progress` with no live-load record (the mandate was never verified), when its own recorded probe says the page cannot load yet it shipped `in_progress` anyway, and when it is `completed` without an evidenced escape. **Passes a `completed` interactive run on the EVIDENCED escape only** — `status: failed` plus `observed`, `run_id`, `control_run_id` (a completed run of the SAME definition that DOES render) and `upstream_ref`; an unevidenced flag exempts nothing, and the escape is reported in `detail`, never silent. Importable: `checkInteractiveRunsLive(pairs, liveLoad)` in `checks.ts`, over `classifyLiveLoadEvidence` (`lib/interactive-live-load.ts`). A payload with no `instance.status` is still reported, not failed (#1162). | A completed run renders "This run is completed… Decisions are read-only" with the status control disabled, so the decision the narrative demonstrates cannot be performed on camera — skip `workflow_save_snapshot` for the interactive dashboard only. The reverse half matters too: a non-interactive run left `in_progress` has no snapshot, so its `par_url` is not a stable deep-link. For the new half: run `demo-data-setup` step 4c's probe — `GET /labs/workflow/api/<def>/pipeline-data/stream/?<scope>&refresh=1` with the labs session — and write the result. The asymmetry is the point: a completed run's page returns early on its snapshot and never opens a stream, while an `in_progress` run re-runs that stream on every load, so the live path is exercised by the interactive dashboard alone. On `poverty-graduation/20260915-1518` it died after its first event and this row, being one-sided, made the dead configuration the only passing one |

| 9 | `dataset_obeys_pdd_constraints` | static | Runs `checkDatasetObeysPddConstraints` (`checks.ts`) over three inputs the producer's step 2c writes: the **derivation** (`specFromDeliverApp(get_opportunity_apps(<opp>, 'deliver'))` — the spec is DERIVED from the app's own `relevant` / `constraint` expressions, and hand-declared currency / cross-field / `uniquePairs` entries are ADDITIONS merged with `mergeDatasetSpecs`, never a replacement), the **branch-scrub report** (`scrubOffBranchFields`), and `auditDataset` over the records as they now stand. Fails on: a clean audit with no derivation behind it and no stated reason; a deliver app that returned 0 questions; any `unparsed[]` gate or bound (an expression the derivation could not read is a gate this run did not audit — a reported finding, never a silent pass); a scrub field that could not be located; and any violation class. **The one escape is `declared_omissions[]`** (step 2c.4, `{field, reason}` per entry): a residual labs STRUCTURALLY cannot emit — a `Trigger` read-aloud label CommCare submits no value for, an image with no labs `ImageConfig` corpus (a REPEAT-group roster was the third class and is **no longer exemptible**: `BeneficiaryCohort.repeat_groups` is in the manifest schema and `auditDataset` descends into an array, so a materialised roster now reads as PRESENT — build it, ace#2432) — is exempted from that field's `conditional-missing` violation AND its `unresolvedFields` entry, but from nothing else: an off-branch value, an integrality violation and an `unparsed[]` gate each name a value that IS present and wrong, and stay unexemptible. A blank `reason` exempts nothing and is named in the failure, exactly as check 13 refuses an unevidenced `below_programme_scale`; the exemptions are echoed into the detail string so they reach the run summary rather than vanishing into a green verdict. Without it the check was permanently red on `poverty-graduation/20260908-0510` — zero off-branch, zero integrality, empty `unparsed[]`, six structural residuals — and the only routes to green were narrowing the spec, which is what this check exists to prevent, or a `fail` nobody reads (ace#2225). | **Re-run the branch scrub — do NOT narrow the spec.** `scrubOffBranchFields(records, spec.conditionalFields)` from `lib/dataset-constraints.ts`, then write the scrubbed `user_visits.json` back to the opp's fixture folder before any dashboard run is minted, **then call `mcp__connect-labs__synthetic_reload_fixtures(<opp>)`** — the write-back alone changes nothing labs will read, because `synthetic_generate_from_manifest` primes the fixture cache itself and an in-place edit is invisible until that cache is dropped (ace#1860; measured on `spark-facilitator/20260828-0703`, where a correctly-scrubbed fixture still previewed pre-scrub numbers). Note this check judges the producer's LOCAL records, so a stale cache passes it — and checks 7 and 11 pass too, since both dashboards read the same stale cache and agree with each other. Carry the per-field counts into the run summary. There is **no manifest-side remedy on this path** — though not for the reason this row used to give. `BeneficiaryCohort.relevance_groups` **does** exist (`connect_labs/labs/synthetic/generator/fixtures/manifest.py:300`, shipped in `dimagi-internal/connect-labs#1331`, merged 2026-08-27), and it is **inert here**: relevance is applied only to questions present in the HQ `FormSchema` (per-question inside the schema loop, `fixtures/fields.py:417-422`), while the trailing orphan-write loop is gated by a set computed **once, before it runs** (`fields.py:484`). A labs-only opp has no Connect `app_structure`, so `parse_form_schema_from_app_json` returns `FormSchema(questions=[])`, every declared path is an orphan, and the controller is not in the record when line 484 evaluates the gate. Measured on `bednet-check-2-visit/20260828-0629`: `relevance_groups` was declared and 36 off-branch values were still emitted (#1833). Declare it anyway on a **schema-backed** opp, where it works — and scrub regardless. (`null_rate` is unconditional; `CorrelationSpec` cannot make a field absent on a branch, so neither substitutes.) For an `unparsed[]` entry, hand-declare that gate as an ADDITION and re-run. This gate exists because a run wrote "0 constraint violations, all hand-checked" into run_state for a set with 251 fractional people-counts, 242 fractional Kwacha amounts, 34 off-branch reasons and 22 did-not-happen meetings carrying 41 attendees — and THIS SKILL PASSED (#1346) — and because `bednet-check-2-visit/20260817-1720` then passed check 9 with `conditionalFields: []` on an app whose two observation fields are both gated on consent, which the next run of the same opp measured at 18 of 276 each (#1658). Report the per-class counts so "0" is measured. **A `spec-addition` resolution is checked, not trusted (ace#2245):** pass `mergedSpec` = `mergeDatasetSpecs(derived.spec, <the ledger's `additions`>)` and `fieldPresence` = `countFieldPresence(records, <every field the merged spec's rules read>)` (`lib/dataset-constraints.ts`). A `spec-addition` clears its unparsed gate only when the merged spec puts a rule on that field that audits the expression: a conditional entry not still listing it in `unparsedGates`, an integer bound, or a cross-field rule. Otherwise the check fails, and it also fails when no `mergedSpec` is passed. An addition whose gate field is carried by 0 records is REPORTED as declared but never firing (`spark-facilitator/20260907-1120`: `enrol_gps_accuracy_band`, 0 of 182). |

| 10 | `dashboard_bindings_are_wired` | static | For each authored workflow, runs `checkDashboardBindings` (`lib/dashboard-bindings.ts`) over **the `workflow_get` response, passed through unchanged**: no pipeline schema still extracting `form.meta.*` (the stock template paths the synthetic generator never writes), `snapshot_inputs.pipelines` covering every alias in `pipeline_sources`, render code that actually READS a declared pipeline rather than a denormalized `worker.visit_count` the generator never back-fills, and — new, ace#1894 — **no declared pipeline id that fails to resolve in the workflow's own scope**, which `workflow_get` reports as `name: null` + `schema_summary.field_count: 0` beside that source. Still zero extra network calls: the resolution metadata rides along on the fetch the producer already made. Pass the raw response, not the stored `{alias: id}` dict — the dict carries no resolution metadata and the scope check silently abstains on it. | ADAPT means RE-POINT — re-point the new pipeline's schema at the same real form paths the scorecard pipeline already resolves, declare the snapshot aliases, and bind the render to the pipeline. Live: workflow 5069 hit the first three at once and rendered `VISITS 0` beside `visits: 835` on data that summed correctly (#1160). For the fourth: **do not `workflow_clone` a dashboard across opps** — pipelines are opportunity-scoped and the clone copies `pipeline_sources` verbatim without the pipelines, so the page renders empty with no error (#1894). Instantiate from template in this opp and re-point. Complements check 7, which catches the same class from the rendered payload; this one catches it from the DEFINITION, before a run is even minted |

| 11 | `cross_dashboard_totals_agree` | static | **Cross-dashboard consistency.** Using the payloads check 7 already fetched: when two or more dashboards read the same `labs_opp_id`, their shared visit total must AGREE. `deriveVisitTotal` takes `sum(total_visits)` from an aggregated pipeline's rows, else the row count of a visit-level pipeline. Program-scoped rollups are excluded (a cross-opp rollup aggregates a different population); a dashboard that deliberately renders a sub-window declares `period_scope: 'partial'` in `source.dashboards[]` and is excluded by name; a dashboard with no visit-shaped rows is reported not-judged, never failed. Pass the manifest's `timeline.end_date` as `opts.timelineEndDate` and a disagreeing dashboard whose `period_end` is at or before it gets the off-by-one named. Importable: `checkCrossDashboardConsistency` in `checks.ts`. | **`period_end` is EXCLUSIVE.** Re-mint the snapshotted run with `period_end = timeline.end_date + 1 day` and repoint its `par_url`. `_date_window_where` (`connect_labs/labs/analysis/backends/sql/query_builder.py`) emits `visit_date >= date_from AND visit_date < date_to`, so a `period_end` equal to the fixture's last `visit_date` drops that whole day — while a live, never-snapshotted sibling is never period-scoped and keeps it. If the two dashboards are MEANT to show different windows, declare `period_scope: 'partial'` on the narrower one rather than tolerating the gap silently |


| 12 | `authored_pipeline_fields_extract` | static | **Every field an authored pipeline DECLARES actually extracts something, judged from a FRESH preview.** For each pipeline the run authored (read the schema with `pipeline_get`, not the columns that came back), call `mcp__connect-labs__synthetic_reload_fixtures(<labs_opp_id>)` once, then `mcp__connect-labs__pipeline_preview(pipeline_id, opportunity_id, sample_size >= 10)`, and pass `{pipeline_id, declared: schema.fields, rows, from_cache: per_opp_metadata[<opp>].from_cache, fields_all_null, fields_suspect}` to `checkPipelineFieldsExtract` (`lib/pipeline-field-extraction.ts`). Fails on: a preview served `from_cache: true` (warm rows are not evidence about the saved schema); a declared field no returned row carries; and a declared field that is null/zero for EVERY row with no `filter_path` to explain it. An all-zero **filtered** count is reported, not failed — a filter may legitimately match nothing. Zero-for-SOME-rows is data and is never flagged. The preview's own `fields_all_null` **and `fields_suspect`** are both folded in so this can never fall below labs' detectors. `fields_suspect` (connect-labs#1882) is the one that catches a field which extracted the WRONG value rather than none — a `count` equal to the row count, or a `filter_value` that matched nothing anywhere. Every other rule in this check judges a field by the ABSENCE of a value, so that class is invisible to all of them: on ace#2431 a `count` over the JSONB column `flag_reason` returned each worker's `total_visits` and passed every gate. Reported, not blocking — both signals are heuristics with a legitimate reading. | Re-point the named field at a path the fixture actually writes, `synthetic_reload_fixtures`, re-`pipeline_update_schema`, and re-preview until `from_cache: false` AND the column is non-dead on at least one row. **Do not delete the field to clear the check** — the render binds it, so a removed column is the same dead demo |
| 13 | `declared_detection_has_a_cohort` | static | **A demo that DECLARES a detection control must have realized a cohort that makes detection non-trivial.** Runs `checkDetectionCohortFloor` (`lib/ddd-scene-actions.ts`) over the producer's own `source` block: fails when `source.detectable_signal` names a control (step 1c's required authoring output) and `source.data_shape.rows` is below `DETECTION_MIN_ROWS` (24). Silent when no signal is declared, when `detectable_signal.below_programme_scale` carries a PDD quote proving the programme's own roster is under the floor (raising the cohort would misrepresent it, and stripping the signal would make the dashboards less honest — an unevidenced flag is rejected), when the documented `none` escape is recorded ("the PDD declares no verification rules"), or when `rows` is unstated — it never guesses. Zero network calls; both fields are written by step 1c/2b before this gate runs. | **Raise the cohort in the MANIFEST and regenerate** — no narration fixes it, because a detection claims unaided scanning is not viable and that is false the moment the whole cohort fits in one look. If the demo is not really about detection, drop `detectable_signal` or record the `none` escape. Complements `checkSceneCardinality`'s vocabulary rule, which catches the opposite half (a narrative CLAIMING detection the data never declared) — that rule is keyed on words like `flag`/`outlier`/`detect`, so a detection demo written in plain descriptive prose evades it entirely: on `bednet-check-2-visit/20260902-1555` the spec carried ZERO detection tokens over a 5-row cohort, the vocabulary rule found nothing, and Phase 7 ended `stopped_not_converged` at concept 2.0/5 on exactly that objection, raised independently by all five per-scene judges (ace#2131) |
| 14 | `scroll_framing_is_anchored` | static | **Every judged frame is framed by something the PAGE owns, not by a pixel guess.** Runs `checkScrollFraming` (`lib/demo-frame-legibility.ts`) over the authored spec's `scenes[]`. A `kind: scroll` is judged only when a still is actually written at the position it set — a `hold` or `snapshot` follows it, or it ends the scene (the scene's end frame is the canonical still); a scroll the spec navigates away from before any capture is transient and is not judged. **Fails** on a raw pixel `value`. `top` / `bottom` are REPORTED, not failed — they name a landmark the page defines and survive a layout change, but they decide only one edge of the frame. Runs only once `demo-narrative` has written the spec; before that this row is not-yet-judged, and `demo-narrative` § Step 3b runs the same function inline at authoring time. This row is the backstop that does not depend on anyone remembering. **A pass here is not automatically a verdict — read `judged` before you report it.** The check reads pixel scrolls only, so a spec that frames every still with `scroll_to` gives `judged: 0`: it evaluated nothing. That is a legitimate and expected state — `scroll_to` is this row's own remedy — but it is not an approval of how those frames are composed, and it must not be written up as one. When it happens the check now says so itself, with a non-blocking `scroll-to-framing-unjudged` finding naming how many actions it declined to look at (ace#2253). Record it as *passed over N unjudged `scroll_to` frames*, never as *passed*. | Replace the pixel scroll with a `scroll_to` action naming the element the frame is about — canopy resolves and centres it live, which holds when the target's centred position is reachable. A target in the first or last viewport-height of a short page is not: the centring scroll clamps back to where the camera already was and the scene films the previous frame (ace#2253) — frame those with `top` / `bottom` or a pixel scroll measured against the live page. **Write only `kind` and `target` on it.** There is no framing key to add: `_ActionBase` sets `extra="forbid"`, which is why the ace#1660 check that demanded one was retracted, and why today's judge remedy ("offset so no partial histogram sits above it") breaks the spec if followed literally. If the header block and the whole table cannot co-exist in one frame, split the beat into two scenes rather than compromising both |
| 15 | `coined_terms_are_defined_on_page` | static | **A term a lay viewer cannot read carries an on-page definition they can reach FROM the term.** **Enumerate every coined term a viewer MEETS on a frame, not only column headers** — KPI subtitles, panel body prose and the explainer panel itself all render into judged frames. On `spark-facilitator/20260907-1120` this check passed both dashboards (13 and 11 labels judged, 0 findings) while the user judge capped `clarity` at 2 on "a repeated entity key", cited independently by six of seven judges — body prose inside the *How to read the columns* panel, i.e. inside the very surface whose job is to remove jargon. An under-enumerated list passes vacuously; the enumeration is the check. Runs `checkCoinedTerms(dashboardTerms, definedTerms)` (`lib/demo-frame-legibility.ts`) over the two lists the producer's step 3d enumerates off the render code it uploaded. Fails on a coined label with no definition anywhere, and on one whose only definition sits in a glossary panel elsewhere on the page (`at_point_of_use` not `true`) while the label renders on a hero surface. An orphan definition matching no label is reported. An EMPTY enumeration is reported, never a silent pass — this check cannot tell a plain-language dashboard from an un-enumerated one, and only one of those is a pass. Deliberately keyed on the ARTIFACT, not on a jargon word list: a vocabulary rule is what ace#1841 pruned for precision and ace#2131 then evaded. | Put the gloss at the point of use — an info affordance on the column header carrying that column's one-line plain read, or a plain-language lead line with the statistics demoted behind it. **Adding another panel is not the fix**, measured: iteration 2 of `poverty-graduation/20260905-1345` added a "What each column means" panel, the judge recorded it VERIFIED PRESENT and correct, and the jargon cap fired on six of seven scenes anyway because the panel is below the fold of every frame that uses the vocabulary. Renaming the label so it needs no gloss is better still where it is available — "Surveys in the 31-point band" → "Small-household surveys (≤3 members)" deletes both the term and the disavowal sentence the glossary had to carry |
| 16 | `rendered_columns_obey_declared_invariants` | static | **Every number a viewer can compare on the page can be true at the same time.** Runs `checkColumnInvariants` (`lib/dashboard-column-invariants.ts`) over each dashboard's declared `column_invariants` and the LIVE `pipeline_preview` rows for its pipeline — the same payload check 12 fetches, judged for RELATIONS rather than extraction. **Fails** on any row breaking a declared relation. Also **fails**, deliberately, when an invariant names a column the payload does not carry, when there are no rows to judge, or when a compared column is null: an invariant that silently does not run reads as a pass it never earned, which is exactly ace#2253. A dashboard that declares NOTHING while rendering two or more comparable numeric columns is REPORTED, not failed — the same posture check 15 takes for `no-terms-enumerated`. This is a different tier from check 9: check 9 derives its spec from the Deliver app and judges RECORDS, and no record was wrong on the run that motivated this (ace#2250) — the contradiction lived between two aggregates, which a form-derived spec cannot express. | Fix the pipeline SCHEMA, not the data. The canonical cause is an aggregation counting RECORDS where the column header promises a count of THINGS — a `count` beside a sibling `count_distinct`, both rendered as the same noun. Re-run `pipeline_update_schema` → `synthetic_reload_fixtures` → `pipeline_preview` and re-judge. If the relation itself was wrong, restate it with the header's own words in `because` |
| 17 | `scenes_show_more_than_one_thing` | static | **The deck shows the viewer more than one thing.** Runs `checkSceneVariety` (`lib/demo-scene-variety.ts`) over the authored spec's `scenes[]`, resolving each scene's surface by carrying the last declared `url` forward. **Fails** when more than half the scenes are one surface, and when a scene follows the previous one on the same surface with no state-changing action — the two deduction rules the arc judge applies literally, each of which caps a dimension at 2 and so holds the whole run at its floor. Judged from the spec alone, before any render. Blocking rather than reported BECAUSE the loop cannot repair it: collapsing or reordering scenes is a narrative change behind canopy's `concept_change` gate, so an unattended run stops instead (`spark-facilitator/20260907-1120`, arc 2 of 5, `stopped_not_converged`). Same relationship to `demo-narrative` § Step 3b as check 14 — the backstop that does not depend on anyone remembering. | Fold the offending beat into the scene before it, or give the scene an action that changes what the page shows — a filter, a selection, a drill-in. For a surface monopoly, give the deck another surface or merge the beats that share one into fewer, denser scenes. Splitting one page across four scenes buys narration, not pictures |
| 18 | `render_reset_is_registered` | static | **A demo with a state-mutating control resets it before every render.** Runs `checkRenderResetRegistered` (`lib/labs-run-state-reset.ts`) over `source.dashboards[]`, `source.render_reset`, and the authored spec's `setup` block. When any dashboard is `interactive: true` (role `review-action` / `review` / `decision`), the handoff MUST carry `render_reset.{required: true, run_id, state_keys, command}` AND the spec's setup block MUST be `rerun: per_render` with a command containing that reset. Each half is a no-op alone: an unregistered reset never runs, and `per_render` over a command that only re-emits `realized.json` re-derives the same data while the run's `spawned_tasks` stay exactly as the last take left them. A demo with no interactive dashboard passes trivially. **So does one whose interactive run took check 8's EVIDENCED live-load escape (ace#2430):** that run ships `completed`, labs answers a state write to a completed run with 409, so nothing can be dirtied and there is nothing to reset — demanding one anyway would leave no configuration that satisfies both gates. An unevidenced or `ok` record stands nothing down. **`render_reset_command_pinned` (ace#2351):** fails when the registered command — or the spec's `setup.command` — contains `/plugins/cache/ace/ace/` or a `/Users/<name>/` (`/home/<name>/`) segment. That is a snapshot of ONE machine's versioned plugin cache: the cache keeps every prior version, so the path keeps resolving to STALE code after every `/ace:update`, and another account cannot run it. The only passing shape is the self-resolving one, `bash "$(python3 -c "…installed_plugins.json…['plugins']['ace@ace'][0]['installPath']")/bin/ace-reset-labs-run" …` (`renderResetCommand()` in `lib/labs-run-state-reset.ts`). A `verified_at` that was honest on the day it was written does not clear this: the command was green THEN, in THAT tree. | Register the reset (`demo-data-setup` step 4b) and wire it (`demo-narrative` step 3). On `render_reset_command_pinned`, re-run step 4b to register the self-resolving form and re-author the spec's `setup.command` from it — never hand-patch the version number, which is the same pin one release later. `state_keys` come from `workflow_get -> saved_runs.snapshot_inputs.state_keys` — labs shallow-merges run state, so a reset naming no key is a silent success. Verified on `spark-facilitator/20260908-2215`, labs run 5508: iterations 0-2 passed only because someone reset the state by hand (ace#2297) |
| 19 | `escaped_payoff_is_honoured` | static | **A demo that took the below-programme-scale escape actually built its payoff elsewhere.** Runs `checkEscapedPayoffIsHonoured` (`lib/ddd-scene-actions.ts`) over `source.detectable_signal`, `source.payoff_control` and the authored spec's `scenes[]`. Silent unless check 13's evidenced `below_programme_scale` escape was taken — so a cohort at or above `DETECTION_MIN_ROWS`, an undeclared signal, the documented `none` escape, and an unevidenced flag (which check 13 fails instead) are all unaffected. **Fails** when the escape was taken and `payoff_control` names no control, and when it names one that no scene ACTS on. Before the narrative exists the declaration half is still judged and the honouring half is deferred, never silently passed. Keyed on the ARTIFACT — an identity match on the canopy target, prefix-aware but not substring — because a vocabulary rule is what ace#1841 pruned for precision and ace#2131 then evaded in plain prose. | Declare the control the payoff acts on (`demo-data-setup` step 0, derived from the PDD's `verification_predicate` BEFORE a template is chosen) and author the payoff against it (`demo-narrative` step 3). **If the dashboard carries no such control, author it onto the page** — the defect is that the template was selected before the demonstration was chosen. Verified on `spark-facilitator/20260908-2215`: the escape was taken with two correct PDD quotes and the right mitigation written down verbatim, and the payoff then hid 10 rows of 12 behind `testid:below-target-filter` — the exact claim the escape promised not to make. Four independent judges capped `use_case_soundness` at 3, pinning concept under the 4.0 bar; three consecutive runs of the opp ended non-converged and nothing was looking at this (ace#2316) |
| 20 | `arc_ladder_ends_on_the_action` | static | **The deck's last scene performs a state-changing action.** Runs `checkArcLadder` (`lib/demo-arc-ladder.ts`) over the authored spec's `scenes[]`. `arc_shape` carries a hard cap at 3 when the finale is not the run's strongest moment; weighted .30 against an `overall_rule: lowest` judge and a 4.0 bar, that ceiling alone blocks convergence. **Fails** when no scene performs a state change, and when the last state-changing beat is followed by inert scenes. Stands down on an EVIDENCED `finale_is_read` declaration (a blank flag silences nothing, same discipline as `below_programme_scale`). Silent on an empty deck — reported as judged 0, never a vacuous pass. **Deliberately weaker than `demo-narrative`'s ladder rule, and this is the point:** it PASSES the run that motivated it (`spark-facilitator/20260909-1211`), whose recap finale technically clicked a disclosure control. Telling "a click that reveals prose" from "a click that changes what the product knows" needs rendered frames — the arc judge's job — and a static heuristic for it would be the fourth retracted rule in this family (ace#1660 retracted, ace#1841 pruned). This row catches a genuinely inert finale; the recap finale is caught by authoring, not by a check. | Move the state-changing beat to the LAST scene and fold trailing commentary (caveats, open parameters) into the framing — it is a reorder of material the deck already has, not new work. See `demo-narrative` § the scene ladder (ace#2339) |
| 21 | `cascade_story_landed` | static | **ace-run only (ace#2510).** `checkCascadeStory` (`checks.ts`, over `lib/cascade-story.ts`): `7-synthetic/cascade-story.yaml` exists; its plan passes `checkCascadeStoryPlan` (≥ 5 invented partners — ≥ 2 for a `partner_source: programme` mirror — on labs-only opportunities, ≥ 8 workers each, ≥ 8 weeks; invented partners carry a REALISTIC invented organisation name with a trailing `(example)` marker — `Tiyende Community Trust (example)`; bare `Partner A` fails (it reads as a real implementer of a one-implementer pilot) and so does `Example partner A` (a placeholder the DDD judge scores at clarity 2, ace#2727); a display roster names every worker (`{username, display_name}`) and followed entity (`{id, name}`) with a human, non-code, non-placeholder name and no real person's name from `--real-people`; every `headline_indicators` entry has an authored `headline_spread` that is not flat across all partners (≥ 5 pts best-to-worst); a `partner_source: programme` plan's partner count equal to the PDD's `implementing_orgs`; all four signals — lagging partner, standout worker, data-quality problem, trend — each on a registry indicator with a PDD citation); and every signal LANDED in the programme report's saved runs (`workflow_history_runs(include_snapshot)` → `verifyCascadeStoryLanded`: the carrier is the extreme row by ≥ 5 pts, the trend moves ≥ 5 pts in its declared direction, and every headline indicator spreads ≥ 5 pts across the graded partner rows). The snapshot is the authority, never the manifest. Skipped (reported, not passed) on the denovo/clone providers. | Fix the pool for the signal that did not land, regenerate that partner, rebuild history (`demo-data-setup` § C5), re-verify. Never re-describe the story to match the output |

| 22 | `worker_review_url_scoped` | static | **ace-run only.** Every `*worker_review_url` in `realized.json` is a run deep-link carrying `&program_id=` (`checkWorkerReviewUrlScope`). `owning_program_id` is a data hint, not a page-scope param: opened cold, labs scopes the page to the session's last opportunity and renders "Workflow definition <id> not found" (ace#2521). | Rebuild each URL with `scripts/worker-review-url.ts --fix <realized.json>` (`lib/worker-review-url.ts`; `demo-data-setup` § C7) — never by hand |
| 23 | `cascade_handoff_complete` | static | **ace-run only.** `checkCascadeHandoff` over `products.synthetic.cascade`: a registry with indicators; ≥ 3 partners, each on a labs-only opp (≥ 10000) with its own opp report; the programme report's run is the LATEST history run; every report mirrored in `products.synthetic.workflows` (ace-web renders that map, so an unmirrored report is invisible to a reviewer). | Complete the handoff (`demo-data-setup` § C4–C7) — record each id as it is minted |
| 24 | `cascade_review_routing_follows_pdd` | static | **ace-run only (ace#2735).** `checkReviewRouting` (`lib/cascade-build-qa.ts`) over the sampled visit rows (`--visits`; the detail names the sampled scope — `pipeline_preview` caps at 200 and returns the home partner, ace#2973): each record raising a flag the registry declares (`display.visit_flags[].column`) carries the plan's `review_routing` route for it (`expect` fields → allowed values, read from the PDD), or the route is `report_only` with the PDD's words. **Fails** on a raised flag with no declared route and on any flagged record off its route. Measured on `spark-facilitator/20261004-1706` (Tiyende, 153 rows): 11 of 12 `repeat_counts_flag` records `approved` with `flagged=false`, against PDD §7.2 S-1's desk review — the DDD judges found it after render. | Write the route's fields onto the flagged visits in the pool, regenerate that partner, rebuild history; if the PDD only reports the flag, declare `report_only`. Never relabel the flag |
| 25 | `cascade_worker_rates_not_binary` | static | **ace-run only (ace#2735).** `checkWorkerRateDenominators` over the latest saved run: no worker-table rate (`display.indicators[id].scorecard`) has a graded denominator (`cell.n`) ≤ 1 for more than half the workers. Spark: Step 7 on time, n = 1 for 60 of 60 facilitators — a rate that can only read 0% or 100%, judged "false precision". Plan-time twin: `checkPlannedWorkerRates` at `demo-data-setup` § C3. | Raise entities per worker, or set the indicator's `flw_applicable: false` / `scorecard: false` so it shows at partner level only |
| 26 | `cascade_displayed_columns_vary` | static | **ace-run only (ace#2735).** `checkConstantDisplayedColumns`: no worker-table column (each scorecard indicator — the entity-count column is NOT judged: since connect-labs #2262 the render moves a count that is the same on every row into the table title, "Educators · 1 facility each", so it is never drawn constant; ace#2972), case field (`display.case_fields`) or visit field/flag (with `--visits`) is the same on every row of the saved run. A column constant only inside one partner's worker table is REPORTED in the detail (the narrative should not film that table for it). Spark: Repeat counts 0.0% across whole partners (reported); its "Communities" = 1 finding predates the labs collapse. **Near-constant worker rates fail too (ace#2746):** a scorecard rate whose spread is under 3 points across the workers of EVERY partner, while another scorecard rate there spreads ≥ 15 points (`NEAR_FLAT_POINTS` / `WIDE_SPREAD_POINTS`), reads as generated. Spark attempt 3 passed this check at 35.8–36.9% Women attending beside 55–98% regularity, and the DDD user judge capped `trust` at 2 on it. The same flatness in only some partners is reported. | Vary it in the pool / roster, or stop showing it (registry display, run-owned template render code). A near-constant rate needs a **per-worker effect** on its per-visit draws (`demo-data-setup` § C3), not a wider partner range |
| 27 | `cascade_drill_levels_branch` | static | **ace-run only (ace#2735).** `checkDrillLevels`: no drill level (`partner>opportunity`, `opportunity>worker`, `worker>entity`) has exactly one child under every parent, unless the plan's `single_child_levels` cites why (a blank reason exempts nothing; an exemption is reported). Spark: one opportunity per partner on all 5 — the judges' "redundant single-row Opportunities level". | Branch the level in the data, or record the PDD-cited exemption and keep scenes from pausing on that level |
| 28 | `cascade_registry_text_is_plain` | static | **ace-run only (ace#2748).** `checkRegistryTextIsPlain` (`checks.ts`, over `lib/registry-display-text.ts`) on the registry (`--registry`: `7-synthetic/semantic-registry-author_registry.json`, or the `semantic_registry_get` response after a registry edit): every string the cascade RENDERS — `display.title`, `targets_note`, the entity/worker/organisation nouns, categories, `reading.label`, `case_fields` / `visit_fields` labels, `visit_flags` labels AND descriptions, each indicator's `meta.label` / `meta.plain` — reads as plain language for a programme manager. **Fails** on a design-document reference (`PDD`, `§`, a section number like `8.1`), an internal indicator code (`P1`, `S-1`, `SF_P1`), a comparison symbol standing in for words (`≥`, `≤`), or a raw snake_case field name. `scope_note` / `means` / `description` are author-facing and not read. Measured on `spark-facilitator/20261004-1706` (registry 7748): `targets_note` "Targets are the PDD's own (§8.1 P1 ≥ 80%, P3 ≥ 75%)" capped the DDD demo at clarity 2 after the build fixes, and its `visit_flags[].description` cite `(PDD §5.4)` / `(PDD §7.2 S-1)`. | Rewrite each named string per `semantic-registry-author` § 3 ("Targets come from the pilot design: a meeting in at least 80% of each community's weeks …"), `semantic_registry_update`, rebuild history (`demo-data-setup` § C5). Keep the PDD citation in `scope_note` |
| 29 | `cascade_build_artifacts_written` | static | **ace-run only (ace#3251).** `checkCascadeBuildArtifacts` (`lib/cascade-build-artifacts.ts`) over `--manifest-index` and `--dashboard-terms`: `7-synthetic/demo-data-setup_manifest.yaml` exists, lists every `cascade.partners[]` opportunity, each with a manifest archived under `7-synthetic/cascade-build/`; `7-synthetic/dashboard-terms.yaml` exists with ≥ 1 dashboard, each carrying a `terms` list. Both are REQUIRED by `lib/artifact-manifest.ts`; on `connect-fi/20261018-1440` neither was written, this gate passed 10/10, and Phase 7 finalize found the gap after the DDD loop. | Run `scripts/cascade-build-artifacts.ts` (`demo-data-setup` § C7), archive each partner manifest at the path the index names, upload both |

### Which checks apply to which provider

`checksForProvider` (`checks.ts`) is the list; the script fails every applicable
check it neither computed nor was handed an outcome for:

| Provider | Checks |
|---|---|
| `ace-run` (the Phase 7 cascade, ace#2510) | 1, 2 (over the cascade's programme + opp reports), 22, 23, 21, 24–29 |
| `denovo` | 1, 2, 2b, 3–13 |
| `clone` | 1, 2, 2b, 3, 6, 7 |

The spec-dependent rows (14, 17, 18, 19, 20) run once `demo-narrative` has
written the spec; before that they are passed in as `not_judged` with the reason,
which the result lists separately and never counts as a pass.

All checks are static (<100ms), no LLM. Binary verdict: any BLOCKER fail →
`fail`; else `pass`.

**Promotion note (Plan A):** these checks are defined here and evaluated at gate
time against `realized.json`. Once the live realized-map shape is pinned in the
joint test, promote them to an importable `checks.ts` + unit test (mirroring
`skills/synthetic-narrative-plan-qa/checks.ts`) for static CI enforcement.

## Writing the result — always, and in the canonical shape

**Every invocation writes `7-synthetic/demo-data-setup-qa_result.yaml`**, whatever
the verdict, and records `phases.synthetic-data-and-workflows.steps.demo-data-setup-qa:
{status: done, verdict: <pass|fail>}` in run_state. Two runs showed what happens
otherwise: `bednet-check-2-visit/20260908-1544` ran 19 checks but hand-wrote them as
`checks_total` / `checks[]` — a shape no reader parses — so ace-web displayed
"Passed (0/0 checks)"; `spark-facilitator/20260926-1800` (ace-run) never ran the gate
and has no result at all.

1. Download the inputs locally: `run_state.yaml`, `7-synthetic/realized.json`, and for
   the ace-run provider `7-synthetic/cascade-story.yaml`, `demo-data-setup_manifest.yaml` and
   `dashboard-terms.yaml` (all `drive_read_file`
   `writeToPath`, default `text/plain`), plus the programme report's saved runs —
   `workflow_history_runs({definition_id: <programme_report.workflow_id>, program_id,
   generated_only: false, include_snapshot: true, limit: 100})` saved as JSON.
2. For the live-labs checks this script cannot compute (7, 8, 10, 12, 16 on
   `denovo` / `clone`), run their `checks.ts` functions and write their outcomes as
   JSON: `[{check, result: {pass, detail, auto_fix_hint}}]`, or
   `{check, not_judged: "<why>"}` for a spec-dependent row.
3. Run:
   ```bash
   node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/demo-data-setup-qa.ts" \
     --run-state <run_state.yaml> --realized <realized.json> \
     [--story <cascade-story.yaml>] [--history <history.json>] [--visits <visits.json>] [--outcomes <outcomes.json>] \
     [--real-people <names.txt>] [--registry <semantic-registry-author_registry.json>] \
     [--manifest-index <demo-data-setup_manifest.yaml>] [--dashboard-terms <dashboard-terms.yaml>] \
     --target <opp>/<run-id> --out <local demo-data-setup-qa_result.yaml>
   ```
   `--real-people` is one name per line — every real person the opp's `inputs/`
   name (PDD contacts, LLO contacts, reviewers); check 21 fails an invented worker
   carrying one (ace#2727). `--visits` is the `pipeline_preview` response(s) for the cascade
   visits pipeline — check 24 fails without it. **It is a SAMPLE, and the check says so (ace#2973).**
   `sample_size` is capped at 200, and the default cascade's visits pipeline is home-scoped to the
   first partner: previewed under any other opp it is "No pipeline with id …", and fanned out with
   `opportunity_ids` the merged sample is the home partner's rows first. So preview it under the home
   opp at `sample_size: 200` and pass that; the script appends `visit rows judged: <n> of <N> (opps …)`
   to checks 24 and 26 (`visitSampleScope`) so the verdict never reads as every partner's visits.
   **To judge every partner, build the full rows from the generator's fixtures (ace#2742):** read each
   partner's `user_visits.json` from its labs fixture folder, map every record through the visits
   pipeline's schema paths (`pipeline_get` → `schema.fields[].path`), cross-check that mapping against
   the 200-row home preview (every field of every previewed row must match its fixture row — any
   mismatch means the mapping is wrong and the full rows are not evidence), and pass the mapped rows
   as `--visits` (`{row_count_before_sample: <N>, rows: [...]}` with `row_count_before_sample` = the
   row count, so the scope reads as complete). Measured on `spark-facilitator/20261004-1706`: 1,594
   rows across five partners, 1,400 field comparisons against the preview, 0 mismatches.
   `--registry` is the registry as it stands — `7-synthetic/semantic-registry-author_registry.json`, or
   the `semantic_registry_get` response once it has been edited — check 28 fails without it.
   `--manifest-index` / `--dashboard-terms` are the two files `demo-data-setup` § C7 writes
   (ace-run) — check 29 fails when either is not passed or does not exist.
   It computes 1, 2, 22, 23, 21 and 24–29 itself, merges your outcomes, **fails every
   applicable check that was not evaluated**, and writes the `lib/qa-types.ts` shape
   through `aggregateQAResult` — which also fails a result that evaluated nothing.
4. Upload the file as `demo-data-setup-qa_result.yaml` in `7-synthetic/` (find-or-update).
   Never hand-write the YAML.

## Why check 7 exists — and what it deliberately does not cover

This skill's job, per `lib/artifact-manifest.ts`, is that **"a dead dashboard
must not reach a stakeholder."** Run as a boundary-fence heal against
`hh-poverty-targeting/20260730-2210` it returned **7/7 pass** — on a demo whose
review dashboard was analytically dead and whose walkthrough scored concept
2.0/5, user 1.0/5, arc 1.0/5 with 21 findings. Every check inspected the
HANDOFF (realized.json's shape, a URL against a regex, plan↔handoff agreement,
an integer, a date); none looked at what a `par_url` renders, and a regex cannot
tell a real run from a fabricated id. The gate would have passed in sequence and
prevented nothing (dimagi-internal/ace#1161).

**Check 8 — the decision behind it (#1162).** Two legitimate requirements
collide: a **completed** run carries a snapshot, which is what makes a `par_url`
a stable idempotent deep-link; an **`in_progress`** run is the only state in
which the review decision the narrative demonstrates can actually be taken.
Phase 7 completed BOTH runs on `hh-poverty-targeting/20260730-2210`, ~14 minutes
before the render, so the payoff scene had nothing to click — all 10 spec
actions degraded to `wait_for`/`hold`, 7 scenes produced 2 distinct images, arc
scored 1.0/5.

Resolved (Jon, 2026-08-14) as **option 1: leave only the interactive
dashboard's run live.** `source.dashboards[].role` already carries the signal,
so the producer needs no new plumbing, and the stability loss is confined to the
one dashboard whose entire point is that a stakeholder acts on it. Check 8 is
two-sided so the opposite sloppiness — every run left `in_progress`, silently
giving up snapshot stability on links a stakeholder keeps — fails too.

## Why check 9 derives its spec instead of accepting one (#1658)

`auditDataset` is only as good as the `DatasetSpec` handed to it, and the spec
used to be built by reading the PDD's prose. Under-declare one entry and the
gate reports a **measured** zero over a spec narrowed to exclude the finding —
#1346's failure mode displaced one level up, into the spec instead of the count.

Measured, same opp / same app / same generator:
`bednet-check-2-visit/20260817-1720` recorded check 9 `pass` justified as "no
counts, no currency, no conditional blocks", while
`get_opportunity_apps(2214, 'deliver')` returned
`"relevant": "/data/agree_again/consent_confirmed = 'yes'"` on both observation
fields verbatim. `20260825-1310` declared them honestly and measured **18 of
276** off-branch on each — the same 18 records. The difference between `pass`
and `fail` was the spec, not the data.

So the derivation is mechanical (`specFromDeliverApp`), and this check refuses
the two shapes that would restore the old behaviour: a clean audit with no
derivation behind it, and an `unparsed[]` expression treated as absence. The
second half of #1658 is why the auto-fix changed: the old hint demanded a
manifest-side constraint that connect-labs does not implement, so an honest run
on any gated form failed for a cause it could not remedy — and the only way to
green was to narrow the spec. The remedy is now the branch scrub, which is a
declared, reproducible, idempotent generator post-step.

## Why check 11 exists — one dataset, two totals, ten passing checks (#1683)

Checks 1–10 each inspect ONE dashboard: its URL, its payload, its bindings, its
run status, the dataset behind it. None of them can see a *disagreement*, because
a disagreement is not a property of any single dashboard — and a disagreement is
the one defect that makes a rendered page **misleading** rather than broken. A
broken page announces itself; a page that renders confidently and states
something untrue does not.

`hh-poverty-targeting/20260824-1404` built two dashboards over one synthetic
fixture (labs opp 10047) and they disagreed on 13 of 14 workers. From each page's
own `#workflow-data`:

```
run 5245 snapshot (period_end 2026-08-30) : total=2186 completed=1563 non_payable=623
run 5249 live                             : total=2237 completed=1592
fixture on Drive                          : total=2237 completed=1592 non_payable=645
run 5250 snapshot (period_end 2026-08-31) : total=2237 completed=1592 non_payable=645
```

Everything passed. Both runs reported their status cleanly, both pages rendered,
`fields_all_null` was empty, and this gate returned green — check 7 asks whether
bound fields are populated, not whether two dashboards over one dataset agree.
The only thing that noticed was the DDD **concept judge**, reading the numbers
off rendered frames and finding they contradicted the narrative's own figures
(the why-brief cites 1,592/645/2,237; the page rendered 1,563/623/2,186). That is
an expensive detector, and it fires only after a render.

The cause is a half-open window. `_date_window_where` in
`connect_labs/labs/analysis/backends/sql/query_builder.py` emits `visit_date >=
date_from AND visit_date < date_to` — correct for back-to-back weekly periods,
which is what its docstring justifies, and a trap for a whole-timeline window,
because the natural authoring move is to pass the manifest's own
`timeline.start_date` / `timeline.end_date`. Period scoping applies only on the
snapshot path (`get_snapshot_pipeline_data`, *"re-aggregated to that half-open
`[period_start, period_end)` visit-date window"*), so the live sibling keeps the
final day and the snapshot loses it. The producer-side fix is
`demo-data-setup` § Process step 3 (`period_end = timeline.end_date + 1 day`);
this check is the backstop that does not depend on anyone remembering it.

It is deliberately a check about AGREEMENT, not about any absolute number: it
needs no fixture, no manifest and no expected total — only two payloads claiming
to describe the same opportunity. And it is cheap: check 7 already fetched both.

## Why check 12 exists — the payoff dies on the NEXT render (#1864)

Check 7 asks whether a rendered dashboard's bound fields carry values. Check 12
asks whether an authored pipeline's DECLARED fields extract. Those sound like
the same question and are not, in three ways that all bit at once on
`spark-facilitator/20260828-0703`.

ACE authored pipeline 5414 with
`{"name": "records", "path": "form.meeting_date.date_of_meeting", "aggregation":
"count"}` on a fixture that carried no such path. An unmatched path is not an
error in labs — it aggregates to `0`. `records` came back `0` for all 12
facilitators, so `verifiedPct = pct(community, records)` rendered `—` on every
row, `judged = records >= MIN_RECORDS` was false everywhere, and the below-floor
filter matched **0 of 12**. The demo's entire payoff — narrow to the three under
the floor, record a decision on one — was dead. `avg_attendance` and
`avg_participation_pct` were null the same way.

Everything passed: this gate 12/12, `recipe_preflight` (targets resolved; it
does not read values), and all six visual judges. The only reason anyone found
out is that preflight happened to re-run on a cold cache 40 minutes later and
the `wait_for text:Showing 3 of 12 facilitators` failed.

Three reasons no existing surface could see it:

1. **labs' own `fields_all_null` is NULL-only.** Verified live 2026-09-06,
   pipeline 5411 / opp 10054, `schema_override` pointing three fields at
   `form.no_such_group.no_such_field`:

   ```
   rows            : records_bad_count 0, steps_bad_distinct 0, avg_bad null
   fields_all_null : ["avg_bad"]
   ```

   `count` and `count_distinct` of nothing are `0`, not `null`. The upstream
   detector misses exactly the field TYPE that gates a filter. Check 12 counts
   zero as dead, which is the single line of judgement the whole issue turns on.

2. **Check 7 enumerates from the RETURNED ROW COLUMNS** (`Object.keys(row)`), so
   a declared field the engine never emitted at all is invisible to it. Check 12
   enumerates from `pipeline_get(...).schema.fields`.

3. **Check 7 reads a completed run's FROZEN SNAPSHOT.** A snapshot minted while
   the values were good keeps rendering them after the binding rots — which is
   the defect's title in one line. Live today: pipeline **5411** still declares
   `records` on the broken `form.meeting_date.date_of_meeting`, and its dashboard
   still renders, because it renders a snapshot. Check 12 previews every
   pipeline the run authored, snapshot or not.

The `from_cache` refusal is the fourth: #1864's first render read a warm cache
and looked perfect. Rows that were not computed against the schema now saved are
not evidence about it, and `synthetic_reload_fixtures` (ace#1860) makes a fresh
one one call away — so a cached preview fails rather than passing.

## Why checks 14 and 15 exist — thirteen checks about DATA, none about READING (ace#2219)

Checks 1–13 are all about whether the numbers are right: bindings, pipelines,
extraction, totals, constraints, cohort size. Not one of them asks whether a
viewer can read those numbers off the frame the render produces. That is not a
gap in any individual check — it is a whole missing tier, and it is the tier
that has now produced five Phase 7 runs at concept **2.0/5**.

The first three (**ace#1841**) were fixed by adding a DETECTION verb to
`checkSceneCardinality`, keyed on the narrative's **vocabulary**. The fourth
(**ace#2131**) evaded that by being written in plain descriptive prose, and was
fixed by `checkDetectionCohortFloor`, keyed on a **declared field**. The fifth,
`poverty-graduation/20260905-1345`, came from a third and unrelated cause that
neither rule can see, because it is not about the data at all:

> The narration is strong throughout; **the frames under-carry it.**

`verdict-user.yaml` returned `projector_test: false` and
`five_second_read_correct: false` on **7 of 7** scenes. Two mechanisms, both
decidable before a single frame is recorded.

**Framing (check 14).** The spec used `kind: scroll` five times — `'400'`,
`'1280'`, `'bottom'`, `'320'`, `'top'` — and `scroll_to` once. Three of the five
are raw pixel guesses about how a page the spec does not own renders at a
viewport it does not control. Scene 5's landed a twelve-row table one row short:
row twelve, Umar Bello, *"sliced through its middle at the bottom edge with its
badge truncated mid-word"* — which, in the judge's words, *"spoils scene 6
without being readable"* — while the same scroll *"puts the title, the
simulated-data disclosure and all four KPI cards off the top"*. Scene 4's
`bottom` *"guillotines the two modal histogram bars (627 and 487) at the frame's
top edge — value labels gone, the two bars rendering at identical heights when
they differ by 140 households"*, while the panel the scene was actually about
sat fully framed below.

This is CLAUDE.md's *"don't guess at what another system owns"*, in the one place
nothing was checking for it. `scroll_to` names an element and lets canopy's
recorder find and centre it; a pixel offset is correct for one row count and one
font stack and reports nothing when it stops being correct.

**This is NOT the retracted ace#1660 check, and it adds no `offset`.** That
check flagged `scroll_to` for LACKING an offset and told authors to write
`offset: 96`, which `_ActionBase`'s `extra="forbid"` REFUSES — following it
turned a passing spec into one canopy rejects. Check 14 inverts it: `scroll_to`
is the remedy, taken exactly as canopy declares it (`kind` + `target`, nothing
else), and what gets flagged is the other action. Worth stating because today's
judge remedy reads *"offset so no partial histogram sits above it"* — literally
the retracted syntax. `test/lib/demo-frame-legibility.test.ts` pins every
remediation string against canopy's own action fields so the class cannot come
back a third time.

**Jargon (check 15).** The `'jargon visible to non-technical users, max 2'` hard
cap fired **independently on six of the seven scenes** — `31-point band`,
`10-point band`, `Surveys in the 31-point band`, `Mean likelihood below the
line`, `Payable` / `Non-payable`, *"coined terms defined nowhere on the page"* —
and *"one percentage column silently switches denominator at row 5"*. The
five-second read of scene 2 came out as *"steady ~460 submissions a week"*
rather than the point of the scene, because *"the grey series carries no number
and reads as chart padding"*.

The second iteration is the measurement that matters. A *"What each column
means"* panel was added; the judge recorded it **VERIFIED PRESENT** and correct,
defining all seven review-table columns; and the cap fired on six of seven
scenes **anyway**, because *"the panel is BELOW THE FOLD of every frame that
uses the vocabulary"*. So check 15 does not ask whether a definition exists on
the page — it asks whether a reader of the LABEL can reach it from the label.
In the judge's own words:

> Expanding an acronym is not the same as glossing it, and moving a definition
> on-screen is not the same as moving it to the point of use.

Both checks take DATA and decide from the artifact. Neither is keyed on a word
list, deliberately: ace#1841 pruned that list FOR precision and ace#2131 evaded
it anyway, so a third vocabulary rule would be the third instance of the same
mistake.

## Change Log

| Date | Change | Author |
|------|--------|--------|
| 2026-10-15 | **Check 26 stops judging the entity-count column (ace#2972); checks 24/26 name their visit sample (ace#2973).** connect-labs #2262 (2026-10-06, `indicator_report_render.js` `sameN`) drops the entity-count column whenever every row of a table has the same count and states it in the title instead, so `noora-health/20261015-1129` (one educator per facility, by design) failed check 26 on a column that was not on screen, with no legal fix. And `pipeline_preview` caps `sample_size` at 200 and returns only the home partner of a home-scoped visits pipeline, so the old "every partner, the whole population in one preview" contract was unachievable; `visitSampleScope` now writes the scope judged into the detail. | ACE team |
| 2026-10-19 | **New ace-run check 29 `cascade_build_artifacts_written` (ace#3251).** The cascade never wrote `demo-data-setup_manifest.yaml` or `dashboard-terms.yaml`, both required by `lib/artifact-manifest.ts`; this gate passed 10/10 on `connect-fi/20261018-1440` and Phase 7 finalize found 7/10 required artifacts after the DDD loop. Now the gate fails at build time. | ACE team |
| 2026-10-06 | **New ace-run check 28 `cascade_registry_text_is_plain` (ace#2748).** The registry's reader-facing text (targets note, labels, plain sentences, visit-flag descriptions) is linted by `lib/registry-display-text.ts` before the DDD judge reads it. spark-facilitator/20261004-1706's `targets_note` ("Targets are the PDD's own (§8.1 P1 ≥ 80%, P3 ≥ 75%)") alone capped DDD run `spark-facilitator-programme-cascade-2026-10-06-004` at clarity 2.0; the check fails it, and the same registry's `(PDD §5.4)` / `(PDD §7.2 S-1)` flag descriptions. New `--registry` input. | ACE team |
| 2026-10-06 | **New ace-run checks 24–27 — the cascade's BUILD output, judged before the DDD hand-off (ace#2735).** Of 89 findings on DDD run `spark-facilitator-programme-cascade-2026-10-06-002`, several were properties of the generated data and saved runs that a render was paying to discover: flagged records contradicting the PDD's review path, a worker rate over one entity, constant columns, a one-child drill level. `lib/cascade-build-qa.ts` + `cascadeBuildOutcomes`; each check fails on that run's real output in `test/lib/cascade-build-qa.test.ts` and passes a control repaired as its hint says. | ACE team |
| 2026-10-06 | Check 21 follows the ace#2727 plan rules: realistic `(example)` partner names (`Example partner A` now FAILS alongside bare `Partner A`), ≥ 5 invented partners, a display roster for workers and entities (`--real-people` names the inputs' real people none may carry), and headline indicators that spread ≥ 5 pts across partners in the plan AND in the graded runs. | ACE team |
| 2026-10-01 | Check 22's fix (and `checkWorkerReviewUrlScope`'s `auto_fix_hint`) now name the builder, `scripts/worker-review-url.ts --fix`, instead of a URL template to hand-assemble — hand assembly from the "Review →" links is how spark-facilitator/20260926-1800 shipped three `owning_program_id` links. | ACE team |
| 2026-10-01 | **The gate always runs, always writes a result, and 0 checks is a FAIL.** `bednet-check-2-visit/20260908-1544` ran 19 checks and hand-wrote them as `checks_total` / `checks[]`, so ace-web read the result as "Passed (0/0 checks)"; `spark-facilitator/20260926-1800` (ace-run provider) never ran the gate. New `scripts/demo-data-setup-qa.ts` computes the checks it can from local files, merges the live-labs outcomes, fails every check that applies to the provider (`checksForProvider`) but was not evaluated, and writes the canonical shape via `aggregateQAResult` (`lib/qa-types.ts`), which fails a result that evaluated nothing. New ace-run checks 22 (`worker_review_url_scoped`) and 23 (`cascade_handoff_complete`); check 21 now has `periodsFromHistoryRuns`. Run against Spark's real artifacts: 4/5 pass, and check 22 FAILS — all three worker-review URLs carry `owning_program_id` but no `program_id` (ace#2521). | ACE team |
| 2026-09-17 | **Check 8 now verifies the mandate it enforces, and carries an evidenced escape (ace#2430).** The row required the interactive run to be `in_progress` and failed it when `completed` — while nothing anywhere had ever loaded that page. The two states are not symmetric: a completed run's page returns early on its snapshot (`workflow-runner.tsx`: `if (snapshotCarriesPipelines) return;`), an `in_progress` one has no snapshot and re-runs the pipeline SSE stream on every load; check 7 even reads the interactive dashboard's data through `api/<def>/pipeline-data/`, a side-channel around the very stream the page uses. On `poverty-graduation/20260915-1518` that stream died after its first event, the dashboard could not load, and this row made the dead configuration the ONLY passing one. Now: an `in_progress` interactive run with no recorded `source.interactive_live_load` fails, one whose own probe says it cannot load fails, and a `completed` one passes only on an evidenced failure (`observed` + `run_id` + `control_run_id` + `upstream_ref`) — reported in `detail`, never silent. Check 18 stands down on the same granted escape, because a completed run's state is immutable (409) and the two gates would otherwise be jointly unsatisfiable. *Enforced:* `test/lib/interactive-live-load.test.ts` plus the new cases in `test/skills/demo-data-setup-qa/checks.test.ts` and `test/lib/labs-run-state-reset.test.ts`. | ACE team |
| 2026-09-17 | **Check 9's `declared_omissions[]` no longer offers a REPEAT-group roster as an exemption (ace#2432).** `auditDataset` could not see inside an array, so a correctly materialised repeat read as `conditional-missing` on every record and the escape's own remedy text told the next run to exempt it. Both are fixed: `lib/dataset-constraints.ts` descends into an array, and `OMISSION_HINT` now names the two surviving classes (`Trigger`, no-corpus image) and says to BUILD a repeat rather than exempt it. Measured on `poverty-graduation/20260915-1518`: three of its thirteen conditional-missing residuals were repeat children present on all 1,379 completed surveys. The check itself is unchanged — it is field-agnostic; what changed is what it reports as available. | ACE team |
| 2026-09-10 | **Check 18 gains the `render_reset_command_pinned` finding (ace#2351).** The check verified the reset block EXISTS and was `verified_at`; both were honest on `spark-facilitator/20260909-2242` and `20260910-0541`, whose commands were pinned to `/Users/<name>/.claude/plugins/cache/ace/ace/0.13.1413/…` and `…/0.13.1426/…`. A pinned path keeps resolving after `/ace:update` (the cache retains old versions), so the reset ran pre-ace#2325 code — no `--workflow-id`, every render `run-not-found` — with nothing to fail it. Now `classifyPinnedResetCommand()` flags `/plugins/cache/ace/ace/` and `/Users|/home/<name>/` in the registered command AND in the spec's `setup.command` (the string canopy actually runs). *Enforced:* `test/lib/labs-run-state-reset.test.ts`. | ACE team |
| 2026-09-17 | **Check 12 now carries labs' `fields_suspect` (ace#2431).** Every rule in check 12 judges a declared field by the ABSENCE of a value, so a field that extracted the WRONG value passes all of them — and `fields_all_null` is blind to it by construction. On `poverty-graduation/20260915-1518` a `count` over the JSONB base column `flag_reason` returned each worker's `total_visits` (235 / 315 / 260 against a truth of 17 / 9 / 11), because an unflagged visit stores `{}` and `NULLIF(flag_reason::text, '')` renders that as the non-empty string `'{}'`. Nothing was null, nothing was zero, every gate went green, and a wrong number is worse than a blank one because it reads as an answer. connect-labs#1882 fixed the SQL and added `fields_suspect` beside `fields_all_null`; `checkPipelineFieldsExtract` folds it in under the same obligation — never report clean where upstream is shouting. Reported, not blocking: both upstream signals are heuristics (an all-flagged cohort; a filter with no match in the sample), and labs itself reports rather than gates. | ACE team |
| 2026-09-08 | **New check 18 `render_reset_is_registered` (ace#2297).** A labs run's `spawned_tasks` persists server-side across renders, so on `spark-facilitator/20260908-2215` (labs run 5508) the scene whose payoff CREATES a coaching task aborted every render after the first — and iterations 0-2 passed only because a human had reset the state by hand, leaving no trace in `run_state.yaml`. Seventeen checks judged the dataset, the dashboards and the deck; none asked whether recording the demo LEAVES THE WORLD as it found it. The check is two-sided because each half alone is a no-op, and reports rather than fails the spec half when the narrative does not exist yet. |
| 2026-09-08 | **Check 9 gains the evidenced `declared_omissions[]` escape (ace#2225).** It had none, so a dataset whose only residual violations were fields labs STRUCTURALLY cannot simulate could never pass — the only routes to green were to narrow the spec (the behaviour ace#1346 / ace#1658 built the check to prevent) or to accept a permanent `fail`, and a permanently-red check is one nobody reads (ace#1744). Measured on `poverty-graduation/20260908-0510`: zero off-branch, zero integrality, empty `unparsed[]`, and six `conditional-missing` residuals that were a REPEAT-group roster, two `Trigger` read-aloud labels, and an image with no labs `ImageConfig` corpus. It takes check 13's shape — evidenced, refused when unevidenced, reported rather than silent — and covers BOTH halves the run tripped, the audit violation and the scrub's `unresolvedFields`; exempting only the first would have left it just as red. Off-branch, integrality and `unparsed[]` stay unexemptible. New producer step 2c.4 writes the list. | ACE team |
| 2026-09-08 | **New check 17 `scenes_show_more_than_one_thing` — the SEQUENCE tier.** Every other check in this skill judges ONE artifact; the arc judge is the only lens in the DDD loop that sees the scenes as a sequence, and it is where `spark-facilitator/20260907-1120` lost two of its three caps. 4 of 7 scenes were the payment ledger at four scroll offsets (`visual_variety` capped at 2, *"more than half the scenes are the same surface"*) and scenes 2, 3 and 4 each followed the previous with nothing but a scroll (`escalation` capped at 2, *"only a scroll between them"*). Both rules are decidable from the scene table before a frame is recorded — and the loop cannot fix either, because collapsing scenes is behind the `concept_change` gate, so the unattended run reported them and terminated `stopped_not_converged`. `checkSceneVariety` + `test/lib/demo-scene-variety.test.ts`, whose fixture is that run's scene table verbatim. | ACE team |
| 2026-09-08 | **New check 16 `rendered_columns_obey_declared_invariants` — the ARITHMETIC tier (ace#2250).** `spark-facilitator/20260907-1120` rendered *Steps closed out* above *Steps covered* on 6 of 12 rows, four of them above the seven-step ceiling the adjacent column header itself declares — while check 9 reported `0 unexempted violations` throughout. Check 9 was not wrong: no RECORD violated anything. The cause was one word in the pipeline schema — `steps_covered` used `count_distinct` over the step path and `steps_completed` used `count` over the completion flag, so three meetings closing one step contributed three — and the relation between two aggregates is not expressible in a spec derived from a form. The user-artifact judge capped `trust` at 2 on it, which alone held the run at 2.0. `checkColumnInvariants` + `test/lib/dashboard-column-invariants.test.ts`, whose fixture is the run's own live pipeline payload. `demo-data-setup` now declares `column_invariants` per dashboard and judges them at authoring time; this check is the backstop that does not depend on anyone remembering. | ACE team |
| 2026-09-08 | **Check 14 now says when it judged nothing (ace#2253).** `checkScrollFraming` reads `kind: scroll` only, so on a spec whose framing is all `scroll_to` it returned `pass: true, judged: 0, findings: []` — and on `spark-facilitator/20260907-1120` that was reported to the operator as "check 14 passed". It had approved nothing: three of those five `scroll_to` actions had silently failed to move the page (it CENTRES, and a centred position outside the page's scroll range clamps back to where the camera already was), the render replaced them with pixel scrolls, and once the spec of record was reconciled to what was actually rendered the same check returned two blocking findings. A non-blocking `scroll-to-framing-unjudged` finding now fires when `judged` is zero AND the spec has `scroll_to` actions framing judged stills — precisely the misleading case, and not on a mixed spec (which did evaluate something) or a spec that frames nothing by scrolling (which declined nothing). Judging `scroll_to` itself is the other half of ace#2253 and is NOT done here: whether a centred target is reachable depends on live page geometry, which a static check over the spec cannot see. | ACE team |
| 2026-09-07 | **New checks 14 `scroll_framing_is_anchored` + 15 `coined_terms_are_defined_on_page` — the LEGIBILITY tier (ace#2219, successor to ace#1841 and ace#2131).** The thirteen checks above are all about DATA; none asks whether a rendered frame can be READ, and that is where the fifth Phase 7 run at concept 2.0/5 came from. `poverty-graduation/20260905-1345` returned `projector_test: false` and `five_second_read_correct: false` on 7 of 7 scenes with the judge's headline "the frames under-carry it" — three raw pixel scrolls slicing a row in half and pushing four KPI cards off the top, and a jargon hard cap firing on six of seven scenes over coined column labels whose glossary panel sat below the fold. `checkScrollFraming` + `checkCoinedTerms` in `lib/demo-frame-legibility.ts` (+ `test/lib/demo-frame-legibility.test.ts`, whose fixtures are the run's own spec values and the judges' own quoted terms). Both read the ARTIFACT, not narrative vocabulary — the two prior fixes were each keyed narrowly and each evaded by the next run. Neither re-adds the retracted ace#1660 check and neither writes `offset`, which is pinned by a test. | ACE team |
| 2026-09-07 | **New check 13 `declared_detection_has_a_cohort` (ace#2131)** — the detection floor was enforced only through `checkSceneCardinality`, which reads the NARRATIVE's vocabulary, so a textbook outlier-detection demo written in plain descriptive language matched nothing. `bednet-check-2-visit/20260902-1555` carried zero detection tokens across its whole spec over a 5-worker cohort; the only cardinality finding was on the unrelated comparison/groups axis, and Phase 7 ended `stopped_not_converged` at 2.0/5 on the very objection the floor exists to pre-empt. Broadening the vocabulary was rejected — ace#1841 pruned that list FOR precision, and a report-only check that cries wolf is how real misses get waved through (ace#1744). The fact was already stated one layer up, without inference: `source.detectable_signal` and `source.data_shape.rows` are written by the same skill into the same block, before `demo-narrative` runs and long before anything renders. `checkDetectionCohortFloor` compares them. | ACE team |
| 2026-09-06 | **New check 12 `authored_pipeline_fields_extract` (ace#1864)** — the first check that judges an authored pipeline's DECLARED field list against a FRESH extraction, rather than a rendered payload's returned columns. Pipeline 5414 declared `records` on a path the fixture never wrote; `count` of nothing is `0`, so labs' null-only `fields_all_null` stayed silent, check 7 saw a warm-cached good snapshot, and the demo's below-floor filter matched 0 of 12 with every gate green. `checkPipelineFieldsExtract` in `lib/pipeline-field-extraction.ts` (+ `test/lib/pipeline-field-extraction.test.ts`), fed by one `pipeline_preview` per authored pipeline. Counts zero as dead, enumerates from the declaration, refuses a `from_cache: true` preview, and folds in `fields_all_null` so it can never fall below the upstream detector. | ACE team |
| 2026-08-26 | **New check 11 `cross_dashboard_totals_agree` (ace#1683)** — the first check that compares dashboards to EACH OTHER rather than inspecting one at a time. Two dashboards over labs opp 10047 disagreed by 51 visits / 29 completed because the snapshotted run's `period_end` equalled the fixture's last `visit_date` and the bound is exclusive; all ten existing checks passed and only the DDD concept judge caught it, after the render. `checkCrossDashboardConsistency` + `deriveVisitTotal` in `checks.ts`, operating on the payloads check 7 already fetches. Excludes program-scoped rollups by construction and honours an explicit `period_scope: 'partial'` declaration, so an intended sub-window is declared rather than silently tolerated. | ACE team |
| 2026-08-26 | Check 7 made runnable against a real payload (ace#1701). It read `instance.snapshot.pipelines` as an array of `{alias, rows}` while labs writes a dict keyed by alias, so it threw `pipelines is not iterable` on every completed run — and the `snapshot-missing-pipelines` branch was unreachable besides. Two further defects found once it could execute: it flagged labs' built-in row columns (null by `terminal_stage`, not by a wrong path — 15 findings on a healthy run) and it demanded a snapshot from the `in_progress` interactive run that check 8 requires, so 7 and 8 could not both pass. Now: both snapshot shapes read, built-ins excluded, and a non-completed run judged from supplied live pipeline rows with a `live-pipelines-unavailable` finding when they are absent. | ACE team |
| 2026-08-26 | Check 9 promoted to an importable `checkDatasetObeysPddConstraints` in `checks.ts`: the spec is derived from the deliver app (hand-declared entries are ADDITIONS), an unparsed `relevant` / `constraint` is a reported finding rather than a silent pass, a clean audit with no derivation behind it fails, and the auto-fix hint points at `scrubOffBranchFields` instead of a manifest constraint that does not exist. ace#1658. | ACE team |
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall