dimagi-internal/ace / skills/demo-data-setup
demo-data-setup
UnverifiedStand up a demo dataset and live labs dashboard, from a brief (de-novo synthetic) or a real opp (clone), and return the dashboard URL. Also run by /ace:demo.
Demo Data Setup
The data + dashboard half of the ACE demo workflow. Given a short demo
brief, it generates synthetic data and authors a live labs dashboard
dynamically (no production traffic, no ACE-built Connect opp required), then
hands the downstream demo-narrative skill a single artifact: the realized
${var} map — a flat JSON containing par_url (the polished dashboard
deep-link) plus any drill URLs.
This skill is the standalone/denovo sibling of the Phase 7 pair
synthetic-data-generate (data) + synthetic-workflow-seed (dashboard). It
composes the same connect-labs atom sequence those skills use, but decoupled
from opp.yaml / the PDD / a Phase 4 Connect opportunity — that decoupling is
the whole point of the demo entry point. For the detailed atom mechanics and
labs-side gotchas it does not repeat inline, it cites those two skills by
section.
Providers (the data-source seam)
| provider | source | status |
|---|---|---|
denovo | a short demo brief (this skill) | implemented (Plan A) |
clone | a real Connect opportunity id | implemented (Plan B) — synthetic_profile_from_prod(mirror) → synthetic_generate_from_manifest → the SAME dashboard-authoring spine as denovo, + a fidelity gate |
ace-run | the Phase 4 opp of a full /ace:run | Phase 7 convergence (Plan C) — Phase 7 becomes this provider. Builds the semantic-layer cascade (multi-partner synthetic programme + indicator report trio over a PDD-authored registry) — § Process (ace-run) C0–C7, ace#2510 |
All three converge on the same handoff: the realized ${var} map. Only the
front half (how the labs-only opp + its data come to exist) differs.
Inputs
| Source | Artifact | Used for |
|---|---|---|
| Operator (CLI) | --brief <text or drive-path> | the demo story: program, KPI focus, named FLWs, the anomaly to surface, the coaching beat |
| Operator (CLI) | --name <demo-name> | the demo folder ACE/<demo-name>/ |
| Procedure | demo run_id | the current demo run folder (scaffolded by agents/demo.md via buildDemoRunState) |
| Operator (CLI, optional) | --pin-monday <YYYY-MM-DD> | fixed timeline anchor; if omitted, compute a recent Monday and record it — never a sliding window (see Gotchas) |
Products
<demo-run>/7-synthetic/demo-data-setup_manifest.yaml— the per-opp generator manifest sent to labs; ace-run cascade: the index of the per-partner manifests archived under7-synthetic/cascade-build/(§ C7,scripts/cascade-build-artifacts.ts)<demo-run>/7-synthetic/realized.json— the handoff: a FLAT${var}map (DDD substitutes${var}verbatim — keep it flat, no nesting). One<key>_par_urlper dashboard the demo builds, plusprimary_par_url(the dashboard the walkthrough opens on) and any<name>_urldrills. E.g.{ "primary_par_url": ..., "program_admin_par_url": ..., "child_recovery_par_url": ..., "audit_good_url": ... }<demo-run>/7-synthetic/cascade-story.yaml— ace-run only: the story plan (partners, weeks, roster, four authored signals) that § C3 gates and § C6 verifies against the saved runs (lib/cascade-story.ts)run_state.yaml…products.synthetic.cascade— ace-run only: registry, program, partners, programme report, worker review, opp reports, cohort, history (§ C7)<demo-run>/7-synthetic/demo-data-setup.md— run summary (labs opp id, record counts, one par_url per dashboard, warnings)<demo-run>/7-synthetic/dashboard-terms.yaml— the step-3d legibility enumeration, one block per dashboard, so what a lay viewer must read and what the page tells them survives the run rather than living in a claim (ace#2219):
An EMPTYdashboards: - key: flw_review terms: # coined column/row labels a viewer meets - label: Surveys in the 31-point band surface: column header prominent: true # renders on a surface a frame shows alone definitions: # what the page tells them, and from where - term: Surveys in the 31-point band at_point_of_use: true # reachable FROM the label, not a panel elsewhere where: info affordance on the column headertermslist is a legitimate answer and must still be written — a recorded empty list is evidence, an omitted one is not.<demo-run>/7-synthetic/branch-scrub_report.yaml— the step-2c ledger, so the numbers survive the run rather than living in a claim (ace#1658):
An entry with a blankderivation: source: deliver-app # or `none` (+ `reason:` — the denovo provider) opportunity_id: <connect opp id> questions_seen: <int> gates_parsed: <int> unparsed: # the RAW specFromDeliverApp output — never edit to [] - kind: relevant # relevant | constraint field: <leaf name> path: /data/<group>/<question> expression: "<verbatim>" reason: "<why it could not be derived>" unparsed_resolutions: # how each unparsed gate WAS audited (ace#2497); check 9 - field: <leaf name> # clears an unparsed entry only on a field+expression match expression: "<verbatim>" # with a known resolution and a non-empty detail resolution: spec-addition # spec-addition | direct-measured-assertion | spec-correction | out-of-scope-form detail: "<the measured count, e.g. 0 of 167 violate>" additions: # hand-declared, merged via mergeDatasetSpecs; check 9 rebuilds conditional_fields: [] # the merged spec from these, so list EVERY addition (ace#2245) integer_fields: [] whole_currency_fields: [] cross_field_rules: [] unique_pairs: [] scrub: applied: true # false = report only; say why in `note` write_back: drive_update_file # drive_update_file | synthetic_register | none records: <int> total_cleared: <int> fields: - field: <leaf name> records_scrubbed: <int> records_gate_missing: <int> unresolved_fields: [] audit: # auditDataset AFTER the scrub total: <int> violations: [{kind, field, count}] declared_omissions: # step 2c.4 — residuals labs CANNOT emit (ace#2225) - field: <leaf name> # exempts this field's conditional-missing reason: "<the mechanism — a Trigger label CommCare submits no value for, an image with no labs ImageConfig corpus. NOT a repeat group: that one is buildable and auditable now, ace#2432>"reasonexempts nothing and is named in check 9's failure — the escape costs the same citationcheckDetectionCohortFloorasks ofbelow_programme_scale, and it never covers an off-branch value, an integrality violation or anunparsed[]gate.run_state.yaml.phases.synthetic-data-and-workflows.products.synthetic.source— the seam contract, populated:source: provider: denovo labs_synthetic_opp_id: <int ≥ 10000> deliver_units: [{slug, name}] narrative_context_ref: <drive path to the manifest> record_counts: # VERBATIM from synthetic_generate_from_manifest (ace#1670) opportunity: 1 user_visits: 276 user_data: 5 # the worker cohort — the row count a filter scene acts on completed_works: 0 completed_module: 0 data_shape: # the three axes demo-narrative's cardinality check reads rows: 5 # entities the dashboards enumerate one line per periods: 4 # distinct time buckets the timeline spans (manifest weeks) groups: 1 # distinct comparison groups (LLOs / sites / opportunities) dashboards: # one per dashboard the Step-0 plan selected - key: program_admin # → ${program_admin_par_url} in realized.json template: program_admin_report role: overview shape: run # saved run_id (env-ensure / workflow_save_snapshot) par_url: <url> - key: child_recovery template: sam_followup role: recovery shape: action # no saved run → mint run_id via workflow_create_run; SAME /run/?run_id= URL par_url: <url> - key: llo_review template: llo_weekly_review role: review-action # INTERACTIVE — its run stays in_progress (see step 3) shape: action interactive: true # set iff role is review-action / review / decision par_url: <url> primary_dashboard: program_admin payoff_control: testid:payability-decision # step 0 — the control the payoff # scene ACTS on. REQUIRED once # detectable_signal carries # below_programme_scale (ace#2316) realized_vars_ref: 7-synthetic/realized.json render_reset: # step 4b — the per-render reset contract (ace#2297) required: true # true iff a dashboard is `interactive: true` run_id: 5508 # the INTERACTIVE dashboard's run (the only resettable one) state_keys: [worker_states, spawned_tasks] # workflow_get → saved_runs.snapshot_inputs.state_keys command: >- # SELF-RESOLVING (ace#2351): never a plugin-cache or /Users path; --workflow-id REQUIRED (ace#2325) bash "$(python3 -c "import json,os;print(json.load(open(os.path.expanduser('~/.claude/plugins/installed_plugins.json')))['plugins']['ace@ace'][0]['installPath'])")/bin/ace-reset-labs-run" --run-id 5508 --workflow-id 5502 --opportunity-id 10060 --keys worker_states,spawned_tasks verified_at: 2026-09-08T21:00Z # when the command was last run green (or `null` + a reason) dataset_constraints: # step 2c, counts not claims (ace#1658) spec_source: deliver-app # deliver-app | none unparsed_expressions: <int> # >0 means gates this run did not audit off_branch_cleared: <int> # branch scrub, 0 is a MEASURED zero scrub_applied: true violations: <int> # UNEXEMPTED violations declared_omissions: <int> # step 2c.4, each with a reason (ace#2225) report_ref: 7-synthetic/branch-scrub_report.yamlrun_state.yaml.phases.synthetic-data-and-workflows.steps.demo-data-setup.status: done(+artifactpath)
Process (denovo)
-
Plan the demo from the ask (interpret + select templates). Turn the raw brief/ask (which may be a forwarded email) into a concrete plan BEFORE generating anything:
-
Enumerate the dashboards the ask needs. A narrative like "program management across LLOs AND individual children getting better" is two dashboards, not one. Give each a
key(e.g.program_admin,child_recovery) and arole. At most one dashboard may take an INTERACTIVE role (review-action,review,decision) — the one whose scene shows a stakeholder taking a decision rather than reading a figure. That role decides step 3's run handling, so choose it in the plan, not at render time. -
Name the DEMONSTRATION before you look at a single template (ace#2316). Derive it from the PDD, not from the palette: read the
verification_predicateand the payment unit, and write one sentence — what decision does this dashboard let a stakeholder take, that the programme's own rules require, and that a spreadsheet could not take at THIS opp's cohort size? Then name the on-page control that performs it and record it assource.payoff_control(a canopy target,testid:…).This ordering is the whole point. Pick a template first and the template's affordances silently become the demonstration. Measured on
spark-facilitator/20260908-2215: the dashboard was instantiated fromllo_weekly_review, whose only interactive affordances are (its own labs description) "an underperforming-only filter, and a one-click coaching task spawn". No control on the page could express the opp's actual payability rule, so the payoff became the filter — hiding 10 rows of 12 on a table that already fitted one screen. All four independent judges cappeduse_case_soundnessat 3 on exactly that, pinning concept below the 4.0 convergence bar. Three consecutive runs of that opp ended non-converged. If the best-fit template carries no control for your demonstration, author the control onto the page — do not re-aim the demonstration at what the template happens to offer. -
Select a checked-in template per dashboard — reuse over scratch for the DATA PLUMBING. Survey the palette with
mcp__connect-labs__list_templatesplus the labsconnect_labs/workflow/templates/library, and map each dashboard to the best fit for its pipeline and entity shape — that is where the reuse genuinely pays, because it is where the binding hazards live (ace#1160, ace#1894, step 3'scheckDashboardBindings). Reuse decides the plumbing; it does NOT decide the demonstration, which you already fixed above. Onlyworkflow_createfrom SCRATCH when nothing in the palette fits, and say so explicitly in the summary.Match on SHAPE, not on domain — the palette has EIGHT families and a domain-keyed lookup only ever finds one of them (ace#2321). Enumerate the live palette with
list_templatesand classify what comes back; the families below were derived from the 28 templates registered on 2026-09-09, so re-derive rather than trust this list if it looks stale. What matters is the unit of a row and whether a row carries a decision:Family Unit of a row Decision per row? Examples per-worker scorecard, one window one FLW for a period yes (statuses) llo_weekly_review,performance_review,chc_nutrition_analysis,mbw_auditing_v5cross-opportunity rollup one opportunity / LLO, drillable usually no program_admin_report,audit_par,kmc_programme_metrics,chc_audit_historylongitudinal per-beneficiary one beneficiary across follow-ups no sam_followup,kmc_longitudinaltrend over saved snapshots one period, read from ANOTHER dashboard's snapshots no flw_audit_trend_dashboard,flw_daily_indicator_tableautomatic indicator report one FLW-day / FLW-week, computed explicitly none flw_daily_indicator_report,flw_weekly_audit_reportaudit / adjudication one reviewed ITEM (a submission, a photo, a record) yes — the decision IS the artifact bulk_image_audit,muac_picture_audit,kmc_image_audit,weekly_dual_track_audit,audit_with_ai_reviewfan-out / work creation one spawned work item n/a — it creates program_audit_creator,ocs_outreach,kmc_flw_flagsfunder-facing verified coverage one ward / site no verified_monitoringIf your demonstration is "the platform DECIDED something about this record", you want the audit/adjudication family, not a scorecard. That distinction is the whole of ace#2316:
spark-facilitator/20260908-2215needed to show a meeting record being decided not-payable, was pointed by a nutrition-keyed fit list atllo_weekly_review(a per-worker scorecard), and inherited a filter-and-coach payoff that a spreadsheet beats at n=12. Six templates in the audit family existed the whole time and the selection method never looked at them. Reuse was never the problem; the selection method was.Worked examples, kept as examples and NOT as the map — these are one nutrition demo's answers:
- multi-LLO / FLW oversight →
program_admin_report(+chc_nutrition_analysisfor the FLW aggregate). - per-child recovery over follow-up visits →
sam_followup(MUAC + recovery-status timeline). Notkmc_longitudinal— it keys on weight, unusable when CHWs have no scales.
The
render_code-from-scratch prohibition below is scoped to those three nutrition templates and does not generalise. The invariants that DO apply to every dashboard, bespoke render code included, are: re-point the pipeline schema at the real form paths, verify withcheckDashboardBindingsbefore minting a run (step 3), and lint utilities pre-upload (step 3b). None of those argue against authoring presentation for this opp. - multi-LLO / FLW oversight →
-
Record each dashboard's
shapefromlist_templates.supports_saved_runs. BOTH shapes render at/labs/workflow/<def>/run/?run_id=<id>&opportunity_id=<opp>(verified live 2026-07-21). Shape only decides whererun_idcomes from: run-shaped (true, e.g.program_admin_report,chc_nutrition_analysis) reuses a saved run_id; action-shaped (false, e.g.sam_followup,kmc_longitudinal) has no saved run, so step 3 mints one withmcp__connect-labs__workflow_create_run. This decides step 4's URL build. -
Derive the data story per dashboard — personas, the anomaly/recovery arcs (MUAC recovery = children moving red[SAM]→yellow[MAM]→green across follow-up weeks), timeline. This becomes the manifest (step 1) and the narrative context
demo-narrativereads. Record the plan (dashboards[]with key/template/role) — it drives steps 1–5 and thesource.dashboardswrite-back.
-
-
Author the per-opp generator manifest from the brief.
Write
demo-data-setup_manifest.yaml. Structure + field rules are identical to the Phase 7 manifest — followskills/synthetic-data-generate/SKILL.md § Process(manifest schema) and its manifest-schema gotchas rather than re-deriving them. Demo-specific requirements:opportunity_id: a labs-only id ≥ 10000 (see Gotchas).flw_personas: first persona is the network manager (flag_rate: 0); the rest carryaccuracy_distributionreflecting the brief's quality spread.anomalies: 0-basedweekindices; only anomalies withreviewer_visible_in: [audit]mint audits. Put the brief's headline anomaly on a completed week.coaching_arcs: 1-basedweek_triggered; transcripts are authored verbatim from the brief's coaching beat.timeline.start_date: the pinned Monday (from--pin-mondayor computed); it MUST equal the env timeline anchor.
-
Generate the synthetic data — pick the mode per dashboard
shape/kind.There are TWO generation paths; a demo may use both:
- realize-env (composite multi-opp rollups —
program_admin_report,audit_par, and theirchc_nutrition_analysisweeklies): the data is produced by the ensurer chain (weekly_runs→run_audits→rollup), which only runs viamcp__connect-labs__synthetic_env_ensure env=<name>. Reuse the checked-inprogram-admin-reportenv (already synthetic MUAC nutrition, opps 10000/10001) or a new committed env. The realized${var}map it returns (par_url,wk4_url,*_good_url,*_incomplete_url) IS the handoff for those dashboards.generate_from_manifestalone does NOT populate a rollup (dry-run finding 2026-07-20). Program ownership: the env'skind: rollupresource MUST set aprogram_id(labs-only, ≥ 10000) so the cross-opp rollup is program-owned — its realizedpar_urlthen comes back scoped with&program_id(the only scope that renders a cross-opp rollup;&opportunity_id404s it). See connect-labs #946. Omittingprogram_idgives the legacy opp-owned rollup that 404s from any other opp context. - from-manifest (FLW-level or per-child dashboards —
sam_followup,llo_weekly_review):mcp__connect-labs__synthetic_create_labs_onlythenmcp__connect-labs__synthetic_generate_from_manifestwith a manifest whosebeneficiary_cohorts(withprogression: improvement_curvefor recovery arcs) +field_distributions(keyed by the template's exact form paths) produce the records the template reads. Mechanics + payment-unit pre-flight:skills/synthetic-data-generate/SKILL.md § Processsteps 1a–3.
Capture
labs_opp_id+deliver_units. - realize-env (composite multi-opp rollups —
2b. Capture the realized dataset's SHAPE, not just its URLs (ace#1670).
Keep synthetic_generate_from_manifest's record_counts VERBATIM — it is the
only place the generated cardinality is ever stated, and it is discarded today.
Then resolve the two axes record_counts cannot answer, from the manifest you
authored in step 1:
periods— the number of distinct time bucketstimelinespans (weeks betweenstart_dateand the last generated week).record_countscarries no dates.groups— the number of distinct groups a comparison could contrast (LLOs / sites / opportunities in the set).record_countscarries no grouping.rows— defaults torecord_counts.user_data(the worker cohort). Override it if the dashboards you planned in step 0 enumerate a different population (a visit-level table hasuser_visitsrows, notuser_datarows) — you are the only one who knows which, anddemo-narrativecannot re-derive it.
Write all of it into source.record_counts + source.data_shape (step 5).
Without it demo-narrative has no cardinality input at all and cannot tell a
filter over 5 rows from one over 500 — which is how
bednet-check-2-visit/20260825-1310 authored a filter demonstration against a
five-worker cohort and burned four render iterations before the concept judge
caught it.
2c. Derive the constraint spec from the APP, scrub the off-branch values, then audit — in that order (ace#1346, ace#1658).
The generated set is the substrate every dashboard reads, so it is checked
and repaired here — **before** step 3 mints a single run, not after step 4
has built the URLs. (This step used to run as `4b`, after the dashboards
were already reading the fixture; it moved so the scrub can land first.)
1. **Derive the spec from the deliver app, don't assert it from prose.**
`mcp__connect-labs__get_opportunity_apps(<connect opp id>, 'deliver')`
→ `specFromDeliverApp(appJson)` in `lib/dataset-constraints.ts`. It
reads every question's own `relevant` — plus any group-level `relevant`
above it — into `conditionalFields`, and every `Int`/`Long` question's
`constraint` into `integerFields` bounds.
**Never hand-declare `conditionalFields: []` off the PDD's prose.**
`bednet-check-2-visit/20260817-1720` did exactly that and recorded
check 9 `pass` on the justification "no conditional blocks", while
`get_opportunity_apps(2214, 'deliver')` returned
`"relevant": "/data/agree_again/consent_confirmed = 'yes'"` on two
fields verbatim. `20260825-1310` — same opp, same app, same generator —
declared them and measured **18 of 276** off-branch on each. The spec
decided the verdict, not the data (ace#1658).
- Hand-declared entries are **ADDITIONS**:
`mergeDatasetSpecs(derived.spec, additions)`. Currency fields,
cross-field rules, and PDD premises (`uniquePairs`, e.g. "1 CBF per
community") have no representation in the app JSON, so they still
come from the PDD — as additions on top of the derived spec, never as
a replacement for it.
- `derived.unparsed[]` is a RESULT, not debris. Each entry is a gate or
bound this run did **not** audit; hand-declare it as an addition, and
report what remains. `demo-data-setup-qa` check 9 fails on a
non-empty `unparsed[]` rather than passing quietly.
- Under the **`denovo`** provider there is no deliver app to read.
Record the reason verbatim (`noDeliverAppReason`) so the gate can
tell "nothing to derive from" apart from "nobody derived it".
1b. **The same response also carries every question's `calculate`. Replay
those expressions; never draw a computed field from a distribution
(ace#2328).**
`specFromDeliverApp` reads `relevant` and `constraint`. It ignores
`calculate`, and `get_opportunity_apps` returns one on every hidden /
`DataBindOnly` question — which is where an app keeps its ARITHMETIC.
Measured on deliver app `61eedcad279046d499d0f05f9ce3dc83` v10: **26 of
the form's questions carry a `calculate`**, and they are the payment
logic itself.
This matters because the manifest is a **distribution language** — it
draws every field independently, so it cannot keep two coupled fields
consistent. A drawn `is_payable` will disagree with the
`meeting_conducted` / `meeting_type` it is supposed to be a function of,
which is the arithmetically-impossible class ace#1346 already forbids,
arriving by a different route.
So when a demonstration turns on a value the app COMPUTES — a payability
flag, a per-entity counter, a deduplication key — do not add it to
`field_distributions`. Replay the app's own expression over the generated
records as a declared generator post-step, under the same three
determinism obligations as any other (§ Hand-authoring a generator).
Walk each entity's records in the order the device would have seen them,
because a `casedb` read is a function of history:
```
/data/is_payable ==> if(meeting_conducted = 'yes' and meeting_type = 'community_meeting', 1, 0)
/data/capped_index ==> min(/data/new_step_index, 3)
/data/key_date_part ==> if(/data/is_payable = 1, '', /data/date_and_place/date_of_meeting)
/data/entity_key ==> concat(username, ' - ', pilot_fcap_step, ' - ', capped_index, ' - ', …)
```
**What it is worth.** `spark-facilitator/20260908-2215` recorded these
fields as "DELIBERATELY NOT POPULATED" on the premise that their
expressions "live in the CCZ and are NOT exposed by
`get_opportunity_apps`" — false, and nothing here said so. Without them
the dataset held no payment decision, so the dashboard had only a
per-worker scorecard and the payoff became a filter over 12 rows; three
consecutive runs ended non-converged. Replaying them on `20260909-1211`
put the real decision in the data — 155 community meetings, 149 distinct
deduplication keys, and 6 records that are complete and correct and earn
nothing because they are the 4th on their step — which is a
demonstration no cohort size makes eyeballable.
2. **Scrub the off-branch values, then write the fixture back.**
`scrubOffBranchFields(records, spec.conditionalFields)` removes every
value the form's own `relevant` says cannot exist on that record's
branch, and returns a per-field report. It is idempotent and pure.
This is a declared, reproducible generator post-step — not hand-patching
records — and on this path it is the only remedy that works.
**The write-back must preserve `visit.id` as an integer, byte-for-byte
(ace#2249).** The labs generator mints `visit.id` as a 60-bit integer —
comfortably over JS `Number.MAX_SAFE_INTEGER` (2^53-1) — so a plain
`JSON.parse` of the fixture text silently rounds it (measured: 490 of
505 ids changed on one real run) before `scrubOffBranchFields` ever
runs. Read and write with `parseJsonPreservingBigInts` /
`stringifyJsonPreservingBigInts` (`lib/dataset-constraints.ts`) instead
of `JSON.parse`/`JSON.stringify` on BOTH ends of this step — they carry
any out-of-safe-range integer as a `bigint` through the round trip.
`scrubOffBranchFields` itself deep-copies with `structuredClone`, so it
no longer corrupts (or throws on) an id you hand it losslessly; getting
the surrounding read/write right is still on the caller.
**The manifest DOES carry a relevance primitive; it is inert here.**
`BeneficiaryCohort.relevance_groups: dict[str, RelevanceRule]`
(`connect_labs/labs/synthetic/generator/fixtures/manifest.py:300`) shipped
in `dimagi-internal/connect-labs#1331`, merged 2026-08-27. Declare it —
but do not expect it to do anything on a labs-only opp, and never let it
stand in for the scrub. Relevance is applied only to questions present in
the HQ `FormSchema`: the schema loop consults the gate per question
(`fixtures/fields.py:417-422`), but the trailing orphan-write loop is
gated by a set computed **once, before it runs** (`fields.py:484`), out of
the record built so far. A labs-only opportunity has no Connect
`app_structure`, so `parse_form_schema_from_app_json` returns
`FormSchema(questions=[])`, every declared path is orphan-written, and the
controller is not in the record when line 484 evaluates the gate — it can
never fire. Measured on `bednet-check-2-visit/20260828-0629` (labs opp
10052): `relevance_groups` was declared and the generator still emitted
**36** off-branch values, which the scrub then cleared (ace#1833).
`FieldDistribution.null_rate` is unconditional and `CorrelationSpec` can
make two fields co-vary but cannot make one ABSENT on a branch, so neither
substitutes either. On a schema-backed opp (`clone`, or an `ace-run` opp
whose deliver app resolves) `relevance_groups` does work — declare it
there, and still scrub (ace#1658).
Write the scrubbed `user_visits.json` back to the opp's fixture folder
(the `folder_id` `synthetic_generate_from_manifest` returned) with
`mcp__plugin_ace_ace-gdrive__drive_update_file`, **before step 3**. That
file IS what labs serves: `connect_labs/labs/synthetic/fixture_store.py`
loads `user_visits.json` from the registered Drive folder on every
labs-only opp read and caches it per `(opp_id, folder_id, endpoint_key)`
on FIRST read.
**Then DROP that cache — the write-back is invisible until you do
(ace#1860).** There is no safe window to write inside: the
`synthetic_generate_from_manifest` call PRIMES the fixture cache as part
of generating, so it is already populated when the atom returns, and an
in-place edit changes a file nobody re-reads. One call fixes it:
```
mcp__connect-labs__synthetic_reload_fixtures(opportunity_id: <labs_opp_id>)
```
It drops the fixture store across every worker process plus the raw and
computed analysis rows a pipeline reads, and it REPORTS what it dropped —
`{invalidated: {registry, fixture_store, sql_cache: {raw, computed_visit,
computed_flw, computed_entity}}, visit_count}` — so "the scrub landed" is
observed rather than assumed. Re-run `pipeline_preview` after it and
confirm the numbers match the scrubbed fixture.
Skipping it fails SILENTLY and green: on
`spark-facilitator/20260828-0703` the first `pipeline_preview` after a
correctly-written scrub still returned `records: 0` (a field the scrub
adds), `avg_attendance: null`, and `community_meetings` counting the
not-held records — the exact off-branch value the scrub had already
removed on Drive. Nothing errored, and no gate could see it: check 9
audits the producer's LOCAL scrubbed copy, and checks 7 and 11 pass
because both dashboards read the same stale cache and therefore agree
with each other.
- **If ACE's service account cannot write that folder** — the grant in
`skills/synthetic-data-generate/SKILL.md` step 3a asks for *Reader* —
copy the five fixture JSONs into an ACE-owned folder, apply the scrub
there, share it with the labs fixture service account, and re-point
the opp with `mcp__connect-labs__synthetic_register(opportunity_id,
gdrive_folder_id)`. The cache key includes `folder_id`, so a folder
change misses in every worker and re-pulls. This is the heavier path,
and it is for a PERMISSIONS failure only — when ACE can write the
folder, `synthetic_reload_fixtures` above is the one-call remedy.
- **If neither write lands this run**, still run the scrub in memory,
write the report with `applied: false`, and carry the counts into the
summary and `run_state`. What is forbidden is narrowing the spec until
the count disappears.
3. **Audit what now stands, and report COUNTS not claims.**
`auditDataset(records, spec)` over the records as they now are. Put the
per-class counts in the summary. The labs manifest is a **distribution
language**: it draws every field independently, and integers are
enforced only where the HQ form schema types the question `Int` — so a
legal-looking set can be arithmetically impossible.
`spark-facilitator/20260813-2126` wrote *"0 constraint violations, all
hand-checked"* into `run_state.yaml` for a set with 251 fractional
people-counts, 242 fractional Kwacha amounts, 34 off-branch reasons, 22
did-not-happen meetings carrying full attendance blocks, and
facilitators roaming 190 distinct (facilitator, community) pairs against
a stated 1 CBF per community. A measured zero and an asserted zero read
the same in `run_state.yaml`; only one of them is true (ace#1346).
4. **Declare — with a reason — the residuals labs STRUCTURALLY cannot
emit (ace#2225).** Some fields the app declares can never appear in a
generated set, however correct the spec and the scrub are. Two recurring
classes, both measured on `poverty-graduation/20260908-0510`:
| class | why no value can exist |
|---|---|
| a **`Trigger`** read-aloud label (`roster_intro`, `ppi_intro`) | CommCare submits no value for one at all |
| an image with **no labs corpus** (`dwelling_photo`) | labs `ImageConfig` ships MUAC and scale corpora only |
**A REPEAT group is no longer one of them — build it, don't exempt it
(ace#2432).** This table's first class used to read *"a per-member REPEAT
group … the generator emits one flat object per visit, not an array, so a
roster row has nowhere to live"*. Both halves of that premise have since
been refuted, three months apart, and each on its own is enough:
`BeneficiaryCohort.repeat_groups: dict[str, RepeatGroupSpec]` is in the
labs manifest schema
(`connect_labs/labs/synthetic/generator/fixtures/manifest.py`), and a
hand-authored generator post-step (§ Hand-authoring a generator) can
write the array directly — `poverty-graduation/20260915-1518` did, on all
1,379 completed surveys; and `auditDataset` now descends into an array,
so a materialised roster resolves as PRESENT rather than reading as
absent on every record. What still holds is only this: a manifest that
does not DECLARE a repeat draws `member_count` directly and emits no
roster — which is a build that skipped the repeat, not a value that
cannot exist, so it is not exemptible. The stale wording cost that run
three `declared_omissions` entries exempting data that was present and
correct, which is the gate losing resolution exactly where the dataset
got better.
Write each as a `{field, reason}` entry in `declared_omissions[]` in
`branch-scrub_report.yaml`. Check 9 then exempts that field's
`conditional-missing` violation and its `unresolvedFields` entry — and
**only** those two: an off-branch value, an integrality violation and an
`unparsed[]` gate are never exempt, because each names a value that IS
present and wrong, which the scrub or an added spec entry can fix.
**The reason is required and is the whole point.** An entry with a blank
reason exempts nothing and is named in the failure, the same way
`checkDetectionCohortFloor` refuses an unevidenced `below_programme_scale`.
Name the mechanism, not the symptom: *"labs ImageConfig ships MUAC and
scale corpora only"* is a reason; *"cannot be generated"* is the
spec-narrowing this gate exists to prevent, spelled differently. The
exemptions are echoed into check 9's own detail string, so they reach
the run summary rather than disappearing into a green verdict.
Products: `7-synthetic/branch-scrub_report.yaml` (the derivation +
scrub + audit ledger + `declared_omissions[]`, schema in § Products) and
the `source.dataset_constraints` block written in step 5.
3. Author each planned dashboard dynamically. Loop over the Step-0
dashboards[]; for each, run the ADAPT-or-SCRATCH flow from
skills/synthetic-workflow-seed/SKILL.md § Process:
mcp__connect-labs__workflow_create_from_template (ADAPT — the default; pass
the dashboard's selected template) or mcp__connect-labs__workflow_create
(SCRATCH — only when nothing fit) →
mcp__connect-labs__pipeline_update_schema →
mcp__connect-labs__workflow_update_render_code /
mcp__connect-labs__workflow_patch_render_code →
mcp__connect-labs__workflow_create_run →
mcp__connect-labs__workflow_save_snapshot — except for the INTERACTIVE
dashboard, where you STOP after workflow_create_run and never call
workflow_save_snapshot (see § The interactive run stays live). Reuse that skill's
alias-consistency, period-scoping, and snapshot-hook guidance by reference —
they are the difference between a populated dashboard and a blank one. Capture
each dashboard's <def_id> + saved <run_id>.
Prove every declared pipeline field extracts, before you mint a run
(ace#1864). After each pipeline_update_schema, call
mcp__connect-labs__synthetic_reload_fixtures(<labs_opp_id>) and then
mcp__connect-labs__pipeline_preview(pipeline_id, opportunity_id, sample_size >= 10), and run the rows through checkPipelineFieldsExtract
(lib/pipeline-field-extraction.ts) against the schema you just saved.
Confirm per_opp_metadata[<opp>].from_cache is false first — a warm cache
is not evidence about the schema now saved, and on
spark-facilitator/20260828-0703 the first render read one and looked
perfect while the same run recomputed 40 minutes later and returned zeros.
Pass the response's fields_all_null and fields_suspect straight
through — the check folds both in, so it can never report clean where labs
is already shouting. fields_suspect (connect-labs#1882) is the only signal
on either side that sees a field which extracted the WRONG value rather than
none, because every deadness rule judges a field by what it is MISSING. On
ace#2431 a count over the JSONB column flag_reason returned each
worker's total_visits — 235 / 315 / 260 where the truth was 17 / 9 / 11 —
and nothing was null, nothing was zero, and every gate went green. A wrong
number on a dashboard is worse than a blank one: it reads as an answer.
Do not read the response's own fields_all_null as the answer. It is
NULL-only, so count / count_distinct over zero matched records read as a
healthy 0 — precisely the field type that gates a demo's filter. Verified
live 2026-09-06 on pipeline 5411 / opp 10054 with a schema_override
pointing three fields at form.no_such_group.no_such_field: the rows came
back records_bad_count: 0, steps_bad_distinct: 0, avg_bad: null, and
fields_all_null named ["avg_bad"] alone. That is the whole of #1864 —
pipeline 5414 declared records on form.meeting_date.date_of_meeting,
every facilitator counted 0, verifiedPct rendered — on every row, and
the below-floor filter matched 0 of 12, killing the demo's payoff scene.
Fixing it is one pipeline_update_schema here; discovering it after the
render costs a whole iteration.
Enumerate coined terms from EVERYWHERE a frame will show them, not only
the column headers. Check 15's input is a list you write, so an
under-enumerated list passes vacuously — the enumeration IS the check. On
spark-facilitator/20260907-1120 it passed both dashboards (13 and 11 labels
judged, 0 findings) while the user judge capped clarity at 2 on "a repeated
entity key", cited independently by six of seven judges. That phrase was body
prose inside the How to read the columns panel — inside the very surface
whose job is to remove jargon — so nobody enumerated it and nothing judged it.
Sweep the render code for KPI subtitles, column subtitles and every explainer
sentence, and write the panel itself in programme language: a reader who needs
the glossary cannot be sent to a glossary that needs one.
Match the aggregation to the noun the COLUMN HEADER promises, and declare
the relations between columns. count counts RECORDS; count_distinct
counts THINGS. Rendering one against the other as if they were the same noun
produces a number that cannot be true, and no check below this line can see
it — check 9 derives its spec from the Deliver app, and the app does not know
what a dashboard will put in a column header. spark-facilitator/20260907-1120
shipped exactly this, and it cost the run:
steps_covered count_distinct form.fcap_step_screen.pilot_fcap_step
steps_completed count form.closing.step_completed (= yes)
Three meetings closing the same step contributed three to steps_completed
and one to steps_covered, so "Steps closed out" exceeded "Steps covered" on
6 of 12 rows and four rows exceeded the seven-step ceiling stated in the
adjacent header. Check 9 reported 0 unexempted violations throughout —
correctly, because no RECORD was wrong. The user-artifact judge capped trust
at 2 on it, and a trust cap alone holds the whole run at 2.0. The fix was
one word: count_distinct over the STEP path, filtered on completion.
So, for every dashboard, write down what must hold between its rendered
columns and put it in the manifest as column_invariants (per dashboard
key), each entry {left, op, right, because} with right either another
field name or a literal bound the header states:
column_invariants:
weekly_review:
- {left: steps_completed, op: "<=", right: steps_covered,
because: 'A step cannot be closed out without being covered.'}
- {left: steps_covered, op: "<=", right: 7,
because: 'The column header states "of the 7 Goal Setting steps".'}
- {left: payable, op: "<=", right: records,
because: 'Only a filed record can earn a payment.'}
Judge them here with checkColumnInvariants
(lib/dashboard-column-invariants.ts) over the same pipeline_preview rows
you just proved extract, and fix the SCHEMA rather than the data when one
fails — a violated relation is almost always an aggregation reading the wrong
noun. Check 16 re-runs this independently, so a skipped declaration surfaces
there rather than reaching a funder. Declare the empty list deliberately if
a dashboard's columns genuinely bear no relation — a recorded empty list is
evidence and an omitted one is not.
period_end is an EXCLUSIVE bound — pass timeline.end_date + 1 day, never
timeline.end_date itself (ace#1683).
A run's window is half-open. _date_window_where in
connect_labs/labs/analysis/backends/sql/query_builder.py emits
visit_date >= '{date_from}' AND visit_date < '{date_to}', and its own
docstring says why: "The window is half-open — >= date_from AND < date_to —
so back-to-back periods (week N ending == week N+1 starting) do not
double-count the boundary day." Correct for consecutive weekly slices, and a
trap for a whole-timeline window: the natural authoring move is to pass the
manifest's own timeline.start_date / timeline.end_date, and a fixture whose
last visit_date IS timeline.end_date then loses its entire final day.
So, when minting a run that a workflow_save_snapshot will freeze:
period_start = <manifest timeline.start_date> # inclusive, as authored
period_end = <manifest timeline.end_date + 1 day> # EXCLUSIVE — add the day
The manifest already carries timeline.end_date, so this is derived, not
guessed. State the derivation (period_end = end_date + 1d) in the run summary
so the next reader does not "correct" it back.
Two things that make this silent rather than self-announcing:
- The scoping only bites on the SNAPSHOT path.
period_scopedis consulted inWorkflowDataAccess.get_snapshot_pipeline_data(connect_labs/workflow/data_access.py, "re-aggregated to that half-open[period_start, period_end)visit-date window"), so anin_progressrun reads the all-time cache and shows the full fixture. A snapshotted dashboard and its live sibling over one dataset therefore disagree while both reportstatus: completed/in_progresscleanly and both render. Measured onhh-poverty-targeting/20260824-1404(labs opp 10047), from each page's own#workflow-data: run 5245 snapshot (period_end 2026-08-30)total=2186 completed=1563 non_payable=623; run 5249 livetotal=2237 completed=1592; the Drive fixturetotal=2237 completed=1592 non_payable=645; re-minted as run 5250 withperiod_end 2026-08-31→2237 / 1592 / 645, exact match. Same definition, same pipeline, same fixture — onlyperiod_endmoved. - Omitting the period is NOT "no window", it is an EMPTY one.
connect_labs/mcp/tools/workflow_create_run.pydoesperiod_start = period_start or today/period_end = period_end or today, so an unqualifiedworkflow_create_run(definition_id, opportunity_id)mints[today, today)— zero-width. Harmless while the run is live (no scoping), and zero rows the moment aperiod_scopedpipeline is snapshotted. Always pass both bounds explicitly for a run you intend to snapshot.
demo-data-setup-qa check 11 (cross_dashboard_totals_agree) is the
backstop — it compares the shared visit total across every dashboard on one
labs_opp_id, which is the only signal that surfaced this without an LLM judge.
ADAPT is the default because reuse carries LEARNED PATTERNS — not because
authoring is forbidden (ace#2321). program_admin_report /
chc_nutrition_analysis / sam_followup are checked-in templates; start
from the best-shaped one via workflow_create_from_template.
This line used to read "never build render_code from scratch." That was
an uncited design preference, not a lesson. It entered in daad0ed9
(2026-07-20), a FEATURE commit introducing template selection — no incident,
no run id, no issue, and nothing in the tracker about a bad de-novo
dashboard. Meanwhile it has now demonstrably COST something: read as a
general ban, it made the template's affordances the demonstration on three
consecutive spark-facilitator runs (ace#2316). A prohibition with no
reproducer behind it is exactly what CLAUDE.md § A guard that PREDICTS another system's rejection must cite a reproducer refuses to ship.
So: author the presentation and the controls this opp's demonstration needs, and make them GREAT. What you may not do is skip the three invariants, none of which care whether the code came from a template:
- Re-point the pipeline schema at the real form paths the generator
writes, and verify with
checkDashboardBindingsbefore minting a run (step 3 — ace#1160, ace#1894). This is where reuse genuinely pays and where de-novo work genuinely risks a dead page. Style withwindow.LabsReportfirst (playbook/integrations/connect-labs.md§ "Stylerender_codewithwindow.LabsReport"): build the page from the shared report library, and add a missing look to the library rather than inlining it. - Lint utilities pre-upload (step 3b — ace#1662): labs purges Tailwind
against its own Django templates, so a utility only your
render_codeuses is dropped silently. - Author the layout against step 3c-layout's rules (ace#2319) — six deductions that cost five of eight convergence lifts on one run.
A dashboard that clears those three and carries the control its demonstration needs is better than a template-shaped one that does not, whatever its provenance.
ADAPT means RE-POINT — a template-instantiated pipeline is not wired
until you change its schema (ace#1160). Run
checkDashboardBindings from lib/dashboard-bindings.ts over each
authored workflow before minting its run.
Feed it the workflow_get response, not a hand-built object. Its
pipeline_sources comes back as an ARRAY of
{pipeline_id, alias, name, schema_summary} — pass that through
unchanged. The fourth check below can only be decided from the
resolution metadata that array carries, and passing the stored
{alias: id} dict instead silently gives up on it (ace#1894).
Four things it decides:
-
stock-template-path— the pipeline still extractsform.meta.*. The synthetic generator writes the run's REAL Deliver-app form paths (form.visit_summary.*,form.ppi_indicators.*) and NEVERform.meta.instanceID/timeEnd/appVersion, so every stock field resolves null or zero. Re-point the schema at the same paths the scorecard pipeline already resolves. -
snapshot-missing-alias—snapshot_inputs.pipelinesmust cover every alias inpipeline_sources, or a completed run snapshots no rows for it. -
pipeline-declared-but-unread/unbackfilled-counter— the render must actually read the pipeline. Binding a denormalizedworker.visit_countinstead reads 0, because the generator writes UserVisit fixtures without back-filling that counter. -
pipeline-unresolvable-in-scope(ace#1894) — the declared pipeline id does not exist in this workflow's own scope. Pipelines are opportunity-scoped, andworkflow_clonecopiespipeline_sourcesverbatim while — by its own atom description — "NOT clon[ing] linked pipelines". So a cross-opp clone declares ids it cannot read, reports success, and renders empty with no error. The signal is already in theworkflow_getresponse: an unresolvable source comes backname: nullwithschema_summary.field_count: 0, because there was no row to read them from. Fix by re-pointing the alias at a pipeline in this scope, or authoring one here.So do not reach for
workflow_cloneto carry a repaired dashboard into a new run's opp. It is the obvious move on a forked or repeated run and it produces a dead page. Instantiate from template here and re-point, which is what this step already does.
Live: hh-poverty-targeting/20260730-2210 workflow 5069 hit all three at
once. Every worker row showed VISITS 0 beside a chip reading
visits: 835, while the sibling scorecard (5065, correctly authored)
credited the same 8 people with 31–125 visits. The data was fine —
total_visits in the same pipeline summed to exactly 835. Three truthful
reads of three different stores; the bindings were wrong. Downstream the
DDD render+judge scored concept 2.0/5, user 1.0/5, arc 1.0/5.
3b. Lint every render_code against the deployed stylesheet BEFORE you
upload it (ace#1662). A non-resolving Tailwind utility is a pre-upload
FAILURE, not a silent no-op.
labs purges its Tailwind bundle against its OWN Django templates. A
workflow's render_code lives in the labs DATABASE and is never scanned,
so any utility labs does not itself use is dropped from the shipped
bundle — and then degrades to the unstyled baseline rather than erroring:
a missing bg-* is transparent, a missing text-* is the inherited
near-black, a missing border-* is the default grey, a missing h-*
collapses the element to 0px. Nothing in the render, the page console, or
demo-data-setup-qa observes it — that gate checks the dashboard is a
live deep-link and paints content, not that it painted in the colours the
code asked for.
Run the check on the FULL post-edit source, before every
workflow_update_render_code / workflow_patch_render_code:
ACE_ROOT="${CLAUDE_PLUGIN_ROOT:-$(python3 -c "import json,os; d=json.load(open(os.path.expanduser('~/.claude/plugins/installed_plugins.json'))); print(d['plugins']['ace@ace'][0]['installPath'])")}"
node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/check-render-code-utilities.ts" <render-code-path> --substitute
# exit 0 = every utility resolves · exit 1 = MISSING utility or a count on a
# hard-coded plural (ace#2248), do NOT upload
# exit 2 = could not read the stylesheet — fix that first; do NOT proceed
- Lint the whole source, never just the patch hunk. On
bednet-check-2-visit/20260825-1310, three of the five misses on workflow 5230 were PRE-EXISTING — not introduced by the edit that found them. For a patch:workflow_get→ apply locally → lint → upload. - On MISSING, substitute.
--substitutenames the resolving near-neighbours. The repair that shipped:text-rose-700→text-red-700,text-rose-800→text-red-800,border-rose-300→border-red-300,bg-emerald-600→bg-emerald-500,border-slate-300→border-gray-300(card) /border-slate-200(panels).text-rose-700styledconsent 89.7% · below the 90% floor— the only pay-affecting figure on that dashboard. - A geometric or arbitrary-value miss has no near-neighbour — a
different value is a different design. Drop the class and set the
property inline:
className="relative h-28 w-full"→className="relative w-full" style={{ height: 112 }};min-w-[52px]→style={{ minWidth: 52 }}. The missingh-28on workflow 5227 left all 12 weekly bars at 0px — a whole invisible chart panel, no error. - The purge is per-UTILITY, not per-family, and nothing mirrors.
text-slate-700ships whilebg-slate-400andborder-slate-400do not. Arbitrary values are not categorically blocked either (text-[11px]ships,min-w-[52px]does not). The rule is the exact string — never reason from a resolving sibling. - Leave every PRESENT utility alone. Swapping a working class for another working class is unforced churn.
- A count concatenated onto a phrase carries its singular form (ace#2248).
Never
t.committee + " were committee meetings"— at a count of 1 that renders "1 were committee meetings" on a funder-facing panel, and 1 is the likeliest value a per-cause count takes. Writet.committee + (t.committee === 1 ? " was a committee meeting" : " were committee meetings"). The same script lints this statically (count +" were|are|have …", or a.length/count-named value +" <plural noun>") and exits 1 on a hit.
Root cause tracked upstream as connect-labs#1294 (reject a non-resolving
utility at labs' render_code write boundary). This gate earns its keep
regardless: the available set drifts whenever labs changes its own UI.
3c. Author dashboard copy for the gap-copy check, and record the render
sources it will scan (ace#1750, ace#1759, ace#2244). A
why_brief.yaml declares typed gaps[] — what the build cannot support.
demo-narrative gates the NARRATIVE against them and nothing gated the
dashboard's own on-screen prose, so a demo can render a page asserting
exactly what its gap list says is unsupported. The brief and the spec can
do it too: on the same run the brief's own decisions-are-recorded spine
item asserted the disposition is recorded "with its reason" while the gap
two screens down declared there is no durable register and no reason field.
This step does NOT run the check — it cannot (ace#2244). The gaps are
declared in why_brief.yaml, which demo-narrative authors one step LATER
(Phase 7 Step 2, after this skill's Step 1; /ace:demo orders them the same
way), so here there is no gap list to check against. The check runs in
demo-narrative § Step 3d, after the brief and spec exist and before
anything renders — still cheap to fix, because a dashboard finding is a
workflow_patch_render_code on a page nobody has filmed. What THIS step
owes it:
- Record each dashboard's labs
workflow_id+ owning scope so 3d can read the render back. The id is in itspar_url(/labs/workflow/<workflow_id>/run/?run_id=…); the scope islabs_synthetic_opp_idfor a per-opp workflow,program_idfor a program-owned one. 3d reads the UPLOADED render_code withworkflow_get, so a later patch is scanned too — never a local copy. - Write gap-safe copy from the start using the rules below and the PDD's own limits; 3d will flag what still repeats a declared gap's subject.
The rules the check applies (so you author to them): it derives each CAPABILITY gap's subject terms (the words the author used
in BOTH the gap id and its detail, minus generic product vocabulary) and
reports every prose line that repeats one. RESEARCH and DECISION gaps do not
constrain what a surface may name — they forbid a qualified claim, not the
subject: RESEARCH ("we don't know the real rate") forbids a quantified claim,
DECISION ("the threshold values are unchosen") forbids asserting a value.
Telling "named it" from "asserted the qualified thing" is a judgement no
keyword match should make, and on the run below every DECISION hit was the
gap's own proposed remedy ("recording the disposition is what makes a
threshold tunable").
It FLAGS, it does not reject — it cannot know which phrasings are load-bearing, and refusing legal copy on a keyword match would be the ace#1238 guard-predicts-a-rejection failure. Resolve each finding: reword the copy, or record why the line is legitimate.
Measured on hh-poverty-targeting/20260827-0323, whose why-brief declared
area-register-does-not-exist (area_ref present on 0 of 2,237 records)
and adjudication-log-is-run-state-not-a-register. Four instances shipped
into two dashboards — "outside the normal range for this area",
"colleagues working comparable areas", "area composition drives it",
"written to the adjudication log with its reason" — and one of them
contradicted the same page's own leave-one-out definition two lines above it.
In every case the narration was clean and the UI copy was the offender, so
only a judge reading rendered pixels caught them, after a full render.
Known limitation, stated so it is not mistaken for coverage: this matches
subject-term repetition, not semantic equivalence. Two shapes still slip, both
observed on that run's iteration-2 judge pass. A coverage claim phrased
without the gap's own nouns — "that is what makes census saturation auditable"
— is NOT caught. Neither is an INVERTED claim: scene 4's payoff line said low
variance → fabrication where the page says fabrication → low variance, the
converse, and exactly what the detection-rates-are-not-evidenced RESEARCH
gap calls unevidenced. Both sentences carry the same nouns in the same
proportions, so no term match can separate them. The check narrows the class;
it does not close it, and the post-render judge is still the backstop.
3c-layout. Author the LAYOUT against the judges' own deduction rules, not
against the template's defaults (ace#2319). Five of the eight dimension
lifts spark-facilitator/20260908-2215 needed to converge were layout, not
story, and every one of them is avoidable at authoring time. These are the
deductions that actually fired, each stated as the rule that prevents it —
they are transcribed from that run's verdict-concept / verdict-user
findings, so they are what the judge does measure, not what it might:
- A caption that claims an ORDER must render in that order. The legend
captioned "The seven FCAP Goal Setting steps, in order" was laid out
column-major — 1,3,5,7 down the left and 2,4,6 down the right — so it
contradicted its own caption. If you write
columns-2/grid-flow-colover an ordered list, either read row-major or drop the ordering claim. - A container holding a filterable list cannot have a fixed height. The
step panel kept ~165px of empty space below "7 Vision" after its list
collapsed from 12 rows to 3, because the container did not reflow. Any
panel whose children come from a filtered
.map(sizes to content. - Load-bearing text is not the smallest type on the page. Seven definitions that decide whether a number can be read at all ("it took place, and it was a community meeting", "target 85%+", "of the seven") were set in the page's smallest size, firing the load-bearing-content-too-small rule. If a viewer must read it to read the number, it is body text.
- A header block is not taller than the rows it labels. Two dashboards ran ~130px of header — taller than four data rows — with labels on four different vertical baselines. Set one baseline and let the data have the frame.
- Disclose one way per surface. The same class of explanation arrived
three ways in one scene — four info bubbles, five inline grey sub-captions
and a full banner — and inconsistent disclosure inside a single header row
capped
visual_polishon its own. Pick the affordance once. - Colour encodes one thing, and only against a stated threshold. The one
green number on a page was a raw count (
155) while the figures beside it were black; red on38%/82%was correct only because the page states a 85% target. Colour without a stated threshold is decoration that reads as judgement.
These are authoring rules, deliberately NOT a lint. A static checker over
render_code for "header taller than its rows" or "smallest type on the
page" cannot see computed layout, and this module has already retracted one
rule (ace#1660) and pruned another (ace#1841) for firing on specs that were
fine. The post-render judge measures these reliably — it caught all six — so
the cost worth removing is discovering them AFTER a render, not the detection
itself. Read this list while authoring, and the loop spends its iterations on
the story instead.
3d. Every coined label carries a definition a reader can reach FROM the label
(ace#2219). A dashboard is read on a projector by someone who has never
seen it. A column or row label the programme invented — 31-point band,
Surveys in the 31-point band, Mean likelihood below the line,
Payable / Non-payable — is unreadable to them unless the page says what
it means, at the place they meet it.
Enumerate two lists off the render_code you just uploaded, and write them
to dashboard-terms.yaml:
terms[]— every coined column and row label, with thesurfaceit renders on. Write the list even when it is empty;demo-data-setup-qacheck 15 cannot tell a plain-language dashboard from an un-enumerated one, and only one of those is a pass.definitions[]— what the page defines, and crucially whether a reader of the label can reach it from the label (at_point_of_use). An info control on the header, a tooltip, an adjacent plain-language lead line with the statistics demoted behind it — yes. A glossary panel elsewhere on the page — no.
Then run checkCoinedTerms(terms, definitions) from
lib/demo-frame-legibility.ts before minting the run.
That last distinction is the whole finding, and it cost an iteration. On
poverty-graduation/20260905-1345 the jargon hard cap ("jargon visible to
non-technical users, max 2") fired independently on six of seven scenes.
Iteration 2 added a "What each column means" panel defining all seven
review-table columns; the judge recorded it VERIFIED PRESENT and correct,
with PPI expanded, MAD expanded and the disavowal sentence in place — and the
cap fired on six of seven scenes again, because "the panel is BELOW THE
FOLD of every frame that uses the vocabulary":
Expanding an acronym is not the same as glossing it, and moving a definition on-screen is not the same as moving it to the point of use.
So the remedy is not another panel. Two moves, in this order:
- Rename the label so it needs no gloss, where that is available. "Surveys in the 31-point band" → "Small-household surveys (≤3 members)" deletes the term AND the disavowal sentence the glossary had to carry (the band has nothing to do with the cohort median of 31.4, which the panel had to say out loud).
- Otherwise put the gloss on the label — the reader OPENS it. Per the objective-data standing rule these are disclosure, not on-chart captions and never a takeaway line.
Two neighbours in the same class, both from the same judge pass, both fixable here and only here: a percentage column that silently switches denominator part-way down a table needs its blocks split with each denominator stated; and a chart series carrying no number reads as padding — scene 2's five-second read came out as "steady ~460 submissions a week" instead of the point of the scene because the grey segment was unlabelled.
- Build a URL per dashboard — the run deep-link, scoped by OWNERSHIP.
https://labs.connect.dimagi.com/labs/workflow/<def_id>/run/?run_id=<run_id>&<scope>where<scope>is the dashboard's OWNING scope:- program-owned cross-opp rollups (
program_admin_report,audit_par) →&program_id=<program_id>. A cross-opp rollup is a program-owned workflow (definition.program_idset, no owning opp);&opportunity_id=<any-opp>404s it with "definition N not found" from any other opp context. The realize-env realized map already emitspar_urlwith&program_id(connect-labs #946) — use it as-is. - opp-owned per-opp dashboards (
chc_nutrition_analysis,sam_followup,llo_weekly_review) →&opportunity_id=<opp_id>. Name each${key}_par_url; setprimary_par_urlto theprimary_dashboard's. Therun_idsource still depends onshape: - run-shaped → the saved run_id (env-ensure realized map or
workflow_save_snapshot). - action-shaped → mint one with
mcp__connect-labs__workflow_create_run(definition_id, opportunity_id, period_start, period_end)and use the returnedrun_id— with the same exclusive-period_endderivation as step 3 (timeline.end_date + 1 day). Omitting the bounds defaults BOTH to today, i.e. a zero-width window. Two traps, both verified to BOUNCE:/workflow/<def>/run/?<scope>with norun_id→ the workflow list;/workflow/<def>/?<scope>(no/run/) → the definition page. Only/run/?run_id=<id>&<scope>renders the dashboard (supersedes the earlierdocs/learnings/2026-06-13picker note; the fix is a real run_id, minted if needed).
- program-owned cross-opp rollups (
4b. Register the per-render reset for the interactive dashboard (ace#2297).
A dashboard that carries a state-mutating control is a dashboard the RENDER
itself dirties: the run keeps what the take created, so the next take finds
the button gone and its must_succeed click aborts. Restoring the
precondition is this skill's job — unconditionally, every render, not on
probe (CLAUDE.md § Phase preconditions are restored, not adapted). Skip this
step only when no dashboard is interactive: true; then write
render_reset: {required: false} rather than omitting the block, because a
recorded false is evidence and an absent key is not.
-
Read the keys, don't guess them.
mcp__connect-labs__workflow_get(<def_id>)returnssaved_runs.snapshot_inputs.state_keys— the workflow's own declaration of what state it mutates (["worker_states", "spawned_tasks"]forllo_weekly_review). A reset that names no key writes NOTHING: labs shallow-merges run state, so{"state": {}}is a silent success. -
Name the WORKFLOW, not just the run — the reset cannot find a page from a run id alone (ace#2325). The CSRF token has to be read off a rendered labs page, and labs routes the run page as
/labs/workflow/<workflow_id>/run/?run_id=<run_id>&opportunity_id=<opp_id>: the workflow id is the PATH segment and the run id is a QUERY parameter. So pass--workflow-id <def_id>(plus--opportunity-idwhen you have it), or--page-url <the dashboard's own par_url>— you already have both insource.dashboards[]. The CLI now refuses up front when neither is given; before ace#2325 it synthesised<base>/labs/workflow/run/<run_id>/, which resolves to nothing, so the POST came back 404 and was reported asrun-not-foundon a run that was perfectly resettable. Because the spec'ssetupblock isrerun: per_render, that blocked EVERY render. -
Run it once, now, and record
verified_at. Run the exact string you are about to register (the self-resolving form below) — not a dev-checkout dev-checkoutscripts/…invocation, because what is proven is the string the spec will carry. It needs the labs UI session (/ace:labs-login), notLABS_MCP_TOKEN. Exercising it here is what turns a registered command into a proven one; a409means you are pointing at a completed run. Retry once before believing a failure — measured 2026-09-09 on labs run 5590, one invocation in four returnedrun-not-foundwith a correct page URL and the identical command succeeded immediately after, so amust_succeedsetup can abort a render on a flake. -
Write the command in the SELF-RESOLVING form — an absolute path that is computed at run time, never a snapshot of this machine (ace#2351). Two facts pull in opposite directions and this form satisfies both. (1) canopy runs
setup.commandthrough a shell (subprocess.run(command, shell=True, cwd=…),runtime/scripts/walkthrough/record_video.py) with cwd = the git toplevel containing the SPEC, never the ACE checkout — so a repo-relativescripts/…resolves against the wrong root and the reset silently never runs. (2) The command is copied VERBATIM into per-run state and executed on every render for the life of the run, possibly from another account — so an absolute path into~/.claude/plugins/cache/ace/ace/<version>/pins a VERSION directory (the cache keeps every prior version, so the path keeps resolving to STALE code after every/ace:update) and a HOME directory (unusable from any other macOS account). Measured twice onspark-facilitator:20260909-2242pinned0.13.1413(pre-ace#2325 — nobuildRunPageUrl, no--workflow-id),20260910-0541pinned/Users/<name>/…/0.13.1426/…. The form:bash "$(python3 -c "import json,os;print(json.load(open(os.path.expanduser('~/.claude/plugins/installed_plugins.json')))['plugins']['ace@ace'][0]['installPath'])")/bin/ace-reset-labs-run" --run-id <interactive run> --workflow-id <def_id> --opportunity-id <opp> --keys <k1,k2>renderResetCommand()inlib/labs-run-state-reset.tsis the exact string. The shim (bin/ace-reset-labs-run) re-resolves the installed root itself — throughinstalled_plugins.json, neverdirname $0— and execsscripts/reset-labs-run-state.tsfrom there, so even a shim reached via an old path runs current code. The rule: never aplugins/cache/ace/ace/<version>/path and never a/Users/<name>/(or/home/<name>/) path inrender_reset.command.demo-data-setup-qacheck 18 failsrender_reset_command_pinnedon either, anddemo-narrativestep 3 refuses to copy one into the spec (ace#2351, the20260909-2242handoff is the repro). -
Write
source.render_reset(shape above).demo-narrativecopies thecommandverbatim into the spec'ssetupblock withrerun: per_render— neveronce, which skips the command the moment the outputs file exists, i.e. from the second render on. -
Full endpoint mechanics (nested body, DOM-only CSRF token, shallow merge, 409):
playbook/integrations/connect-labs.md § Resetting a live run's state between renders.
4c. Prove the interactive run's page actually LOADS, and record the probe (ace#2430). A run that cannot load is not a dashboard.
The in_progress mandate is not free: an interactive run has no
snapshot, so its page re-runs the live pipeline stream on EVERY load —
including every render take — while a completed run returns early on its
snapshot and never opens one (workflow-runner.tsx:
if (snapshotCarriesPipelines) return;). Until this step existed, nothing
in ACE ever exercised that dependency: step 3 proves the pipeline
EXTRACTS (pipeline_preview) and QA check 7 reads the data through
api/<def>/pipeline-data/ — a side-channel around the stream the page
itself uses. On poverty-graduation/20260915-1518 the stream died after
its first event, the interactive dashboard could not load at all, and
check 8 enforced that state: it fails a completed interactive run, so
the only QA-passing configuration was the one that could not render.
So, before the par_url is recorded, run the probe labs documents for
itself (WORKFLOW_REFERENCE.md § "Fastest way to tell these apart"):
SESS=$(python3 -c "import json,os;d=json.load(open(os.path.expanduser('~/.ace/labs-session.json')));print([c['value'] for c in d['cookies'] if c['name']=='sessionid' and c['domain']=='labs.connect.dimagi.com'][0])")
curl -sS -N --max-time 180 -b "sessionid=$SESS" \
"https://labs.connect.dimagi.com/labs/workflow/api/<def_id>/pipeline-data/stream/?opportunity_id=<opp>&refresh=1" \
-o <run-dir>/interactive-live-load.sse
refresh=1 forces the cold path past the 1-hour pipeline cache — a
cache-served pass is not evidence about what the render will do, the same
reason step 3 requires from_cache: false. Scope it the way step 4 scopes
the par_url: opportunity_id for an opp-owned dashboard, program_id
for a program-owned rollup. Then classify the transcript — do not eyeball
it, because a terminal event is not enough (a multi-opp failure lives at
metadata.per_opp[<opp>].error, never at the top level):
npx tsx -e "
import {readFileSync} from 'node:fs';
import {classifyLiveLoadStream} from './lib/interactive-live-load';
const r = classifyLiveLoadStream(readFileSync(process.argv[1],'utf8'));
console.log(r.verdict, '|', r.detail); process.exit(r.ok ? 0 : 1);
" <run-dir>/interactive-live-load.sse
Write the result to source.interactive_live_load
({dashboard, status, probe_url, verified_at, observed, run_id} —
evidenceFromProbe() builds it from the probe result, so the archived
observation is the transcript that was read rather than a second account
of it). demo-data-setup-qa check 8 REFUSES an in_progress interactive
run with no such record: the mandate now carries its proof.
painted→ ship itin_progress, exactly as § The interactive run stays live says.died/errored/empty→ do NOT ship a dead link. Re-run once (a stream can flake), and if it still fails: mint the same definition as a completed control run and load that. If the control renders, the live path is the problem, not the dashboard — complete the interactive run too, say on the page that the decision control is read-only for this recording, and record the escape withstatus: failed,control_run_id: <the control>andupstream_ref: <the labs issue>. The standing labs-side issue isdimagi-internal/connect-labs#1884; cite a newer one if the symptom differs. A blank flag exempts nothing (same discipline asbelow_programme_scale). If the CONTROL also fails to render, the dashboard is broken — that is a step-3 bindings defect, not an escape.- Tell
demo-narrativewhich it was. On the escape the payoff scene cannot perform a state change, so the narrative declares check 20's evidencedfinale_is_readrather than authoring a click that will degrade to ahold.
An expired labs session returns a login page here, not a stream —
classifyLiveLoadStream reads that as empty, so re-run /ace:labs-login
before concluding anything about labs.
-
Emit the handoff + write back.
Write
realized.json— the flat multi-var map ({ primary_par_url, <key>_par_url per dashboard, <name>_url drills }) — the summary.md, and thesourceblock above (includingdashboards[]+primary_dashboard) into the demorun_state.yamlviamcp__plugin_ace_ace-gdrive__update_yaml_file(merge: 'deep'— nevertwo-level, which would drop sibling sub-keys; see CLAUDE.md gotcha). Set thesteps.demo-data-setupblock tostatus: donewith therealized.jsonartifact path.Write
branch-scrub_report.yaml(step 2c) alongside it, and mirror its headline numbers intosource.dataset_constraints. Surface them in the summary.mdtoo — the off-branch count and anyunparsed[]gate belong in the run summary a reader actually opens, not only in a YAML the gate reads (ace#1658).
5b. Archive the manifest as the SAME BYTES you sent, then round-trip parse it
(ace#1737). The archived demo-data-setup_manifest.yaml is the run's
source-of-truth narrative artifact and the input a later fork replays. It is
only either of those if it is the manifest that actually ran and it loads.
-
Author once, to a local file, and send THAT FILE — never its contents through a tool call (ace#2433). Generate with:
ACE_ROOT="${CLAUDE_PLUGIN_ROOT:-$(python3 -c "import json,os; d=json.load(open(os.path.expanduser('~/.claude/plugins/installed_plugins.json'))); print(d['plugins']['ace@ace'][0]['installPath'])")}" node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/run-labs-synthetic-generate.ts" <manifest.yaml> --opportunity-id <labs-opp-id> --registry-gate-skip "<provider>: no semantic registry is authored"The script requires one registry-gate flag:
--registry-eval-verdict <file>on theace-runprovider (§ C2/C3), or--registry-gate-skip "<reason>"here, where no registry exists. Record the skip line it prints in the step summary.Then publish the same file as the artifact (
drive_create_filea placeholder, thendrive_update_file({localFilePath})— which uploads the bytes off disk). Do NOT type the manifest a second time for the archive.Why the script rather than the atom:
synthetic_generate_from_manifesttakesmanifest_yamlas an inline string and has no file-handle parameter, so calling it as an MCP tool forces you to re-emit the YAML into the tool call. The wire payload is then one emission and the archive another — identical only if you reproduce the file byte-for-byte, which is precisely the assumption that failed below. The script reads the file and POSTstools/callserver-side, so there is exactly ONE emission and the archived file IS the wire payload by construction rather than by diligence. Same_path-companion pattern asscripts/run-nova-media-upload.ts.--dry-run(appended to that same command) prints the exact request without sending it and needs no token — generation is a real mutation against a labs opportunity, so inspect it first if anything about the manifest is uncertain.The instruction this replaces ("send that file's contents to
synthetic_generate_from_manifest") was unsatisfiable: it named the file as the source of truth while the only available parameter took a string. -
Then read it back and parse it.
drive_read_file→yaml.safe_load, and re-validate against the labsManifestmodel if a connect-labs checkout is reachable. Fail the step if it does not parse.
Why: on hh-poverty-targeting/20260824-1404 the archived manifest did not
parse at all — the flow mappings were column-aligned, so the single longest
key (…ppi_q6_sachet_water) got zero spaces between its : and its {, and
YAML requires whitespace there. The generation itself had succeeded (2,237
visits), which is the tell: the string the atom received and the string that
reached Drive were two different emissions of "the same" YAML. Nothing failed
— the run was green, demo-data-setup-qa passed (it checks dashboards, not the
manifest), and the break only surfaced when the next run tried to fork it.
Drive is not the culprit and re-testing it is wasted effort: a round-trip probe
of the exact pathological shape (key:{ z: 3 }) through
drive_create_file → drive_read_file returns it byte-for-byte. The defect
enters at authoring, on the second emission.
Cheap and worth it beyond this class: the same round-trip check immediately
caught an unrelated real break in a step-2c report on
20260827-0323 — a leading > line that YAML reads as a root block scalar.
Then gate on demo-data-setup-qa before demo-narrative consumes the map.
Process (clone)
Clone reuses the denovo spine — only the data source changes (steps 1–2);
Step 0 (plan/template-select, now informed by the real opp's shape) and steps 3–5
(author dashboards, mint runs via workflow_create_run, build /run/?run_id= URLs,
emit the realized map) are IDENTICAL. Input: a real Connect --opp <id>.
1c. Profile the real opp → a PII-free manifest.
mcp__connect-labs__synthetic_profile_from_prod(opportunity_id, mirror=true) —
reads the opp's export endpoints server-side and returns a manifest reproducing
the real statistical shape. mirror=true carries a de-identified per-entity
transplant pool so per-case trajectories are reproduced (its own example is an
infant growth curve — i.e. exactly a child's MUAC recovery arc), not just column
means. Pass form_json_paths to pin the fields the chosen template reads (e.g.
the sam_followup MUAC paths) when auto-discovery misses them.
- Multi-LLO variant: for a program-admin clone spanning several sites, use
the cohort path —
synthetic_clone_profile(spec_yaml with allopportunity_ids,bundle_root: 'gdrive:') →synthetic_clone_generate— which registers each as a labs-only opp; then author dashboards over them. - Access note: profiling keys on the Connect
opportunity_id(delivery data is aggregated in Connect, independent of which HQ the deliver app lives on). If the opp isn't reachable with the labs caller's token, that access gap is the thing to close (multi-cluster / membership) — fail loud, don't fake it.
2c. Generate into a fresh labs-only opp — synthetic_create_labs_only then
synthetic_generate_from_manifest with the profiled manifest (identical to
denovo step 2, from-manifest mode). Capture labs_opp_id + deliver_units.
2c-fidelity. Gate on demo-fidelity-check (clone-only) — wraps
mcp__connect-labs__synthetic_fidelity_report(bundle_dir) to confirm the clone
reproduces the source's per-field distributions + correlations before it reaches
a funder. Fail → regenerate (or fall back to denovo authoring) — never show a
low-fidelity clone as if it were the real program.
2d. Copying REAL values into a clone is a PII risk — the operator's call, never
yours, and never "the fields look harmless". A clone deliberately carries no
real value (names, villages and GPS come back empty), and a demo often wants them
back. Do not overlay real values unless ALL of these hold, and say which ones you
checked in the summary:
- The operator asked for it, for THIS workflow, in their own words. Their
knowing the data is safe is the authorisation. Never infer it from field names:
a borehole's name is harmless, a household's GPS is identifying, and both are
"a name and a location".
- The values are read with the operator's own raw-visit access. That means the
source's /export/opportunity/<id>/user_visits/ rows, reached via their token.
ACE's own labs identity usually cannot read the source, and that refusal is the
gate working. Do not route around it.
- The copy is no more visible than the source. Narrow the clone's
allowed_domains (synthetic_set_allowed_domains) to the people who can already
read the source, unless the operator says otherwise.
- The surface says so. Tell recipients the values are real, in the email and in
the scripts doc. The report's own banner still reads "Built on synthetic data",
which is now wrong: connect-labs#2150.
The supported path is connect-labs#2150 (a verbatim copy gated on raw-visit access,
audited as a bulk-PHI export). Until it ships, any overlay is a hand-built exception.
Name it as one. (Origin: Jon, 2026-10-02, after ACE overlaid real waterpoint
names and GPS onto opps 10092/10093: *"this should still be considered a PII risk,
I just happen to know its okay on this workflow, so this should only be allowed
full fill user-visit permissions."*)
Then run Step 0 (template selection, informed by the real opp) and steps 3–5 exactly
as denovo. A clone whose real data has no live activity yet → there's nothing to
profile; use denovo for that program until it has real delivery data.
Process (ace-run)
The Phase 7 convergence provider — the data source is the current /ace:run's
Phase-4 Connect opportunity and its PDD/app structure, not a brief or a real
external opp. Same spine as denovo; the manifest's provenance changes, and step
1c is a REQUIRED authoring step this provider adds — a denovo brief states its
own headline anomaly, a PDD does not hand you one.
1a. Resolve the run's opp + app structure. Read the run's
phases.connect-setup.products.connect.opportunity (connect_int_id, apps) and
the PDD (inputs/pdd.md) / deploy summaries — the KPI fields, deliver units,
and personas come from the built apps, not invented.
ace-run builds the semantic-layer cascade (default — ace#2510)
Operator decision (Jon, 2026-09-26): every run produces a programme cascade
story, not a bespoke dashboard — "the synthetic data is going to prove to be a
key part of people wrapping their head around the system." After 1a the ace-run
provider runs C0–C7 below; steps 1b–2a and the Step 0 template spine that
follow are the legacy single-opportunity path, used only when the operator asks
for it (--legacy-dashboards) or the archetype has no followed entity worth a
cascade.
What the viewer gets, all live in labs and all following ACE's Labs template workflows while the run is live (§ C7; frozen off them when Phase 7 ends):
| level | surface | who it is for |
|---|---|---|
| programme → partner → opportunity → worker | indicator_programme_report (program-owned, multi-opp, saved weekly runs) | the programme manager |
| worker → case → the case's visits | its companion indicator_worker_review (created with it) | the PM drilling in |
| one partner's opportunity + anonymous benchmark | indicator_opp_report, one per partner, receiving the programme's saved runs by hand-down | that partner's network manager |
Worked, live example (all ids real, spark-facilitator/20260926-1800):
docs/examples/cascade/spark-facilitator/ (pool, pipeline, registry scripts);
programme report 6371, worker review 6373, opp reports 6376/6378/6380, program
10082, registry 6369, cohort 5.
C0. Size the story. Default 5 partners × ~12 workers × ~13 weeks — and 5
is also the floor for invented partners (DEFAULT_FLOORS, ace#2727): three
partners, all at 100% on Spark's Step 7, gave the concept judge nothing to show
why sorting partners by need matters. For a
programme whose PDD fixes one entity per worker (Spark: one community per CBF),
a worker is a row in the partner table and a case is that worker's entity. The
real partner org of the run (connect_orgs.nm_org, e.g. ace-nm-org) is never
touched — partners exist only as llo_map labels over labs-only synthetic
opportunities. Label them neutrally unless the PDD names
candidate LLOs, and even then prefer neutral labels: a synthetic figure next to a
real organisation's name is a claim about that organisation.
Invented partners say they are illustrative — on a REALISTIC name. The
multi-partner programme view is a deliberate demo device — a partner comparison
and an anonymous benchmark need peers — not a claim about how many organisations
deliver the pilot. Read the number of implementing organisations from the PDD
(default 1: a single-LLO pilot) and record it as the plan's
implementing_orgs. When the partners are invented (partner_source unset or
invented), each label is an invented, culturally plausible organisation name
for the PDD's geography with a trailing (example) marker, e.g. for a Malawi
pilot Tiyende Community Trust (example), Kuunika Outreach Network (example),
Tsogolo Women's Forum (example), Umodzi Development Group (example),
Mphamvu Health Alliance (example). Name the synthetic programme so it says the
same (program_name: "<opp title> (illustrative: five example partners)", the
plan's programme_name). Never a real organisation's name, never a name from
the opp's inputs.
Why both halves, and why not Example partner A (ace#2727). Bare Partner A/B/C read to an outsider as three implementers of a one-implementer pilot
(spark-facilitator/20261001-2208, be59309d). The be59309d fix required
Example partner A/B/C — and canopy's DDD user-artifact judge, which reads the
screens AS the programme manager, scored those placeholder labels at clarity 2:
spark-facilitator/20261004-1706 stalled at 2.0 across 15 judged passes in
three DDD runs on exactly that. Two rails read the same label and neither knew
the other; a realistic name with an (example) marker satisfies both — a viewer
reads an organisation and is still told it is invented. With partner_source: programme the partners mirror the PDD's real implementers, so their count must
equal implementing_orgs. Enforced: checkCascadeStoryPlan
(lib/cascade-story.ts ILLUSTRATIVE_LABEL + exampleOrgLabelProblem, which
refuses both Partner A and Example partner A), at the C3 gate and in
demo-data-setup-qa check 21.
Name the people and places too — a display roster (ace#2727). The same judge
scored worker codes (cbf_a07) and hashed community ids at clarity 2. So the
plan carries a roster a viewer can read:
workers: [{username, display_name, partner}]— one per worker (the sum ofpartners[].workers);usernamestays the generator's identity, the key the snapshot rows carry;display_nameis an invented, culturally plausible human name for the PDD's geography (Chikondi Banda,Tadala Phiri), distinct across the roster.entities: [{id, name, worker}]— one per followed entity (Spark: a community);nameis an invented human place/group name (Kalemba), never the hashed id.
Never a real person's name from the inputs — the PDD's contacts, the LLO's
contacts, reviewers. List those names one per line and pass them as
checkCascadeStoryPlan(..., { realPeople }) / the QA script's --real-people; a
display name containing one fails. Codes, digits, _, and role-word placeholders
(Worker A, Community 3) fail too (displayNameProblem).
Headline indicators must differ across partners (ace#2727). Declare
headline_indicators (what scene 1's tiles and the partner table show) and, for
each, headline_spread: {<indicator>: {<partner label>: <value the pool is written to produce>}}. A headline flat across ALL partners (best-to-worst
< 5 pts) fails the plan: it shows the programme manager nothing to sort
partners by. A saturated indicator is not wrong, it is just not a headline —
demote it. C6 re-checks the spread against what labs graded.
C1. Create the synthetic programme. One labs-only opportunity per partner:
the first synthetic_create_labs_only({label, gdrive_folder_id: 'pending-generate', org_name, program_name, notes, allowed_domains: ['@dimagi.com', '@dimagi-associate.com', '@dimagi-ai.com']})
leaves program_id unset, which makes its own id the program id; every later
partner passes program_id: <that id>. @dimagi-associate.com is Dimagi (owner, 2026-10-16:
associate designers review these dashboards). Include ACE's own domain in
allowed_domains or the ACE identity cannot read what it just created. Write each
{label, opportunity_id} to products.synthetic.cascade.partners and
program_id immediately (ace#2412). The label is the opportunity's NAME and
is filmed (page title, scope picker), so it carries no sweep handle — the handle
goes in notes; org_name keeps it (§ C7 "Sweep handles", ace#2733).
C2. Registry. Invoke semantic-registry-author with the partners and
program id; it validates, runs semantic-registry-author-qa, and creates the
record. Then run semantic-registry-author-eval — BEFORE C3. Do not generate
data against a registry that has not passed QA AND been graded — the story
signals, manifests and report trio are authored against its indicator ids, so a
finding after generation rebuilds the cascade (all three 2026-10-15 runs graded
it last). Enforced: C3 generates through scripts/run-labs-synthetic-generate.ts --registry-eval-verdict <local copy of 7-synthetic/semantic-registry-author-eval_verdict.yaml>,
which exits 3 when the verdict is missing, ungraded or fail
(lib/registry-eval-gate.ts).
C3. Author the story, then the data. Write 7-synthetic/cascade-story.yaml
(lib/cascade-story.ts CascadeStoryPlan): partners (realistic (example)
names, § C0), programme_name, weeks, the display roster (workers[],
entities[], § C0), headline_indicators + headline_spread, and the four
signals — lagging_partner, standout_worker, data_quality, trend —
each naming a registry indicator, the PDD clause it derives from (pdd_ref, same
rule as step 1c: an uncited signal is an invented one) and what a viewer sees.
Gate it with checkCascadeStoryPlan(plan, registryIndicatorIds, undefined, { realPeople })
BEFORE generating.
Set implementing_orgs from the PDD (§ C0). When the partners mirror a real programme's own (not invented example partners), set
partner_source: programme, which makes 2 partners legal. With no PDD, set anchor: app
and cite the Deliver app's form and rule in each pdd_ref (semantic-registry-author
§ 2, "No PDD").
The data-quality signal is step 1c's obligation carried over: pick it from the
PDD's own review flags / verification controls, never a fraud pattern the design
does not describe.
Author what the build must look like, not only what it must show (ace#2735).
Three more plan fields, each because the DDD judges found its absence after a
render on spark-facilitator/20261004-1706 (89 findings, several of them build
output):
review_routing: [{flag, pdd_ref, expect | report_only}]— for every flag the registry declares (display.visit_flags[].column), what the PDD does with a flagged record:expectthe fields a flagged record carries (a desk-review flag →{flagged: [true]}), orreport_onlywith the PDD's own words. Then WRITE those fields onto the flagged visits in the pool — the generator's default isapprovedwith nothing sent to review, which is how 11 of 12 repeat-count-flagged records were filmed contradicting PDD §7.2 S-1. The mechanism: on each flagged pool visit setreview: {flagged: true, status, review_status, flag_reason}(connect-labs #2264 — the transplant pool's authored review outcome; before it, status/flagged were always drawn from the persona'sflag_rate), and set every persona'sflag_rate: 0so nothing that raised no PDD flag is sent to review (ace#2740). Required with adata_qualitysignal (checkCascadeStoryPlan).- Worker rates need a denominator.
checkPlannedWorkerRates(plan, entityCountRateIndicators(registry))(lib/cascade-build-qa.ts) fails when most workers own ≤ 1 entity while a worker-level rate counts entities — the rate can then only read 0% or 100% (Spark's Step 7 on time, judged "false precision"). Fix the roster, or set the indicator'sflw_applicable: falseso it shows at partner level only. single_child_levels: [{level, reason}]— a drill level the PDD fixes at one child (Spark: one community per facilitator), cited. Any other level with one child everywhere fails at § C6.
Then generate one manifest per partner with synthetic_generate_from_manifest —
sent through scripts/run-labs-synthetic-generate.ts <manifest> --opportunity-id <partner opp> --registry-eval-verdict <local verdict copy> --expect-generated (the registry gate, § C2) —
using mirror mode with an AUTHORED transplant pool
(beneficiary_cohorts[0].longitudinal: {mode: mirror, numbers: authored, jitter_frac: 0, transplant_pool}): one series per followed entity with owner = its worker, and
per visit {day, form, values, cats, dates} keyed on the released Deliver app's
exact form paths. Synthetic slot mode cannot express this — it draws a random
entity per visit, so one case would be visited by every worker. This is still
curation, not mirroring of real data ("curate invents signal"): the pool is
written, not profiled.
numbers: authored is required on every cascade pool (connect-labs #2718).
Labs treats any mode: mirror pool as a near-copy of real cases
(provenance.replays_real_cases), so without the mark it never marks the
partner opps as generated, and canopy's agent panel and data_query refuse
agents on them (every cascade opp built before the mark — Kaduna, Bednet,
Newborn care, Spark, Smallholder, Apedor — has generated_folder_id = NULL).
numbers: authored is ACE's word that it wrote every series itself, from no
real case; labs takes it, so set it ONLY on a pool ACE wrote. A clone's
profiled pool (§ Process (clone), synthetic_profile_from_prod(mirror=true))
replays real cases and must never carry it. Labs' answer is the authority:
synthetic_generate_from_manifest returns generated, and on a cascade it must
be true. --expect-generated enforces both halves: it refuses (exit 4) before
sending a manifest with an unmarked mirror/modelled pool
(lib/labs-generated-provenance.ts), and exits 4 when labs answers
generated: false, printing labs' not_generated_because. On that exit stop
and report it with labs' reason — do not build the trio on data labs does not
count as generated.
Mechanics that bite (all observed, ace#2510):
- give every worker its own per-worker effect on every per-visit draw, not
only partner-level ranges (ace#2746). Over ~30 visits a worker, values drawn
per visit from one partner range all converge on the partner mean, so ten
facilitators read 35.8–36.9% while regularity spans 55–98%, and the DDD user
judge reads that as generated data (
trust2 onspark-facilitator/20261004-1706). Draw a worker offset once, for example Gaussian with a few points' SD, and add it to that worker's draws. Check 26 (checkConstantDisplayedColumns) fails a scorecard rate that is under 3 points across every partner's workers beside a column spread ≥ 15 points; jitter_frac: 0— jitter turns people-counts fractional and breaks a deliberately repeated count;- a categorical constant across an entity's visits is stamped onto ALL its
visits (
_series_constants) — omit branch-only answers that never vary; - a labs-only opp has no form schema, so a numeric-looking code comes back
"1.0"— the registry reads it with a cast (semantic-registry-authorstep 1); kpi_configneeds one entry even though the cascade does not read it;- the labs MCP endpoint refuses a large request body before any tool code
runs: HTTP 413 on a 3.9 MB manifest, while 2.44 MB was accepted
(
smallholder-seasonal/20261015-1402, ace#3007). A programme-length pool must be slimmed. Write a per-series constant once (_series_constantsre-stamps it on every visit of the series), but ONLY for a path whose absence reads the same as its value: a path the cohort shows to be time-varying is not re-stamped (_time_varying_paths) and reads NULL on the other visits. That run lost its harvest units this way and under-counted kg until those paths were exempted. Every registry and dashboard reader should filter per-form values by form name. Send a multi-MB manifest over a direct JSON-RPC call to the endpoint, not as an inline tool argument through the model.
Put the display roster on screen (connect-labs #2251, deployed 2026-10-06).
The reports show a worker's display name and a case's name only when the
synthetic data carries them — otherwise they show the username and case id,
which is how cbf_a07 and hashed community ids reached the screen (ace#2727).
- Workers: in each partner's manifest, one
flw_personas[]entry per roster worker:{id: <workers[].username>, display_name: <workers[].display_name>}. The personaidstays the username (it is the identity the reports key on);display_nameis what the worker tables show (byFLW[].name). - Entities:
beneficiary_cohorts[0].entity_names: [<entities[].name>, ...], in the SAME order as the cohort's entity series (transplant_pool); the Nth entity gets the Nth name and lands in each visit'sentity_name. The cohort must belongitudinal(it is, above) — otherwise every visit is a new case and a name does not follow its case. - Registry: set
display.entity.label_field: entity_namein the indicators document (semantic-registry-author§ 3) so the case table and the worker review's case heading show the name, not the id. - Verify, don't assume: after generating, read one partner's visit rows
(
pipeline_preview) and confirm every entity id carries the name the plan gave it and every worker's rows sit under that worker's display name; a mismatch is a generation defect to fix before § C4, not a narration to write around. Saved runs keep the names they were BUILT with — build them after this step, never before. Narration names a worker or community by its plan name only where the page shows it (demo-narrativestep 3b's on-screen rule).
C4. Instantiate the trio.
workflow_create_from_template({template_key: 'indicator_programme_report', program_id, opportunity_ids: [all partners], registry_source: {registry_id}, name: '<Programme> programme report (synthetic)'})— one call also creates the companion worker review and its run (companions.worker_review). The name is the page breadcrumb, so it carries NO sweep handle; put the handle in the workflow'sdescriptionstraight after create (workflow_update_definition({ patch: {description: '(synthetic, ace <opp>/<run-id>)'}})) — § C7 "Sweep handles", ace#2733. Bind WITHOUT a home-scope key (seesemantic-registry-author§ 4 binding gotcha). Then bind the companion worker review too. Labs creates it UNBOUND:workflow_geton it readsregistry: {source: "disk", name: "kmc", note: "Unbound: computes from the on-disk registry..."}, so a worker's drill-down grades against kangaroo-care definitions on any other programme (observed live on connect-fi/20261018-1440, worker review 11442, read 2026-10-21).workflow_update_definition({workflow_id: <companion>, program_id, patch: {registry_source: {registry_id, program_id}}, expected_version}), then re-read withworkflow_getand requireregistry.source: "record"with your registry id.- The template's
visitspipeline carries only Connect's base columns. Replace its schema with the registry's Layer-1 fields (pipeline_update_schema,terminal_stage: visit_level, one{name, path, aggregation: first, transform?}per column; ids fromworkflow_get→pipeline_sources). Preview first (pipeline_previewwithopportunity_ids) and readfields_all_null. When an existing report already carries the right pipeline, passpipelines_from: {workflow_id, program_id}at create instead. - Grade one date before writing history:
workflow_preview_as_of({definition_id, program_id, as_of: <last period end>, include_opportunities: true})— every partner present, no indicator null that should not be.
C5. Saved weekly history + hand-down. Create the partner opp reports FIRST
so hand-down has somewhere to go. Read the synthetic org's slug from
labs_context({search: '<run-id>'}), never derive it — take the slug of the
organization whose name is the org_name you passed to
synthetic_create_labs_only (the sweep-handle name carries the run id, so the
search finds it; synthetic_create_labs_only does not return the slug). Labs mints
it as labs-synthetic- + a char-by-char slug (every non-alphanumeric → -, runs
NOT collapsed; synthetic_org_slug, connect_labs/labs/synthetic/org_tree.py), so
a conventional slugify of the name misses: … partners (ace spark-facilitator/20260926-1413)
is …-partners--ace-spark-facilitator-20260926-1413, and the guessed
…-partners-ace-spark-facilitator20260926-1413 fails with "organization … is not
accessible to your account" (ace#2522, spark-facilitator/20260926-1413). Then
benchmarks_cohort_create({organization_id: <that slug>, min_peers: 2, min_denominator: 1, require_complete_series: false}) →
benchmarks_cohort_add_opportunities →
benchmarks_create_opp_reports({cohort_id, template_key: 'indicator_opp_report', source_workflow_id: <programme report>, source_program_id}) (response shared: true confirms they reference the programme's pipelines and registry). Then
workflow_rebuild_history({definition_id, program_id, cadence: 'weekly', start: <first Monday>, end: <last Sunday>, limit: 13}) — one completed run per week,
handed down to every opp report (hand_down_queued: true). Publish the benchmark
for each saved run in chronological order (benchmarks_publish({cohort_id, workflow_id, run_id, program_id})) — the opp report's Benchmarks tab shows the
most recently PUBLISHED as-of, not the latest period. Plan that list with
planBenchmarkRepublish (lib/benchmark-republish.ts; it refuses a missing
week) and confirm the Benchmarks tab's as-of equals the last period; the same
re-publish is REQUIRED after any registry edit + rebuild
(semantic-registry-author § After a registry edit, ace#2717). If the
operator's permission classifier refuses benchmarks_publish, surface it to the
operator — never work around it. min_peers: 2 is the floor
(with five partners a partner has four peers, so it never binds); record it in
the summary.
The latest saved run is the programme report's run_id; each opp report's latest
handed-down run (workflow_history_runs(..., generated_only: false)) is its
run_id. Re-running C5 on a history that already exists (a registry edit,
a regenerated pool) replaces every period's run with a new id — record the
listings before and after and relink every artifact per
semantic-registry-author § After a registry edit (scripts/relink-rebuilt-history.ts,
ace#2700) — for run_state.yaml that is --run-state over the WHOLE Phase 7
products block, source.dashboards[].par_url included, never a hand list of
fields (ace#2739); otherwise realized.json, the dashboard links and every
summary link point at runs that no longer exist.
C6. Verify the story LANDED — against the saved runs, not the manifest.
workflow_history_runs({definition_id, program_id, generated_only: false, include_snapshot: true}) → map each run's data.snapshot.state.snapshot
{byLLO, byFLW, programInd} to GradedPeriod and call
verifyCascadeStoryLanded(plan, periods). Every signal must land; a signal that
did not is fixed in the pool and regenerated (the ids stay; re-run C5), never
re-described to match what came out. demo-data-setup-qa re-runs this check.
Then judge the BUILD OUTPUT the judges would otherwise judge after a render
(ace#2735): cascadeBuildOutcomes(plan, latestSnapshotFromHistoryRuns(<same response>), <visit rows>) — the visit rows from pipeline_preview of the
cascade visits pipeline under its HOME opp, sample_size: 200. That is a sample,
not every partner (ace#2973): the cap is 200, and the default cascade's visits
pipeline is home-scoped, so it does not resolve under another partner's opp and
an opportunity_ids fan-out samples the home rows first. Pass the scope with
visitSampleScope(<responses>); checks 1 and 3 then say what they judged. To judge
EVERY partner instead, build the full rows from each partner's generator fixture
(user_visits.json) mapped through the visits pipeline's schema paths, and
cross-check the mapping against the 200-row home preview (0 field mismatches)
before trusting it — demo-data-setup-qa step 3 has the recipe (ace#2742). Four
checks, each failing the build:
- review routing — every record raising a declared flag carries its
review_routingroute (checkReviewRouting); a raised flag with no route fails; - worker rates not binary — no worker-table rate (
display.indicators[].scorecard) has a graded denominator (cell.n) ≤ 1 for most workers (checkWorkerRateDenominators); - displayed columns vary — no worker-table scorecard column, case field or
visit field/flag is the same on every row (the entity-count column is not
judged: labs moves a uniform count into the table title, "1 facility each",
so a PDD fixing one entity per worker never draws it — ace#2972)
(
checkConstantDisplayedColumns; a column constant only inside one partner's table is reported — keep the narrative off it); - drill levels branch — no level has exactly one child under every parent
unless
single_child_levelscites why (checkDrillLevels). A failure is fixed HERE, in the pool, roster or registry display, and re-verified before the hand-off. The DDD loop can fix these too, because registry, pipelines and data are Labs data it writes (Phase 7's dispatch block). But a defect this check can see costs a render plus a judge round when the loop finds it instead, and it is filmed in the meantime. Nothing here needs a connect-labs deploy unless it is a genuine Labs product bug or missing feature.
C7. Handoff. realized.json (flat) carries:
{
"primary_par_url": "<programme report run URL>",
"programme_par_url": "https://labs.connect.dimagi.com/labs/workflow/<prog>/run/?run_id=<latest>&program_id=<program>",
"worker_review_url": "https://labs.connect.dimagi.com/labs/workflow/<review>/run/?run_id=<review run>&program_id=<program>&flw=<opp>%3A%3A<username>&source_run=<latest>",
"lagging_partner_label": "Tsogolo Women's Forum (example)",
"standout_worker": "cbf_b03",
"standout_worker_name": "Tadala Phiri",
"data_quality_worker": "cbf_a07",
"data_quality_worker_name": "Chikondi Banda",
"tsogolo_womens_forum_opp_report_par_url": "https://labs.connect.dimagi.com/labs/workflow/<opp report>/run/?run_id=<handed-down run>&opportunity_id=<opp>"
}
*_worker stays the username (URLs and flw= keys use it); *_worker_name is
the roster's display_name — the name narration uses, and only where the screen
renders it (§ C3 TODO).
one <partner>_opp_report_par_url per partner, and a worker_review_url for each
worker carrier. Every *worker_review_url carries &program_id=, never
owning_program_id alone — the programme report's own "Review →" links use
owning_program_id, but that is only a labs data-access hint, not a page-scope
param (labs.context.CONTEXT_PARAMS is organization_id/program_id/
opportunity_id). A scene that opens the review cold gets the session's last
opportunity_id appended and renders "Workflow definition not found under
opportunity " (ace#2521, spark-facilitator/20260926-1413). demo-data-setup-qa
check 2 fails it (checkWorkerReviewUrlScope).
Never hand-assemble a worker-review URL, and never copy one off a "Review →"
link. Prose alone did not hold: spark-facilitator/20260926-1800 wrote all three
*worker_review_urls with owning_program_id after this paragraph already said
not to. Generate them with the one builder (lib/worker-review-url.ts), AFTER
the cascade block below is in run_state, from a carriers file naming each worker
([{"keyPrefix": "", "opportunityId": <opp>, "username": "<user>"}, {"keyPrefix": "standout_worker", …}]), and merge its output into realized.json:
node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/worker-review-url.ts" \
--run-state <local run_state.yaml> --carriers <carriers.json>
A realized.json already written with owning_program_id is repaired in place,
not re-derived by hand:
node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/worker-review-url.ts" --fix <local realized.json>
Write products.synthetic.cascade (registry, program_id,
partners, programme_report, worker_review, opp_reports[], cohort_id,
history, story_verified) AND mirror the reports into products.synthetic.workflows
({programme_report: {workflow_id, run_url}, <partner>_opp_report: {...}}) — that
map is what ace-web's run summary renders. labs_opp_id = the first partner's
opportunity.
Then write the two required build-half files (ace#3251 — neither existed on
the cascade path, so connect-fi/20261018-1440 reached Phase 7 finalize with
verify_phase_artifacts at 7/10 required and could not reach status: done):
7-synthetic/demo-data-setup_manifest.yaml— on the cascade this is an INDEX, not a generator manifest: one entry per partner naming its opportunity and where the bytes sent for it are archived,7-synthetic/cascade-build/manifest_<partner-slug>.yaml(partnerManifestPath; record a different archive path ascascade.partners[].manifestand the index uses it). Archive each partner's sent manifest there in § C3, per step 5b (the same bytes, round-trip parsed).7-synthetic/dashboard-terms.yaml— step 3d from the registry: every indicator label is a column header on each report, defined at point of use (the definitions popup) only when the indicator carries ameta.plainsentence; visit-flag labels likewise via theirdescription.
Generate both, never hand-write them:
node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/cascade-build-artifacts.ts" \
--run-state <local run_state.yaml> --registry <local semantic-registry-author_registry.json> \
--out-dir <local dir>
and upload both into 7-synthetic/ (find-or-update). demo-data-setup-qa
check 29 (cascade_build_artifacts_written) fails the build when either is
absent or the index misses a partner, so the gap is caught here, not at finalize.
Then point every report at ACE's template workflows (2026-10-16). The trio is
created following the DEPLOYED labs code template, which only a connect-labs
release can change — so the DDD loop could never fix a page defect (ace#2337,
connect-labs#2578). ACE keeps one Labs template workflow per report template
(config/labs-report-templates.yaml: page render as data, draft → preview →
publish, versions + rollback). Following it makes a page fix one publish, which
the DDD loop does in-loop (agents/synthetic-data-and-workflows.md § Step 3):
node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/labs-report-templates.ts" follow \
--run-state <local run_state.yaml> --out <patch.json>
Hand patch.json to update_yaml_file({merge: 'deep', localFilePath}) — it
records products.synthetic.cascade.report_templates (state: following, each
report's template and the version it started on). The report's own config keys
(companion links, the coach action) stay its own — labs keeps them as overrides —
and the page renders the same until a template is published. A skipped report
names why; state: partial is reported in the phase summary, never ignored.
Never write render code onto a cascade report: a follower refuses it, and a fork
would take that page off the template the loop fixes. Phase 7 freezes the reports
off the template when it ends (§ Step 4 there). ensure (same script) checks the
templates are live, public and editable when a follow fails.
Sweep handles. Registries have no delete atom and synthetic opps are only
disabled (synthetic_disable), so the ids in products.synthetic.cascade are the
only way to find these objects again. Every labs object carries the handle
(synthetic, ace <opp>/<run-id>) — but only in a field no frame renders
(ace#2733):
| Object | Rendered? | Where the handle goes |
|---|---|---|
| programme report workflow | yes — its name is the breadcrumb on every frame | description |
opp reports (benchmarks_create_opp_reports) | yes — page title | description |
synthetic opportunity (synthetic_create_labs_only label) | yes — page title and scope picker | notes |
| registry | no | name (semantic-registry-author § 4) |
| benchmark cohort | no | name |
synthetic org (org_name) | no | name — § C5 finds the org's slug by searching the run id |
Why: the DDD user-artifact judge scored the run id in the programme report's
title as a trust defect (spark-facilitator/20261004-1706, smallholder-seasonal/20261012-1549),
and on noora-health/20261015-1129 the opp slug put a real organisation's name on
a page framed as "illustrative — not Noora Health's programme". /ace:sweep's
labs leg (sweep-labs) matches the handle in name, description OR notes.
Known residuals (labs, 2026-09-26). An opportunity renders as "Opportunity " where a label map would help; the drilled partner header reads the programme's case count; the case table shows the entity id (a followup form carries no clean entity name) and worker rows show the username. The last two are what the display roster (§ C0) closes once the labs half lands (§ C3 TODO, ace#2727). None blocks the drill.
Legacy single-opportunity path (1b–2a, then the Step 0 spine)
1b. Author the manifest from the PDD/apps (the story-coherent per-opp manifest
that today's synthetic-narrative-plan + synthetic-data-generate produce) —
keyed on the real deliver-app form paths so the dashboards read real fields.
1c. REQUIRED — give the demo something to detect, derived from the PDD's own declared evidence model.
A demo of detection needs something to detect. A manifest that ships
anomalies: [] and one undifferentiated accuracy_distribution across every
persona produces a cohort with no signal in it, and then every dashboard is
honest and every scene is inert. That is not a rendering problem the DDD loop
can fix downstream — it is an authoring omission, and the loop's
use_case_soundness finding is "the scenario does not exercise the feature it
demonstrates." Measured on hh-poverty-targeting/20260824-1404: the ace-run
manifest carried anomalies: [] and no per-persona overrides; completion rates
clustered 64–77%, mean PPI scores clustered ~34, the review queue ranked 15
undifferentiated workers, and the payoff decision landed against an empty
adjudication log. Phase 7 terminated stopped_not_converged at concept 2.0/5.
Four obligations. All four are REQUIRED; which control you pick is yours.
-
Enumerate the PDD's declared controls, verbatim. Read the PDD's verification / fraud / data-quality section and list every automated flag, threshold, and review rule it names, quoting each. This list is the ONLY menu you may author from.
-
Instantiate at least one of them in the manifest as an
anomalies[]entry and/or a per-persona divergence (field_distributions/period_rates/accuracy_distributionthat separates the carrier from its peers). One well-formed signal a reviewer can actually find beats three vague ones. -
Cite the clause each signal derives from — a
# derived from PDD §N: "<quote>"comment on the manifest entry, echoed in the run summary. An uncited anomaly is an invented one. -
Make the carrier distinguishable. State, before generating, what a reviewer would SEE that separates the carrier from the cohort (which column, which dashboard, roughly what magnitude). If the answer is "it would look like everyone else", the signal is not authored yet. A cohort whose metrics all cluster inside one band ranks fifteen workers nobody can choose between — which is what the measured run shipped.
-
Size the cohort so finding the carrier is not trivial —
rows >= DETECTION_MIN_ROWS(24). A detection demo claims unaided scanning is not viable: that the control finds a worker a supervisor would otherwise miss. That claim is false the moment the whole cohort fits in one look, and a judge will say so.demo-data-setup-qacheck 13 fails the run here, beforedemo-narrativeconsumes the map, by comparing thedetectable_signalyou just declared against thedata_shape.rowsyou just realized.When the PROGRAMME itself is smaller than the floor, raising the cohort would misrepresent it, and stripping the signal would make the dashboards less honest — so do neither. Keep
detectable_signal, setdetectable_signal.below_programme_scaleto the PDD quote that proves the roster (the same cite-your-source rule obligation 3 imposes), and build the demo's payoff on something this scale DOES support. bednet-check-2-visit is exactly this case: its PDD specifies "3-5 front-line workers", so its dashboards should carry the consent-rate signal for honesty while the narrative lands on, say, the follow-up completion rate against the PDD's own >=70% target — a claim that holds at any cohort size. Check 13 accepts the escape only with a non-empty quote; an unevidenced flag is a silencer.Taking the escape OBLIGES you to name where the payoff went, in
source.payoff_control— check 19 fails the run if you don't (ace#2316). "Build the payoff on something this scale DOES support" was unenforced prose for as long as it existed, and it was broken the first time it was relied on.spark-facilitator/20260908-2215took the escape with two correct PDD quotes (~12 facilitators, one per community) and wrote the right mitigation verbatim — "the narrative's payoff is built on the qualifying-share rule, which holds at any cohort size, rather than on a claim that unaided scanning is impossible" — and then the authored payoff clickedtestid:below-target-filterto hide 10 of 12 rows, which IS that claim. Nothing compared the promise to the scene. Declare the control, andcheckEscapedPayoffIsHonoured(lib/ddd-scene-actions.ts) checks that a scene actually acts on it. Re-pointing the declaration at the template's filter to clear the check re-creates the defect with a green verdict — the identity match cannot detect that, so it is on you.No narration fixes an under-sized cohort — the remedy is always the manifest.
bednet-check-2-visit/20260902-1555authored a well-formed, PDD-derived consent-rate outlier (100.0% against a cohort's 85.2%, a clean +14.8pp separation) over five workers, and Phase 7 still endedstopped_not_convergedat concept 2.0/5 — all five per-scene judges independently said five rows is eyeballable. The signal was right and the cohort was too small for it to mean anything (ace#2131).
Never invent a fraud pattern the design does not describe. A demo that
fabricates a detection story is worse than a boring one: it shows a funder a
capability the program did not specify and cannot operate. If the PDD genuinely
declares no verification controls, record
detectable_signal: none (PDD declares no verification rules) with the quote
that proves it, say so in the run summary, and let demo-narrative author a
story that does not claim detection — do not fill the gap with a plausible
invention.
Worked example (hh-poverty-targeting, the run this step exists because
of). Its PDD §6 "Verification and fraud rules" is [FIXED] and names five
automated flags, including verbatim: "Response patterns: flagged (not
auto-rejected) if an FLW's score distribution is a statistical outlier vs.
peers — e.g., implausibly uniform answers or a spike just below/above an
eligibility threshold." Two authorable signals fall straight out of it, both
derived rather than invented:
- band-boundary clustering — the instrument's own point table
(
inputs/INSTRUMENT — Nigeria PPI 2020, indicator 2 "How many members are there in the household?", response "A. 3 or less" = 31 points, the largest single-answer jump in the scorecard) makes that one answer the obvious place a score gets pushed over a line. One persona's household-size responses skew to the 31-point band far harder than its peers'. - response-pattern uniformity — one persona's score distribution is implausibly tight versus the cohort spread.
Both are the PDD's own words turned into distributions; neither adds a mechanism the design does not already claim to detect.
Two manifest mechanics that silently void this whole step — get them
wrong and you ship anomalies: [] by a different route:
anomalies[].week/weeks[]are 1-based on this path (synthetic_generate_from_manifest), matched againstVisitSlot.week_index;week: 0is falsy in_anomalies_atand matches nothing at all. (The 0-based reading belongs to therealize-envaudit ensurer, which this provider does not use.)coaching_arcs[].week_triggeredandfield_distributions[].period_rateskeys are 1-based. See § Gotchas for the source lines.- Put the signal on a week the demo's dashboards actually render — a completed
week, inside
timeline. Out-of-window anomalies are skipped in silence. coaching_arcsstill ships EMPTY to the atom (step 2a below).
2a. Generate into the run's labs SyntheticOpportunity — synthetic_create_labs_only
(or reuse the run's existing labs opp) + synthetic_generate_from_manifest.
Send coaching_arcs: [] — keep the authored arcs in the saved manifest as
the source-of-truth narrative, but the in-generate Task-create path 500s at
POST /export/labs_record/ and a single entry aborts the ENTIRE generation, so
no visits land either (jjackson/ace#594; the strip is spelled out in
skills/synthetic-data-generate/SKILL.md § Process step 3). Arcs are created
separately and reliably via task_create_synthetic.
Then Step 0 (template selection) and steps 3–5 (author dashboards, mint runs,
build /run/?run_id= URLs, emit the realized map) are EXACTLY the denovo spine,
and demo-narrative authors the DDD narrative — so Phase 7 becomes
demo-data-setup(ace-run) → demo-narrative → DDD, the same pipeline as
/ace:demo. Archetype branching is preserved at the Phase-7 agent level, and it
names the one EXCEPTION rather than enumerating the archetypes that pass:
focus-group is a hard skip, every other archetype proceeds. (This line used to
list atomic-visit / multi-stage only, which is how longitudinal-visits hit
neither branch — ace#1691, ace#2312. See
agents/synthetic-data-and-workflows.md.)
The interactive run stays live
workflow_save_snapshot completes a run. That is how a par_url becomes a
stable, idempotent deep-link — reopen it next month and the page renders the
same figures — and it is what every dashboard a stakeholder keeps a link to
should have.
It also makes the page read-only. The run view's completed branch prints
"This run is completed… Decisions are read-only" and disables the status
control. So for the one dashboard whose scene shows a reviewer taking a
decision, completing the run is what makes that scene unperformable.
On hh-poverty-targeting/20260730-2210 both runs were completed at
2026-08-01T01:12Z, ~14 minutes before the render at ~01:26Z. The result read
as several unrelated defects and was one: every one of the 10 spec actions
degraded to wait_for/hold because nothing was clickable, 7 scenes produced
only 2 distinct images (5 of 6 adjacent pairs differed by 0.00% of pixels), and
the arc scored 1.0/5 (dimagi-internal/ace#1162).
The rule (Jon, 2026-08-14 — option 1 of three):
- the dashboard whose
roleisreview-action/review/decision:workflow_create_run, then stop. Its run staysin_progress, itspar_urlis built exactly as in step 4, and it is the only link that is not snapshot-stable. Mark itinteractive: trueinsource.dashboards[]. - every other dashboard: unchanged —
workflow_save_snapshot, run completed.
Two alternatives were weighed and not taken: minting two runs for the review workflow (one completed for framing, one live for the decision beat) doubles the handoff for a gain only a longer narrative would use; completing every run after the render makes link stability depend on a render that may fail or be re-run. Revisit if a demo ever needs the review dashboard's link to outlive the recording.
demo-data-setup-qa check 8 (checkInteractiveRunsLive) enforces both halves —
an interactive run found completed fails, and so does a non-interactive run
left in_progress.
The mandate is not free, and it is now VERIFIED rather than assumed
(ace#2430). Completing a run is also how its page stops needing anything
live: the runner returns early on a snapshot (if (snapshotCarriesPipelines) return;) and never opens a stream, while an in_progress run has no snapshot
and therefore re-runs the pipeline SSE stream on every load — the reviewer's,
and every render take's. So this rule buys a performable control at the price of
a live dependency, and on poverty-graduation/20260915-1518 that dependency
failed: the stream died after its first event
(send_sse_event("Loading workflow configuration..."), emitted before any
network I/O) and the interactive dashboard rendered a shell plus a banner
guessing at three causes. The same definition over the same pipeline rendered
perfectly as a completed run — because a completed run never opens the stream.
Check 8, being one-sided, then enforced the dead configuration: the only
QA-passing shape was the one that could not render.
Two things follow, and step 4c is where both happen:
- Probe the live load before recording the
par_url— the stream endpoint, classified byclassifyLiveLoadStream, recorded assource.interactive_live_load. An unprovenin_progressrun now FAILS check 8. - There is an evidenced escape. When the live load is observed failing and
a completed control run of the same definition DOES render, the interactive
run may ship
completed— withcontrol_run_idandupstream_ref. The demo is then honest-but-degraded (the decision is visible, not performable), which is strictly better than a blank page, anddemo-narrativedeclares check 20'sfinale_is_readinstead of authoring a click that cannot land.
Note what did NOT change: the rule itself. Probed live on 2026-09-17, both definitions from that run streamed to completion in 4s and 7s — the labs-side failure did not reproduce two days later. A mandate whose truth moves between Tuesday and Thursday is exactly one that has to be checked at the moment it is recorded, not reasoned about once.
A live run accumulates what the camera did to it. The same decision that
makes the payoff performable makes the run STATEFUL across takes: the task the
scene creates is still there on the next render, the button is gone, and a
must_succeed click aborts the whole thing. Iteration 0 passes and every later
iteration fails on a scene nobody touched. So the interactive run carries a
mandatory companion — the per-render reset registered in step 4b and run by the
spec's setup: {rerun: per_render} block. Three iterations of
spark-facilitator/20260908-2215 (labs run 5508) passed only because a human
had reset the state by hand, which left no trace in run_state.yaml or any
artifact — the run read as reproducible while depending on an undocumented
manual step (ace#2297). demo-data-setup-qa check 18 is the backstop.
Hand-authoring a generator instead of the manifest DSL
Sometimes the manifest DSL cannot express a design and the data gets authored by a hand-written Python script. That is allowed. What is not allowed is shipping one whose determinism is claimed rather than demonstrated — the fixture is the substrate of every rendered scene, so a dataset that cannot be regenerated means any later fix is hand-editing JSON or re-authoring every narration number against a different dataset (ace#1388).
Three obligations, all three or use the DSL:
-
Never derive a persisted value from builtin
hash(). This is the one that actually shipped.hash()of a str — or of a tuple containing one — is randomised per process by design (PEP 456). It is perfectly stable within a process, which is exactly what makesh = hash((user, week)) % 100such an inviting way to derive a fixed per-key value, and it silently reshuffles on the next run. In the spark-facilitator generator it decided the record count. Use a stable digest:def stable_hash(x): return int.from_bytes(hashlib.blake2b(repr(x).encode(), digest_size=8).digest(), "big") -
Sort every iteration that a draw is consumed inside —
for k in sorted(d), neverfor k in some_set. Same symptom by a different route: the RNG is seeded correctly and the draw order is not. -
Run a CROSS-PROCESS determinism check before upload, and publish the generator alongside the fixture with the assertion that it reproduces the uploaded bytes.
The obvious check does not work. "Generate twice in one process, assert byte-identical" passes over both defects above:
PYTHONHASHSEEDis drawn once per process, so both generations see the samehash()values and walk any set in the same order. Measured — in-process identicalTruewhile three processes gave three different digests. A green check over a live bug is worse than no check.The check that catches it re-executes the generator under two different
PYTHONHASHSEEDvalues and compares output bytes. Take the exact snippet fromlib/generator-determinism.ts(CROSS_PROCESS_SELFCHECK) rather than rewriting it.
Before uploading, lint the generator:
npx tsx -e "
import {checkGeneratorDeterminism} from './lib/generator-determinism';
import {readFileSync} from 'node:fs';
const r = checkGeneratorDeterminism(readFileSync(process.argv[1], 'utf8'));
for (const f of r.findings) console.log(\`[\${f.kind}] line \${f.line}: \${f.detail}\`);
process.exit(r.ok ? 0 : 1);
" <generator.py>
The lint catches the two known causes; the cross-process check catches the ones nobody has enumerated yet. Run both — neither is a substitute for the other.
Gotchas (encode every one — they are the difference between a live demo and a dead scene)
-
Labs-only opp ids ≥ 10,000 have no CommCare HQ app. Anything needing a real HQ app (deliver-unit introspection, enabling "Create Review") can't be driven for a synthetic opp — the generator writes records directly. Pin the opp id ≥ 10000.
-
aggregation: "avg"must never carrytransform: "int"— it kills the WHOLE pipeline, and the error names no field. Postgresavg()over an integer column returnsnumeric, which reaches the labs serializer as aDecimal, and the preview fails withAnalysis pipeline failed: Object of type Decimal is not JSON serializable. Nothing points at the offending column, so on a 15-field schema this reads as "the pipeline is broken" and costs an isolation pass withschema_override. Usetransform: "float"(verified live 2026-09-08, pipeline 5501 / opp 10060: the two previews differ in that one word). Reported upstream asdimagi-internal/connect-labs#1630; until it lands, this is an authoring rule. -
A pipeline field's
defaultis documented but does nothing.WORKFLOW_REFERENCE.md § Field Definition Referencedeclares"default": null # Optional. Default value if extraction yields null, and the SQL backend ignores it —pipeline_update_schemaaccepts the key, and rows still come backnull. Measured 2026-09-08 on pipeline 5504:default: 0on four integer fields anddefault: "not_held"on a select, all null on the records where the branch is off. Consequences worth planning around, not just knowing: null-guard in render code regardless, and remember that a visit-level payload over a gated form legitimately carries nulls — whichdemo-data-setup-qacheck 16 treats as unjudgeable, so per-rowcolumn_invariantsbelong on an AGGREGATED dashboard (whose sums are non-null on every row) rather than on the visit-level one. Reported asdimagi-internal/connect-labs#1632. -
Pin the timeline to a fixed Monday. An unpinned trailing window slides off "today" and strands already-seeded runs/flags/audits/tasks on the wrong week, breaking idempotency.
start_datemust be a Monday and must equal the env anchor. -
Do NOT pre-seed the flagged current-week worker's audit/task. The live-demo scene creates them on camera; pre-seeding leaves nothing to click. The flagged worker is the one on the in-progress current week.
-
First persona = network manager,
flag_rate: 0, across flags / task-creator / rollup label. -
A "resolved" cluster needs every audit completed AND every task closed. The rollup drill selector needs one fully-resolved cluster (the "good" drill) and one still-open cluster in a different opp — otherwise
good_*/incomplete_*vars are omitted. -
period_endis EXCLUSIVE (visit_date < period_end). A run window is half-open, so aperiod_endequal to the manifest'stimeline.end_datedrops every record dated on it. Passtimeline.end_date + 1 day. Nothing fails: both runs report cleanly and both pages render — a snapshotted dashboard just silently disagrees with its live sibling over the same fixture. Omitting the bounds is worse, not safer: they default to[today, today). Full mechanism, source quotes and the measured numbers in step 3 (ace#1683). -
Anomaly weeks are read on TWO different bases by two different consumers — state which path you are on.
coaching_arcs[].week_triggeredis 1-based (PositiveInt,_week_start()does(week - 1) * 7ingenerator/fixtures/tasks.py) andfield_distributions[].period_rateskeys are 1-based (period_rates: {2: …}= the SECOND week) on every path. Butanomalies[].week/weeks[]are:- 1-based under
synthetic_generate_from_manifest(thefrom-manifestpath, and the only path theace-runprovider uses)._anomalies_atingenerator/fixtures/engine.pytestsif a.week and a.week == week_indexagainstVisitSlot.week_index, whichexpand_visit_schedulebuilds asfor week in range(1, timeline.weeks + 1)— 1-based by declaration, same counterperiod_ratesuses. Note theif a.weekguard:week: 0is falsy and matches NOTHING, so a 0-based anomaly on the first week is a silent no-op, not an off-by-one. - 0-based under
synthetic_env_ensure's audit ensurer (therealize-envpath).ensure/ensurers/run_audits.pyindexes the resolved week list directly —if week_idx < 0 or week_idx >= len(weeks): continuethenmonday_iso = weeks[week_idx].
Out-of-window anomalies are silently skipped on both paths, so a wrong base costs a demo its entire detectable signal with no error anywhere. See
playbook/integrations/connect-labs.md § binary distributionforperiod_rates' source + live measurement (ace#1518). - 1-based under
Change Log
| Date | Change | Author |
|---|---|---|
| 2026-10-24 | § C3: every cascade pool declares numbers: authored, and labs' generated must come back true (connect-labs #2718). Labs treated ACE's hand-written mirror pools as replaying real cases, so no cascade opp was ever marked generated and canopy's agent panel was refused on all of them. C3 now writes numbers: authored and generates with --expect-generated (lib/labs-generated-provenance.ts): refuse an unmarked pool before sending, exit 4 on generated: false with labs' reason. The clone path never gets the mark. | Claude |
| 2026-10-16 | § C7: the cascade reports follow ACE's Labs template workflows (scripts/labs-report-templates.ts follow). The trio used to follow the deployed code template, so a page defect needed a connect-labs release and the DDD loop pinned every one out of scope (ace#2337, connect-labs#2578). Template workflows (config/labs-report-templates.yaml, home program 10157) make a page fix one publish; the run records cascade.report_templates, and Phase 7 freezes the pages at its end. | ACE team |
| 2026-10-16 | § C7 "Sweep handles": the handle lives only in a field no frame renders (ace#2733). Programme report and opp reports carry it in description, the synthetic opportunity in notes; registry, cohort and org keep it in name. The run id in the programme report's breadcrumb was scored a trust defect by the DDD user-artifact judge on three runs, and on noora-health/20261015-1129 put a real organisation's name on an illustrative page. sweep-labs matches name, description or notes (sweepHandleField). | ACE team |
| 2026-10-16 | Step 3c no longer runs checkGapCopy; demo-narrative § 3d does (ace#2244). The check needs the why-brief's gaps[], which demo-narrative authors one step later, so at this position it could never run. 3c now records each dashboard's workflow_id + scope for 3d to read the uploaded render back, and keeps the authoring rules. | ACE team |
| 2026-10-15 | § C3 names the labs request-size ceiling (ace#3007). HTTP 413 at a 3.9 MB manifest (2.44 MB accepted) on smallholder-seasonal/20261015-1402; the mechanics now say how to slim a programme-length pool safely (dedupe only paths whose absence reads as their value; _time_varying_paths are not re-stamped), filter per-form values by form name, and send multi-MB manifests over direct JSON-RPC. | ACE team |
| 2026-10-06 | § C3 review_routing names the write path (ace#2740). Each flagged pool visit carries review: {flagged: true, status, review_status, flag_reason} (connect-labs #2264 — before it, a mirror pool had no way to file a record as sent to review; status/flagged were always drawn from the persona's flag_rate), and every persona sets flag_rate: 0 so nothing un-flagged is sent to review. Found on spark-facilitator/20261004-1706 Phase 7 attempt 3. | ACE team |
| 2026-10-19 | § C7 writes demo-data-setup_manifest.yaml (an index of the per-partner manifests under cascade-build/) and dashboard-terms.yaml (from the registry's labels) on the cascade path (ace#3251). Both are required by lib/artifact-manifest.ts; the cascade wrote neither, demo-data-setup-qa passed 10/10 on connect-fi/20261018-1440, and finalize then found 7/10 required artifacts after the DDD loop. New lib/cascade-build-artifacts.ts + scripts/cascade-build-artifacts.ts; demo-data-setup-qa check 29 fails a build missing either. | ACE team |
| 2026-10-06 | Build QA before the hand-off, so the demo judges stop finding build output (ace#2735). Operator: "is the demo catching bad output from the program build? ... how can we be better at the program build level in terms of QA or other so DDD isn't the one catching it?" § C3 adds review_routing (what the PDD does with each flagged record — and the pool writes it), checkPlannedWorkerRates (a worker-level rate over ≤ 1 entity is binary) and single_child_levels; § C6 adds cascadeBuildOutcomes — review routing, binary worker rates, constant displayed columns, one-child drill levels — judged against the saved run and the visit rows. Each was a DDD finding on spark-facilitator/20261004-1706; each now fails there, in test/lib/cascade-build-qa.test.ts, on that run's real output. | ACE team |
| 2026-10-06 | Invented names a viewer can read, five partners, a headline that differs (ace#2727). § C0: invented partners are realistic organisation names with a trailing (example) marker (Tiyende Community Trust (example)), not Example partner A — be59309d's honesty fix satisfied ACE's rail with a placeholder canopy's DDD judge scores at clarity 2, and spark-facilitator/20261004-1706 stalled at 2.0 across 15 judged passes on it. The story plan now carries a display roster (workers[{username, display_name, partner}], entities[{id, name, worker}]; no codes, no placeholders, no real person's name from the inputs), headline_indicators + headline_spread (a headline flat across all partners fails), and five partners by default and floor. § C3 carries a TODO where the labs half (snapshot display names, registry display.entity.label_field) will be wired; § C7 adds *_worker_name. Enforced: checkCascadeStoryPlan / verifyCascadeStoryLanded, test/lib/cascade-story.test.ts. | ACE team |
| 2026-10-05 | § C5 plans the benchmark publishes with planBenchmarkRepublish (ace#2717). Oldest first, refusing a missing week; the Benchmarks tab's as-of must equal the last period; the same re-publish is required after a registry edit + rebuild. A classifier refusal of benchmarks_publish is surfaced to the operator. | ACE team |
| 2026-10-01 | § C7 builds every *worker_review_url with one builder (ace#2521 recurrence). spark-facilitator/20260926-1800 wrote all three worker-review links with owning_program_id and no program_id, after § C7's prose already forbade it — the URL was hand-assembled from the programme report's "Review →" links. lib/worker-review-url.ts (buildWorkerReviewUrl, workerReviewUrls, rescopeWorkerReviewUrl) is now the only constructor; § C7 calls scripts/worker-review-url.ts to build them from the run_state cascade or repair a written realized.json in place. checkWorkerReviewUrlScope stays the gate. Enforced: test/lib/worker-review-url.test.ts (fixtures: that run's realized.json + run_state). | ACE team |
| 2026-09-17 | New step 4c: the in_progress mandate is PROVEN before its par_url is recorded (ace#2430). § The interactive run stays live required exactly one run to stay live and check 8 enforced it, but nothing ever loaded that page — step 3 proves the pipeline extracts and check 7 reads the data through api/<def>/pipeline-data/, a side-channel around the stream the page itself uses. The two run states are not symmetric: a completed run's page returns early on its snapshot (workflow-runner.tsx: if (snapshotCarriesPipelines) return;), while an in_progress run has no snapshot and re-runs the pipeline SSE stream on every load, every take. On poverty-graduation/20260915-1518 that stream died after its first event and the interactive dashboard could not load at all — and check 8, being one-sided, enforced the dead state: a completed interactive run failed, so the only QA-passing configuration was the one that could not render. Now: probe api/<def>/pipeline-data/stream/ with refresh=1, classify it with classifyLiveLoadStream (lib/interactive-live-load.ts), record source.interactive_live_load, and — when the live load fails against a completed control run of the SAME definition that DOES render — ship the run completed under an evidenced escape carrying control_run_id + upstream_ref. Probed live 2026-09-17 on definitions 5714 and 5695 / labs opp 10065: both streamed to completion (4s / 7s, 137 + 9 rows), i.e. the labs-side failure did NOT reproduce — which is the argument for checking at record time rather than reasoning once. Enforced: test/lib/interactive-live-load.test.ts (fixtures trimmed from those real captures). | ACE team |
| 2026-09-17 | A REPEAT group is no longer a declared_omissions[] class — step 2c.4 said it was structurally impossible, and both halves of that premise are refuted (ace#2432). auditDataset resolved a conditional field by leaf name against a flat record, and leafPaths treated an array as an opaque LEAF — so /data/roster/member_name had no path at all while its gate, at the record's top level, resolved fine. Measured on poverty-graduation/20260915-1518 (deliver app e4594937038c42d2be4d01f45df44209 v7, 2,207 records): member_name, is_member and member_flag each reported conditional-missing on 1,379 records — exactly the 1,379 carrying a non-empty form.roster, whose roster[].member_flag sums re-derive the app's own /data/member_count with 0 failures. The run spent three declared_omissions entries exempting data that was present and correct, so materialising the repeat scored the same as skipping it. leafPaths now descends into an array (index elided, so one spec field addresses every row), which fixes auditDataset and the scrub's unresolvedFields together and makes the scrub clear an off-branch value from EVERY row. The other half: BeneficiaryCohort.repeat_groups: dict[str, RepeatGroupSpec] is in the labs manifest schema, so the generator can emit one. Enforced: test/lib/dataset-constraints-repeat-groups.test.ts, against real captures of that run's deliver app and records — including the negative controls, since an empty repeat and rows lacking the leaf must both still read as absent. | ACE team |
| 2026-09-10 | This skill is now the DECLARED producer of products.synthetic.{provider, labs_opp_id, workflows, source, render_code_patched_this_run} (ace#2354). PRODUCT_PRODUCERS in lib/phase-products-schema.ts attributes each products.* key to the skill that writes it, and ace-web's skill fork trims by it — so a Phase 7 fork at demo-narrative now carries exactly these five keys and drops narrative / ddd_*. Nothing changes in what this skill writes; if it starts writing a new synthetic.<key>, add the attribution there too (the coverage test fails on a schema-declared key with no producer). | ACE team |
| 2026-09-10 | Step 4b's registered reset command is now SELF-RESOLVING, and a pinned one fails QA (ace#2351). The step said "write the command with an ABSOLUTE path", and the path that produced was ~/.claude/plugins/cache/ace/ace/<version>/scripts/reset-labs-run-state.ts under one user's home. demo-narrative copies it verbatim into the spec's setup.command (rerun: per_render), so per-run state pinned a VERSION directory — the cache keeps every prior version, so the path kept resolving to STALE code after every /ace:update — and a HOME directory. Measured on spark-facilitator/20260909-2242 (pinned 0.13.1413, pre-ace#2325: grep -c buildRunPageUrl = 0, no --workflow-id) and 20260910-0541 (/Users/<name>/…/0.13.1426/…). The absolute-path rule stays (canopy runs setup.command via subprocess.run(shell=True, cwd=<spec toplevel>), record_video.py:398); the path now resolves ITSELF: bash "$(python3 -c "…installed_plugins.json…['plugins']['ace@ace'][0]['installPath']")/bin/ace-reset-labs-run" …, where the new shim re-resolves the installed root (never dirname $0) and execs the reset there. Verified live on labs run 5590 / workflow 5502 / opp 10060 through Python shell=True from a foreign cwd. renderResetCommand() in lib/labs-run-state-reset.ts is the canonical string; demo-data-setup-qa check 18 gains render_reset_command_pinned. Enforced: test/lib/labs-run-state-reset.test.ts, test/scripts/ace-reset-labs-run-root.test.ts. | ACE team |
| 2026-09-09 | Step 4b's registered reset command must name the WORKFLOW, and the CLI now refuses without it (ace#2325). reset-labs-run-state.ts defaulted its CSRF page to <base>/labs/workflow/run/<run_id>/, which is not a labs route — the run page is /labs/workflow/<workflow_id>/run/?run_id=<run_id>, workflow id in the PATH and run id in the QUERY, and the CLI never had the workflow id. So the token was read off a non-page and the POST returned 404, reported as run-not-found on a run that was fine. The natural invocation — the one this step registers into source.render_reset.command and demo-narrative copies into setup.command — was the broken one, and because that block is rerun: per_render it blocked EVERY render of a state-mutating demo. Reproduced on spark-facilitator/20260909-2242 (labs run 5590 / workflow 5502 / opp 10060): without the flag run-not-found exit 1, with it cleared record_reviews exit 0. Now: buildRunPageUrl in lib/labs-run-state-reset.ts, --workflow-id/--opportunity-id on the CLI, a fail-fast when neither a page URL nor a workflow id is given, and a 404 detail that names the wrong-token-page reading alongside the missing-run one. Enforced: test/lib/labs-run-state-reset.test.ts. | ACE team |
| 2026-09-09 | New step 2c.1b: the deliver app exposes its calculate expressions, and a computed field must be REPLAYED rather than drawn (ace#2328). specFromDeliverApp reads relevant and constraint and ignores calculate, and nothing in this skill mentioned it — grep -niE "\.calculate|hidden field|DataBindOnly" over the skill, synthetic-data-generate and lib/dataset-constraints.ts returned zero matches. But get_opportunity_apps returns one on every hidden/DataBindOnly question, which is where an app keeps its payment arithmetic: 26 of deliver app 61eedcad… v10's questions carry one, including is_payable, capped_index, key_date_part and the entity_key the deliver unit submits as its dedup id. spark-facilitator/20260908-2215 recorded them as "DELIBERATELY NOT POPULATED" on the false premise that they were not exposed, so its dataset carried no payment decision, its dashboard had only a per-worker scorecard, and its payoff became a filter over 12 rows — three consecutive non-converged runs. Replaying them on 20260909-1211 produced 149 distinct dedup keys over 155 community meetings and 6 records that are complete, correct and unpaid because they are the 4th on their FCAP step. The manifest cannot do this: it draws every field independently, so a drawn is_payable contradicts the fields it is a function of — the ace#1346 class by another route. | ACE team |
| 2026-09-08 | New step 4b: register the per-render reset for the interactive dashboard (ace#2297). A labs run's spawned_tasks persists server-side across renders, so the second take of the scene whose payoff CREATES something finds the control gone and its must_succeed click aborts — order-dependent, invisible on the first pass. On spark-facilitator/20260908-2215 (labs run 5508, workflow 5502, opp 10060) iterations 0-2 passed only because a human had reset the state by hand, leaving no trace in run_state.yaml, and the reset itself lived as reset_and_realize.py inside the DDD run dir — uncommitted, not worktree-portable (ace#2287), swept. Now: scripts/reset-labs-run-state.ts (+ lib/labs-run-state-reset.ts) in the repo, a source.render_reset block in the handoff whose state_keys come from workflow_get → saved_runs.snapshot_inputs.state_keys, and demo-narrative wiring it as setup: {rerun: per_render}. Backstop: demo-data-setup-qa check 18. | ACE team |
| 2026-09-08 | Step 2c.2's write-back must preserve visit.id losslessly (ace#2249). The labs generator mints visit.id as a 60-bit integer, over JS Number.MAX_SAFE_INTEGER, so a plain JSON.parse/JSON.stringify round trip silently rewrote 490 of 505 ids on a live run while every gate stayed green (auditDataset judges values, not keys). Fixed at two points in lib/dataset-constraints.ts: scrubOffBranchFields's internal deep copy now uses structuredClone instead of JSON.parse(JSON.stringify(...)), and new parseJsonPreservingBigInts / stringifyJsonPreservingBigInts helpers (an out-of-safe-range integer literal round-trips as a bigint) replace JSON.parse/JSON.stringify on both ends of the caller's own read/write. | ACE team |
| 2026-09-08 | New step 2c.4: declare, with a reason, the residuals labs STRUCTURALLY cannot emit (ace#2225). Check 9 had no evidenced escape, so a run whose only residual violations were such fields could reach green only by narrowing the spec — the behaviour ace#1658 built the derivation to prevent — or not at all. Both runs of poverty-graduation took the permanent fail. Three recurring classes, all measured on 20260908-0510: a per-member REPEAT group the flat generator cannot produce, a Trigger read-aloud label CommCare submits no value for, and an image with no labs ImageConfig corpus. New declared_omissions[] block in branch-scrub_report.yaml; the reason is required, and a blank one exempts nothing. | ACE team |
| 2026-09-07 | New step 3d: enumerate every coined label and prove its definition is reachable FROM the label (ace#2219). This skill had ZERO guidance on legibility — `grep -niE "jargon | coined |
| 2026-08-27 | Step 3c narrowed to CAPABILITY gaps, and gap prose is no longer scanned (ace#1762). Measured against the real artifacts rather than its fixtures, the narrative pass ran at ~44% precision: 9 findings on hh-poverty-targeting/20260827-0323, of which 3 were a DECISION gap firing on claims that merely NAME thresholds (its own proposed remedy, not a contradiction) and 1 was one gap's proposed_action naming another gap's subject. Both are subtractions: constraining is now CAPABILITY only — the RESEARCH carve-out's reasoning covers DECISION verbatim, since both forbid a qualified claim and no keyword match can tell that from naming the subject — and narrativeSources() no longer emits why_brief.gaps[] at all, replacing the narrower self-exemption (exemptGapId removed as dead). Measured after: 9 → 5 findings over 4 distinct strings, all genuine area / adjudication CAPABILITY hits. Precision is the whole asset for a report-only check — a gate that cries wolf is how the real misses get waved through (ace#1744). | ACE team |
| 2026-08-27 | Step 3c also reads the NARRATIVE artifacts, not just dashboard render_code (ace#1759). The same run's why-brief contradicted a gap it itself declared — decisions-are-recorded asserts the disposition is recorded "with its reason" while adjudication-log-is-run-state-not-a-register declares no durable register and no reason field. Only a post-render judge caught it, on iteration 2. checkGapCopy's sources now takes already-prose entries alongside render_code, and narrativeSources() builds them from why_brief.spine[].{claim, rationale}, why_brief.gaps[].{detail, proposed_action} and unified_spec.scenes[].{concept_claim, show, narrative}, each labelled by origin. A gap is exempt from its OWN detail/proposed_action. Still report-only; still term matching, so an inverted claim (scene 4's low-variance→fabrication converse) is not caught. | ACE team |
| 2026-08-27 | Step 3c: check dashboard COPY against the why-brief's declared gaps (ace#1750). The narrative was gated against gaps[] and the dashboard prose was not, so hh-poverty-targeting/20260827-0323 shipped four on-screen assertions of exactly what its own gap list declared unsupported — across two dashboards, one contradicting the same page's leave-one-out definition two lines above it. In all four the narration was clean and the UI copy was the offender, so every instance had to be caught by a post-render LLM judge. New checkGapCopy in lib/gap-copy-check.ts, report-only. Catches subject-term repetition, not semantic equivalence — a coverage claim phrased without the gap's nouns still slips. | ACE team |
| 2026-09-17 | Step 5b now sends the manifest FILE, via scripts/run-labs-synthetic-generate.ts (ace#2433). The ace#1737 remedy was unsatisfiable as written: it said to send "that file's contents" to synthetic_generate_from_manifest, but that atom takes manifest_yaml as an inline string and exposes no file-handle parameter, so the model had to re-emit the YAML into the tool call — making the wire payload one emission and the archive another, which is the exact defect ace#1737 identified. The script reads the file and POSTs tools/call server-side, so the archived file IS the wire payload by construction. | ACE team |
| 2026-08-27 | Step 5b: the archived manifest must be the same bytes that were sent, and must round-trip parse (ace#1737). hh-poverty-targeting/20260824-1404 published a demo-data-setup_manifest.yaml that does not parse as YAML — flow mappings were column-aligned, so the longest key got zero spaces before its {. Generation had succeeded, so the archived copy was a SECOND emission rather than a capture of the wire payload; the break surfaced only when the next run tried to fork it. Drive was ruled out by direct probe (the pathological shape round-trips byte-for-byte). Fix: author once to a local file, send that, upload that file's bytes, then read back and yaml.safe_load. | ACE team |
| 2026-08-26 | period_end is exclusive — derive it as timeline.end_date + 1 day (ace#1683). A run window is half-open (visit_date >= date_from AND visit_date < date_to, query_builder._date_window_where), and the natural authoring move — pass the manifest's own timeline.start_date/end_date — drops the fixture's entire final day from any snapshotted dashboard while its live sibling keeps it. Measured on hh-poverty-targeting/20260824-1404 (labs opp 10047): snapshot run 5245 total=2186, live run 5249 total=2237, fixture 2237, re-mint at period_end 2026-08-31 → 2237, exact. Nothing failed; every existing check passed. Also documented: period scoping bites only on the SNAPSHOT path, and omitting the bounds defaults to [today, today), a zero-width window. New backstop: demo-data-setup-qa check 11. | ACE team |
| 2026-08-26 | Step 1c (ace-run): giving the demo something to detect is now a REQUIRED authoring step, derived from the PDD's own declared controls. hh-poverty-targeting/20260824-1404 shipped anomalies: [] and no per-persona divergence, so the cohort had no signal in it — completion 64–77%, PPI ~34, 15 undifferentiated workers in the queue, one decision against an empty adjudication log — and the DDD loop ended stopped_not_converged at concept 2.0/5 on use_case_soundness ("the scenario does not exercise the feature it demonstrates"). Four obligations (enumerate the PDD's controls verbatim → instantiate ≥1 → cite the clause → state what a reviewer would SEE), an explicit ban on inventing a pattern the design does not declare, an honest detectable_signal: none escape, and the two manifest mechanics (1-based anomaly weeks with week: 0 a silent no-op; coaching_arcs: [] to the atom) that void the step by a different route. | ACE team |
| 2026-08-26 | Gotcha rewritten: anomalies[].week is read on two different bases by two different consumers — 1-based under synthetic_generate_from_manifest (_anomalies_at vs VisitSlot.week_index, and if a.week makes week: 0 match nothing), 0-based under synthetic_env_ensure's run_audits ensurer (direct index into the week list). The previous single-sentence "anomaly weeks are 0-based (audits)" was true only of the path the ace-run provider never takes. | ACE team |
| 2026-08-26 | Step 4b becomes step 2c and grows two halves: the DatasetSpec is now DERIVED from the deliver app (specFromDeliverApp, hand-declared entries merged as ADDITIONS via mergeDatasetSpecs, unparsed[] reported), and a post-generation branch scrub (scrubOffBranchFields) removes the off-branch values the labs manifest has no primitive to avoid — written back to the fixture folder before any dashboard run is minted. It moved earlier in the process because the old position ran after the dashboards were already reading the fixture. New product 7-synthetic/branch-scrub_report.yaml + source.dataset_constraints. ace#1658. | ACE team |
| 2026-08-26 | Persist the realized dataset's SHAPE into the handoff — source.record_counts (verbatim from synthetic_generate_from_manifest) + source.data_shape (rows / periods / groups), plus step 2b resolving the two axes record_counts cannot answer. demo-narrative had no cardinality input at all, so it could not tell a filter over 5 rows from one over 500; on bednet-check-2-visit/20260825-1310 that authored a filter demonstration over a five-worker cohort and ended the DDD loop stopped_not_converged at concept 3.0 after four render iterations. ace#1670. | ACE team |
| 2026-08-26 | Step 3b: utility-resolution gate before every render_code upload (ace#1662). labs purges Tailwind against its own Django templates, so a utility used only in DB-stored render_code is dropped from the bundle and degrades silently to the unstyled baseline. text-rose-700 styled consent 89.7% · below the 90% floor — the only pay-affecting figure on the LLO weekly-review dashboard — and rendered near-black for an unknown number of runs; a missing h-28 left all 12 weekly bars at 0px. New pre-upload lint scripts/check-render-code-utilities.ts over lib/tailwind-utility-resolution.ts, with loud-failure semantics, substitution guidance, and the lint-the-whole-source rule (3 of 5 misses were pre-existing). Root cause tracked upstream as connect-labs#1294. | ACE team |
Not in scope
ace-runprovider — Phase 7 convergence (Plan C); Phase 7 becomes this provider.- Rendering / judging / video — owned by canopy DDD, invoked after
demo-narrativebyagents/demo.md.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 192k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 192k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 192k | Updated | Claude CodeCodex | Verified | Install |