Skip to content

dimagi-internal/ace / skills/app-screenshot-capture

app-screenshot-capture

Verified

Run app smoke recipes on a local Android emulator and capture per-step screenshots (e.g. for a training deck). Needs Maestro recipes and released apps.

App Screenshot Capture

Run the smoke recipes from app-test-cases.yaml against a local AVD, capture PNGs at every takeScreenshot step, and ship a thin per-app UX smoke judge so Phase 6 has a meaningful (but cheap) signal that the built apps are usable end-to-end. Deep, per-journey UX grading lives in /ace:qa-deep → app-ux-eval — this skill is intentionally shallow.

Inputs

ParameterUsed forResolver
journeys_docpersona summary for the UX judge prompt; archetype contextexplicit arg (a Drive file id); default: ACE/<opp>/runs/<run-id>/2-scenarios/pdd-to-app-journeys.md; persona fallback: ACE/<opp>/inputs/pdd.md
test_cases_catalogsmoke-recipe selection (is_smoke: true) + recipe pathsexplicit arg (a Drive file id); default: ACE/<opp>/runs/<run-id>/3-commcare/app-test-cases.yaml
hq_domainthe ${HQ_DOMAIN} env varexplicit arg; default: 3-commcare/app-deploy_summary.md
app_output_keysthe output key each app's previews index names (§ Step 4.9, resolveAppOutputKey)explicit arg; default: run_state phases.commcare-setup.products (the apps block, sole writer app-deploy)
organization_slugthe org for EVERY Connect call in this skill. On a PM→NM run the opportunity is HELD by the NM org, but the PM org's URL serves every surface this skill reads (invite read-back, dashboard, learn/deliver progress); live matrix in playbook/integrations/connect-api.md § PM→NM org-URL matrix. runConnectOrgs() (lib/connect-orgs.ts) is the ruleexplicit arg; default: run_state phases.connect-setup.products.connect.pm_org_slug (legacy: organization_slug)
opportunity_id, opportunity_namethe opportunity under test; ${OPP_NAME} (verbatim, logging/screenshot context only)explicit arg; default: run_state phases.connect-setup.products.connect.opportunity.{id, name}
opp_run_id${OPP_RUN_ID}, the deterministic tile matcher (§ OPP_RUN_ID)explicit arg; default: phases.connect-setup.products.connect.opportunity.tile_token when present (a multi-role run, where every role's tile carries the run id — skills/_worker-roles.md), else top-level run_state.yaml.run_id. Outside a run, pass the token that uniquely identifies the opportunity's tile
out_folder_idwhere previews, forensics and the summary goexplicit arg; default: the run's 3-commcare/previews/ and 6-qa-and-training/ folders
test userthe pool test worker the device signs in as (test_worker in the mobile_ensure_avd_running result; PHONE/PIN auto-injected into recipes). It may differ from ${ACE_E2E_PHONE}, the only phone connect-opp-setup Step 7 invites, so Step 3a invites the signed-in worker when needed (ace#3228)the session's test worker (see the connect-login.yaml notes below)

Outside a run, pass every explicit parameter. The run_state write-back is skipped; the summary goes to out_folder_id and the caller records it. Learn completion is one-way per (test user, opportunity), so on a live opportunity a consumed test user records satisfied-by-prior-completion for Learn (Step 2.7) rather than re-walking it.

Recipes are read by path from the entries in app-test-cases.yaml (recipe_path field). They were composed and validated by Phase 3's app-test-cases skill — this skill does NOT compose or validate recipes itself. If a smoke recipe is missing or malformed, halt and point at app-test-cases.

Products

  • 3-commcare/previews/<app-output-slug>/<NN>-<step-name>.png — per-step PNGs of a smoke leg, filed with the phase that BUILT the app (Phase 3), not the phase that walked it (anyone-with-link permission set at upload for Slides ingest). § Step 4.9 says which folder each leg writes.
  • 3-commcare/previews/<app-output-slug>/_previews.yaml — one per app: the authoritative preview index ace-web reads (§ Step 6.2). This skill is the ONLY writer of both app folders.
  • 6-qa-and-training/screenshots/<recipe-base>/… — Phase 6's own FORENSICS only: per-step .xml ui-dumps, *-FAILURE.{png,xml}, 00-postlearn-landing.xml. No passing-leg PNG goes here any more.
  • 6-qa-and-training/app-screenshot-capture_manifest.yaml — fileId/alias index consumed by training-flw-guide and training-deck-generate. Still lists EVERY frame by file_id; its consumers read by id, so the PNGs moving folder does not touch them.
  • 6-qa-and-training/app-screenshot-capture_verdict-shallow.yaml — thin per-app UX smoke verdict

Per-opp content only. Common Connect navigation screenshots come from the standalone connect-baseline-screenshots skill (NOT a Phase 6 dispatch).

Process

Step 0: Prewarm the device FIRST (the lease boots while you prep)

Before reading anything, call:

mobile_ensure_avd_running({ avdName: '${ACE_AVD_NAME}', prewarm: true })

It returns at once. On the lease backend it starts acquiring ACE's emulator machine (a multi-minute EC2 boot) in the background, so the boot overlaps Steps 1–2.8 instead of following them; on the local backend it is a no-op (prewarm: "skipped" — the AVD cold-boots inside Step 3's blocking call). It is idempotent: if qa-and-training already prewarmed in its Step 0, this joins that acquire (prewarm: "joined" / "held").

  • Step 3's normal (blocking) mobile_ensure_avd_running joins the in-flight acquire, then registers the test user and verifies — nothing else changes.
  • If a gate below halts before Step 3 (Step 2's input pre-flight, Step 2.6's recipe-sanity probe, Step 2.8's fully-consumed), call mobile_stop_avd before returning, so the prewarmed machine is released rather than left to idle out.
  • A prewarm refusal is immediate and typed (LEASE_STATE_SELECTOR_MISMATCH, LEASE_STATE_UNAVAILABLE, LEASE_NOT_CONFIGURED): halt on it exactly as you would on Step 3's call. A failure later in the background boot is logged, not thrown — Step 3's call re-acquires and surfaces it typed.

Lease idle window — keep the device steps flowing

A lease is released after idle_release_s (900 s) with no lease call (mobile_ensure_avd_running's lease.idle_release_s). Every mobile atom is a lease call and an atom in flight never idles out — but the gaps BETWEEN atoms count. So, from the moment the machine is ready (Step 3 returns, or a prewarm finishes booting) until the last device step:

  • Keep device steps back to back. Do the LLM-heavy work — Drive uploads (Steps 5.7/5.8), the manifest (Step 6), judging (Steps 7–8), anything that writes a PR — AFTER the last device step, not between legs.
  • Bridging an unavoidable long gap (e.g. a long read of Learn progress between legs): call mobile_diagnose — a cheap read that resets the idle clock — at least every ~10 minutes.
  • After a prewarm, reach Step 3 within 900 s of the machine becoming ready. Steps 1–2.8 are reads and normally take a few minutes; if they ran long, Step 3 still works (it replaces a lost machine) — it just pays a second boot.
  • LEASE_LOST mid-skill means the machine is gone, with everything on it. Re-run mobile_ensure_avd_running, then Step 4's prerequisite recipes (login, claim/resume) before repeating the step — login, claim and Learn state do not survive onto a new machine. A Learn leg that was already walked to completion on Connect is NOT re-walkable (Step 2.7): the new machine proceeds to the Deliver leg.

The local backend has no idle window; these rules are harmless there.

Step 1: Read upstream artifacts

Read 2-scenarios/pdd-to-app-journeys.md and 3-commcare/app-test-cases.yaml from Drive. If either is missing or empty, halt with a structured error pointing at the upstream phase:

  • Missing 2-scenarios/pdd-to-app-journeys.md → Phase 1 (pdd-to-app-journeys)
  • Missing 3-commcare/app-test-cases.yaml → Phase 3 (app-test-cases)

Do NOT generate recipes or test cases independently — Phase 6 is an executor, not a synthesizer.

Step 2: Input completeness pre-flight

Before booting the AVD, verify the upstream Phase 3 outputs are structurally complete. Phase 3's app-test-cases SKILL contracts BOTH the master yaml AND a per-journey Maestro recipe (see skills/app-test-cases/SKILL.md § Outputs). Half-emitted state — the master yaml present but recipes missing — is the canonical "upstream incomplete output" failure mode. Catch it here before AVD wall-clock burns.

Read app-test-cases.yaml. Filter journeys[] to entries with is_smoke: true (this is the only set Phase 6 walks). Deep (non-smoke) journeys carry recipe: deferred — Phase 3 does NOT author their recipe files (they are generated lazily by /ace:qa-deep; see skills/app-test-cases/SKILL.md § Products + jjackson/ace#605). This skill never reads or runs deep journeys, so a recipe: deferred entry is expected and must NOT be treated as a missing-recipe failure. Scope every check below to the is_smoke: true subset. Then group the smokes by the app: field. Run these checks in order. Halt at the first failure with a structured PLATFORM-tag auto_surfaced entry naming the exact remediation command:

Failure modePLATFORM messageRemediation
Master yaml has zero is_smoke: true journeysapp-test-cases.yaml has no is_smoke:true journeys; upstream Phase 3 (app-test-cases) emitted no smoke set/ace:step app-test-cases <opp>/<run-id>
app: learn smoke count != 1 OR app: deliver smoke count != 1app-test-cases.yaml smoke set malformed: expected exactly one is_smoke:true journey per app, got learn=N deliver=M/ace:step app-test-cases <opp>/<run-id>
3-commcare/recipes/ subfolder does not exist on Driveapp-test-cases.yaml declares is_smoke:true journeys but 3-commcare/recipes/ subfolder is missing — upstream Phase 3 produced incomplete output (master yaml without per-journey recipes)/ace:step app-test-cases <opp>/<run-id> BEFORE retrying this skill
One or more smoke journeys' recipe_path doesn't resolve to a real filerecipe_path journey-<app>.yaml referenced by app-test-cases.yaml does not resolve under 3-commcare/recipes/ — upstream Phase 3 produced an incomplete output set/ace:step app-test-cases <opp>/<run-id> BEFORE retrying

(A non-smoke journey with recipe: deferred and no recipe file is NOT a failure — those are generated on demand by /ace:qa-deep. Only the is_smoke: true journeys are checked here.)

Each of these halts writes the incomplete-mode verdict shape (see Step 9 below) with verdict: incomplete and the matching PLATFORM auto_surfaced entry. Do not write verdict: fail for these — fail is reserved for cases where the recipes ran but a smoke recipe broke. Upstream gaps are incomplete, not fail.

This pre-flight is not optional, even when journeys + recipes are on disk. The count check is the load-bearing gate, not the recipe- presence check. Caught in vivo on malaria-itn-app run 20260517-1829: Phase 3 emitted smoke_journeys_per_app: {learn: 0, deliver: 1} (a known-incomplete state) and Phase 6 ran the Deliver recipe anyway — producing a recipe-vs-app-state mismatch (post-claim handoff landed in Learn, not Deliver, because Connect gates Deliver behind Learn- assessment completion; see docs/learnings/2026-05-18-connect-gates-deliver-on-learn-completion.md). The agent SHOULD have halted at row 2 of the failure-mode table the moment it read learn=0. If the agent finds itself rationalizing past this check ("the recipe is on disk, the AVD is healthy, I'll just run the Deliver smoke and skip the Learn one") — that's the exact anti- pattern this table prevents. Halt instead.

The agent-level pre-flight (agents/qa-and-training.md § Pre-flight checklist) catches the same class of gap before the skill is dispatched. This skill-level check is the second line of defense for direct /ace:step app-screenshot-capture invocations.

Step 2.5: Trust the upstream restore (no local probe)

Recipes assume the AVD is at the Connect "New Opportunities" home — the app is configured, the test user is signed in to PersonalID, the opp tile is reachable. This is a precondition of running the smoke recipes, not a state we adapt to. Per the "phase preconditions are restored, not adapted" pattern (see CLAUDE.md § Phase preconditions), the restore happens in mobile_ensure_avd_running, unconditionally:

  • Local backend (since 0.13.203): loadSnapshot('registered-test-user') every Phase 6 dispatch. ~3s, deterministic. If the snapshot doesn't exist (fresh machine, deleted, etc.), tier-2 auto-bootstrap fires: install CommCare APK if missing → registerTestUser → saveSnapshot. ~3-5 min on fresh-machine first dispatch; ~3s thereafter via the freshly-saved snapshot. Phase 4's connect-opp-setup already invited ${ACE_E2E_PHONE} to the run's opp, so the CONNECT-ID-3F server-side invite-check precondition is satisfied automatically. /ace:mobile-bootstrap is no longer required as an operator pre-step inside /ace:run — useful only for ad-hoc workstation setup outside the lifecycle.
  • Lease backend (the default where no AVD is provisioned): each lease is a fresh EC2 machine from ACE's image, leased through canopy-web (infra/capabilities/android-emulator/README.md), booted with the CommCare version the selector map is for (cc-<ACE_CONNECT_APK_VERSION>). The image applies the environment baseline at boot; the client checks its fingerprint (lease.baseline_parity) and re-applies it when it differs (lease.baseline_actions), then registers the test user live and verifies the first screen. Same precondition contract, different mechanism. A machine this session already holds is reused only if its screen classifies ready; anything else is restored in full.

After mobile_ensure_avd_running returns, AvdInfo.heal.deviceUserState carries the outcome: { classified_as, attempted, healed_via, verified_as, ui_dump_signal, bootstrap_steps }. Local shows attempted: true, healed_via: 'local-bootstrap'; the lease shows healed_via: 'lease-bootstrap' (or 'none' with bootstrap_steps: ['verified-ready-reused-lease'] when a reused machine was already ready).

A successful return means the restore sequence ran, NOT that every step was independently confirmed (ace#1067). Read the heal block, not status, for confidence — and do not halt on either of these:

  • verified_as: "unknown" is the ORDINARY post-bootstrap verdict on a healthy device (the ready definition is deliberately narrow). Not a fault, not a reason to stop.
  • A -unverified suffix on a registration step (registered-unverified, register-already-unverified) means the registration call returned but no probe confirmed the resulting device state. Also normal. The suffix exists so the log can't claim registered next to verified_as: "unknown" — the exact overclaim that made a caller trust a device that wasn't registered and burn a full recipe cycle.
  • On the lease backend the heal ends with a launch + UI-dump probe, so verified_as is ready on success — and a lease heal that cannot show ready throws DeviceUserStateError instead of returning. (The retired cloud backend ran no probe and left verified_as absent; a run log from before 2026-10-14 may still show that.)

What SHOULD stop you is a thrown typed error — see below.

Recovery escalation. If mobile_ensure_avd_running threw DeviceUserStateError, two recoverable error codes:

  • snapshot-load-failed — the snapshot doesn't exist (fresh machine, snapshot was deleted) OR the emulator console couldn't load it. The remediation is /ace:mobile-bootstrap — it'll register the test user, configure the app, and save a fresh snapshot.
  • needs-app-config / needs-personal-id / commcare-not-installed (as the verify: segment in the error attempts) — the snapshot loaded but post-restore probe still showed a wiped state. This is snapshot corruption or post-snapshot APK upgrade drift. Same remediation: /ace:mobile-bootstrap re-snapshots.
  • uiautomation-unavailable / device-unreachable / probe-failed — the probe did not observe the device, so the class says nothing about registration or app state (ace#1155). Do NOT run /ace:mobile-bootstrap on these; it resets a working device's PIN for nothing. uiautomation-unavailable specifically means another automation client (usually a sibling ACE session on this host) holds Android's single UiAutomation slot — the ui_dump_signal names every adb server attached to the serial. Kill the competing client, then retry the same dispatch.

This step is now ~zero code in the skill. The state-classifier lives in mcp/mobile/client.ts (post-restore verification only); no duplicate probe at the skill level. Earlier versions of this skill documented a local probe table here — that was the "tolerate any starting state" anti-pattern. Removed 0.13.202.

The recipe-error → failure-mode table further down (Step 5) is the second-line classifier for things the upstream restore couldn't catch (e.g. CCZ content issues, Maestro recipe drift, OPP_NAME collisions in long invite lists). It stays.

Step 2.6: Recipe-sanity pre-flight (static probe)

Before booting the AVD, run a static comparison between recipe expectations and live app + Connect state. This catches the failure classes the turmeric/20260515-0536 cycle surfaced one-at-a-time over 8 attempts (~80 min of wall-clock burn) — module-name == form-name, expected-module-not-in-app, opp-name-mismatch, tile-name-collision.

Helper: mcp/mobile/recipe-sanity-probe.ts (pure function — same inputs always produce the same verdict; no MCP calls inside the probe itself).

Inputs the caller assembles:

  • Smoke recipe text(s) — read from 3-commcare/recipes/journey-*.yaml on Drive
  • Nova app structures — one nova_get_app({app_id}) per app in app-test-cases.yaml (typically learn + deliver)
  • Form fields, per walked form — nova_get_form({app_id, moduleUuid, formUuid}) (Nova is uuid-addressed since 2026-07-31; the uuids come off the nova_get_app({app_id}) responses above, whose blueprint prints [uuid …] on every module, form, and field — see playbook/integrations/nova-integration.md § The 2026-07-31 uuid-addressing migration), mapped onto forms[].fields[] as {id, kind, label, hint, options, children, relevant} — include each field's relevant when Nova reports one; its PRESENCE on trailing labels is what arms score-gated-quiz-over-advance (ace#1118). Include each field's hint (flattened to its rendered text, the same way label is) — CommCare renders label -> hint -> EditText, so the hint is what a field-list input's focus anchor must name, and it is what arms input-anchor-skips-hint (ace#1554/#1299). OMIT the key for a field with no hint; never pass "" to mean "none" — the check reads only fields that positively carry a hint, so under-supplying degrades to a no-op rather than to a false "this field has no hint". Optional, but supplying fields is what makes the probe screen-shape aware: it turns on the label-screen carve-out (#858), the group-field-list-per-question-walk check (#862), the leading-label check (#1045), the score-gated over-advance check (#1118), and the content-form-finish overrun check (#2574). Omit it and all degrade to the old field-blind behaviour — which false-positives on any label-heavy Learn app.
  • Live Connect opp — connect_get_opportunity({org_slug, opportunity_id}) from run_state.yaml.phases.connect-setup.products
  • (optional) OPP_NAME the recipe was authored against — read from the recipe's env.OPP_NAME or from app-test-cases.yaml
  • (optional) Visible tile names — mobile_capture_ui_dump after a quick login. Skipping this skips only the tile-name-collision check; everything else still runs.

Failure classes the probe surfaces (each with a canonical remediation):

ClassRemediation
expected-module-not-in-appRecipe references a MODULE_NAME matching neither a module NOR a form anywhere in the supplied apps. Re-author via /ace:step app-test-cases — the live app structure has drifted. MODULE_NAME is the visible label of ANY suite row, module or form (learn-tap-module.yaml header; its Branch A is the tapped-a-form-row case), so a name matching a form is CORRECT and no longer flagged (ace#1649).
expected-form-not-in-moduleSame as above — module/form structure has drifted.
opp-name-mismatchPass OPP_NAME verbatim from run_state.yaml.phases.connect-setup.products.connect.opportunity.name (NOT slug-reassembled). Fallback only if missing: connect_get_opportunity({org_slug, opportunity_id}).name.
tile-name-collisionClean up prior-run invites OR use Resume-branch (exact-match claim).
answer-tap-before-leading-label-advanceRecipe taps an answer with FEWER bare form-advance steps behind it than the walked form's leading kind: label screens require. Each label node renders as its own screen with nothing to answer, so the answer selector isn't on screen yet and the tap fails selector-not-found — the Learn leg dies, learn_progress never reaches 100%, Deliver stays locked. Emit one bare form-advance per leading label between the menu-walk entry step (learn-tap-module / deliver-form-walk) and the first answer tap; re-author via /ace:step app-test-cases. This is the inverse of the form-advance-without-answer-tap carve-out below — that one bounds the count from ABOVE, this one from BELOW — and it is the static enforcement of the previously prose-only #710/#684 rule (ace#1045). Field-gated: inert without fields.
form-advance-without-answer-tapRecipe chains consecutive form-advance steps with no answer step between them — required-input questions will stall on warning_root. Re-author via /ace:step app-test-cases: for each required field, read its label/options via Nova get_form and emit a tapOn:text:"<literal>" (or inputText / photo-capture sequence) BEFORE the form-advance. Label-aware when fields is supplied: label screens have nothing to answer and can only be crossed by nav-next, so the threshold is (longest run of consecutive label screens) + 2. Without fields it stays at 2 and false-positives on label-heavy apps (#858).
score-gated-quiz-over-advanceRecipe chains a bare form-advance between the last required answer and form-submit on a score-gated quiz (trailing relevant-gated result labels, #569). form-submit.yaml performs the answer→result advance ITSELF, so the extra advance leaves it tapping nav_btn_next on the FINISH-only result screen. Remove the explicit advance (ungated trailing label screens each license one); re-author via /ace:step app-test-cases (ace#1118). Field-gated: inert without fields, and needs relevant on the trailing labels.
content-form-finish-overrunA form walked by content-form-finish.yaml (12 nav_btn_next slots) or content-form-finish-to-suite.yaml (14) has more screens left — after the bare form-advance steps in front of the call — than the recipe's unrolled loop can tap. The loop runs out, the terminal anchor assert fails, the form never finalizes, its learn_module never registers, Learn stays <100% and Deliver stays locked. Emit the excess as bare form-advance steps before the call, per skills/app-test-cases § Multi-screen content forms; re-author via /ace:step app-test-cases (ace#2574). Field-gated: inert without fields; counts a lower bound (skips hidden and relevant-gated fields).
group-field-list-per-question-walkRecipe advances the form BETWEEN two children of the same Nova group. A group compiles to a CommCare field-list — all children render on ONE scrollable screen — so the advance fires with required siblings still unanswered (warning_root) and reaches for options that may be off-screen. Re-author via /ace:step app-test-cases as a single-screen walk: answer every required child on the one screen, then exactly ONE trailing form-advance (#862).
input-anchor-skips-hintA field-list input focus tap (tapOn: below: + inputText) anchors on the QUESTION LABEL of a field that carries a hint. CommCare renders label -> hint -> EditText, so below: <question label> resolves to the hint TextView; tapping a TextView moves no focus, the tap reports success, and the value appends into whatever was focused before — silent data corruption, not a failed leg (cbf_name = "Thandiwe Banda0991234567" with a required phone_number empty, spark-facilitator/20260813-2126). Anchor on the hint instead, in BOTH the centring scroll and the below:; re-author via /ace:step app-test-cases (ace#1299 § 1, ace#1554). Hint-gated: reads only fields that positively carry a hint, so it is a no-op when hints were not supplied (observed.hint_data_supplied: false).
input-focus-scroll-is-guardedThe centring scroll onto a field-list input's focus anchor is wrapped in when: notVisible: <that same anchor>. The real failure is "anchor visible, its EditText still below the fold", which that guard is structurally blind to — so no scroll fires and the tap lands off-target. Per ace#1299 § 2 this is "the more important half of the bug" (all 14 inputs of that run, wrong-anchored or not). Make the scroll unconditional (speed: 30, centerElement: true); guarded scrolls stay correct only where the anchor IS the tap target, e.g. an option label (ace#1070). Needs no field data — the defect is pure recipe shape.
option-tap-guard-is-unscopedA global when: notVisible: "<option>" guard (and its scroll) in front of a below:-scoped option tap — the option-tap sibling of input-focus-scroll-is-guarded. A when: condition owns its own element selector and below strictly narrows the candidate set, so the guard asks "is there no No anywhere" while the tap needs "a No below MY question". On a field-list of identically-labelled Yes/No questions a bare option is essentially always on screen, the guard is a runtime no-op, the scroll never fires, and the tap dies as soon as its own options fall below the fold (poverty-graduation/20260915-1518: Element not found: Text matching regex: No, Below: ...MILK...; scoping all 13 pairs took the leg to pass). Give the guard AND its scrollUntilVisible the tap's own below: anchor — scope parity, not "always scope": a bare tap under a global guard is still correct (ace#1070). Re-author via /ace:step app-test-cases (ace#2426). Needs no field data — pure recipe shape.
input-without-eraseAn inputText step with no eraseText immediately before it. Maestro's inputText appends at the cursor — it does not replace — so a field already holding a Nova casedb preload, an XForm default, or a stray character receives the two values concatenated. Live: spark-facilitator/20260828-0703 typed 40 into hh_represented_at_the_meeting, submitted 140, and the form's cross-field constraint refused to advance; the leg then died two screens later on an unrelated scroll, reading as a selector fault it was not. Wherever it does not trip a constraint the corruption is silent and the leg reports pass on wrong data. Emit - eraseText before every - inputText (a redundant erase on an empty field is a runtime no-op); re-author via /ace:step app-test-cases <opp>/<run-id> (ace#1844). Needs no field data — the defect is pure recipe shape.
markdown-eaten-option-matcherA text: matcher matches an authored Nova option label but NOT what CommCare displays for it. Choice labels render as markdown, so a leading N. / N) / - / # / > is consumed: 25. At most 25 surveys a day are paid shows as At most 25 surveys a day are paid and the matcher dies selector-not-found on a healthy screen (smallholder-seasonal/20261012-1549; ace#1689, ace#2876). Re-compose the matcher from renderMarkdownLabel(label).rendered (lib/choice-label-integrity.ts) via /ace:step app-test-cases. Field-gated: needs each field's options in fields.
brief-label-driftRecipe has a tapOn:text:"X" matcher where X matches a PDD-brief naming pattern (^[LFM]\d+ — , ^Stage \d+ — ). Nova rewrites these during autobuild and the matcher won't resolve live. Re-author via /ace:step app-test-cases: read the live label from Nova get_form/get_module and use it verbatim.
deliver-smoke-rewalks-learnRe-author the Deliver smoke as resume-only (connect-resume-opp → deliver-launch.yaml) via /ace:step app-test-cases. The Learn leg already completes Learn.

Record which probe actually ran. Copy observed.field_data_supplied, observed.max_label_screen_run, observed.nova_groups_seen and observed.module_form_checks_ran into the verdict. Two caveats make a clean verdict weaker than it looks, and both must be reported as a WARN rather than an unqualified pass:

  • field_data_supplied: false — the screen-shape checks never ran, so the chain check was field-blind and both group-field-list-per-question-walk and answer-tap-before-leading-label-advance could not fire at all. Name the missing nova_get_form call.
  • hint_data_supplied: false — no supplied field carried a hint, so input-anchor-skips-hint never ran (ace#1554). Deliberately NOT a warnings[] entry: unlike the two below, a false here is ambiguous between "the caller omitted hints" and "no question in this app has one", and the latter is a common, legitimately clean state. Report it as a caveat only when the walked forms plausibly carry hints.
  • module_form_checks_ran: false — no recipe bound a readable MODULE_NAME, so expected-module-not-in-app and expected-form-not-in-module never ran (ace#1068). The probe now emits this itself as a module-form-checks-not-run entry in verdict.warnings[], naming the recipe.
  • module-name-equals-form-name — a warnings[] class, not a failure (ace#1649). MODULE_NAME == FORM_NAME makes CommCare render an intermediate one-row form list, which learn-tap-module.yaml Branch B and deliver-form-walk.yaml branch 2b cross on their own; recipe and palette ship in the same plugin version, so the un-handled state the old failure guarded is not reachable. No action unless the walk actually stalls on a one-row form list carrying the module's own name.

Copy verdict.warnings[] into the verdict YAML verbatim. Warnings never flip ok — they qualify a pass. A pass reported without them is the "configured vs configured correctly" gap: the probe returned ok: true having never performed the check you were relying on. The canonical Phase-3 fix for module-form-checks-not-run is to pass MODULE_NAME/FORM_NAME in the entry step's runFlow.env; for palette steps that take no env (deliver-form-walk), verify the names by hand against nova_get_app and record that in the verdict.

On any failure, halt with the incomplete-mode verdict shape (see Step 9), verdict: incomplete, and a per-class PLATFORM auto_surfaced entry naming the remediation. Do NOT proceed to Step 3 — every one of these classes is structurally guaranteed to produce a recipe-level failure 5-10 min later. Fail fast at the boundary.

Step 2.7: Learn-completion is one-way — never re-walk, never mutate to diagnose (#568)

Learn completion is permanent per (test user, opportunity). Once the test user has completed Learn on an opp, Connect routes "Continue Learning" to the Deliver download gate, not the Learn home — the Learn flow cannot be walked again on that opp+user. This is documented in docs/learnings/2026-05-18-connect-gates-deliver-on-learn-completion.md; this step makes it operational so the Learn-walk smoke does not halt on a consumed opp.

Two hard rules:

  1. A recipe-defect diagnosis must NOT mutate one-way live state. If the Learn smoke recipe is broken (e.g. a missing FINISH press), fix the recipe — do NOT manually complete Learn on the live opp to "confirm the fix." Completing Learn consumes the Learn-not-complete precondition permanently, and the only restore is a fresh opportunity (run independence). The bednet-spot-check/20260529-1124 cycle burned two Phase-6 attempts + a failed opp re-mint exactly this way.

  2. An already-Learn-complete opp is NOT a re-walkable state and NOT a blocker to "recover" by re-walking. The Learn leg (Step 5) must branch on it:

    • If, after the claim/resume prefix, the device is at the Deliver gate (Learn already complete — connect-claim-opp.yaml surfaces this via its already-Learn-complete branch, #570), record the Learn sub-verdict as satisfied-by-prior-completion (NOT fail, NOT incomplete — Learn genuinely completed, just on a prior pass) and proceed directly to the Deliver leg, which is unlocked.

    • If the device is on the Deliver home — Learn complete and the Deliver CCZ already installed, so there is no download gate left to land on (#863; the #747 already-installed fast path, or any resumed run that previously reached the Deliver leg) — record the same satisfied-by-prior-completion sub-verdict and proceed to the Deliver leg. connect-claim-opp.yaml surfaces this as the screenshot claim-already-learn-complete-deliver-app-installed, branched on deliver-home-daily-visits.

      This is a third recognized surface, not a variant of the second. It cannot be detected with nsv_home_screen/learn-home-screen, which is the generic StandardHomeActivity ScrollView and renders identically on both homes — that is exactly why this landing used to fall through the fresh-Learn branch and send the Learn walk into the Deliver suite, failing selector-not-found on a healthy opp. Do NOT use deliver-home-job-card/viewJobCard to tell the two homes apart: #893 disproved it (it renders on both once the opp is claimed).

    • Only attempt the actual Learn screenshot walk when the device lands on the Learn home (nsv_home_screen) and the Deliver-home anchor is absent — i.e. Learn is genuinely not yet complete.

If a fresh Learn-walk screenshot set is specifically required (e.g. the prior completion produced no usable captures), that needs a fresh opportunity — and therefore a fresh Deliver cc_app_id. Do NOT re-mint a Phase-4 opp on the same released Deliver app: Connect shares one DeliverUnit across opps on the same cc_app_id, so the fresh opp can't get a payment unit (#573).

Cheapest correct route — copy the apps, do not delete them. A full new /ace:run works but rebuilds everything, and the soft-delete route (commcare_delete_app → refresh_deployment → upload_app_to_hq) breaks the app links of whatever opportunity is currently live on those apps. Instead:

commcare_linked_app_copy({ upstream_domain: D, downstream_domain: D,   // SAME domain
                           upstream_app_id: <existing>, name: <UNIQUE>,
                           linked: false })
commcare_make_build  ->  commcare_release_build

then create a capture-only opportunity against the copies. Live-validated 2026-09-01 on turmeric-market-study/20260828-1108. Three reasons it wins:

  • Nothing is destroyed. The prior opp stays demoable and keeps its OCS TaskType, so Phase 5 does not regress.
  • It sidesteps ace#1643. A document copy preserves appearance="acquire" and per-module display_style, so the app-hq-settings re-apply is not needed. Verified in the built CCZ and on-device.
  • Fresh cc_app_id ⇒ new CommCareApp row ⇒ the posted passing_score is honoured rather than silently inherited, and the new DeliverUnits are unbound so the payment unit attaches.

Two traps, both live: the name must be UNIQUE in the domain (the new id is recovered by re-listing and matching on name), and a timeout on that recovery re-list does NOT mean the copy failed — re-list before retrying or you create a duplicate.

Name the capture opp so its run-id token is unique (see § OPP_RUN_ID matcher below) — if the superseded opp is still active and its name shares the token, the tile anchor matches two tiles.

This branch is the structural fix, and the claim-prefix landing screen IS the signal — the walk cannot mis-enter once all three surfaces above are recognized.

The landing-screen branches above are what keep the walk CORRECT. Step 2.8 below makes it CHEAP — it catches the unwalkable case before the AVD boots rather than ~10 minutes in.

Step 2.8: Pre-AVD consumed-precondition probe (#796)

Runs before mobile_ensure_avd_running. Read-only, headless, ~1s.

Both preconditions Phase 6 needs are one-way per (test user, opportunity), and a retry on this run reuses the same opp — so when both are spent there is nothing left to walk and the AVD boot (or the prewarmed lease) is pure waste. Connect can answer that server-side:

  1. Read domain and opportunity_id from phases.connect-setup.products.connect.opportunity in run_state.yaml.
  2. Call connect_get_learn_progress and connect_get_deliver_progress with them (grep docs/atom-schemas.md for the exact signatures).
  3. Classify with classifyOppConsumption from lib/opp-consumption.ts, passing the two workers arrays and the test user's display name.

Branch on the returned verdict:

verdictaction
freshproceed — both legs walkable
learn-consumedproceed — Deliver leg only; Step 2.7 will record satisfied-by-prior-completion
fully-consumedhalt before Step 3's boot — and if Step 0 prewarmed a lease, mobile_stop_avd to release it. verdict: incomplete, reason precondition-consumed, remediation = the classifier's reason string (which names the fresh-/ace:run restore and warns off an opp re-mint per #573)
worker-not-foundproceed — the roster could not answer; the on-device branches stay authoritative

If either atom errors, proceed. This is a cost-skip, not a correctness-skip (CLAUDE.md § attempt the transition, treat the conflict as the skip). It exists only to avoid a doomed boot; it must never be the thing that decides a walk doesn't happen. Every ambiguous read fails open by construction — see the fail-open cases in test/lib/opp-consumption.test.ts.

Step 3: Boot AVD + ensure apps installed

Call mobile_ensure_avd_running (blocking — no prewarm). It joins Step 0's prewarm on the lease, or cold-boots the local AVD, then restores the test-user precondition.

Do not call mobile_install_apk in the normal path. CommCare is already on the device when this returns: the local funnel installs the pinned ACE_CONNECT_APK_VERSION APK from its SHA256-validated cache when missing, and a lease machine boots with that version BAKED into its image (the state is chosen from the same variable). mobile_install_apk is only for deliberately testing a different build: on the lease it uploads the APK and adb install -rs it over the baked one, once per machine (a repeat with the same bytes is a no-op), and recipes then run against an APK the selector map was not calibrated for — so record that you did it.

mobile_ensure_avd_running is the single source of truth for "AVD is ready for Maestro" — since 0.13.165 it also probes the on-device Maestro driver's gRPC channel and auto-heals (force-stop + uninstall + reinstall of dev.mobile.maestro) if the driver isn't responding. Same code path is what mobile-bootstrap calls and what ace-doctor probes (read-only via mobile_probe_maestro_driver). One healing implementation, three callers.

If mobile_ensure_avd_running throws MaestroDriverError (the heal exhausted), halt with verdict: fail + severity: BLOCKER naming /ace:mobile-bootstrap as the operator recovery. Do NOT downgrade to verdict: incomplete — Maestro driver health is solvable on the workstation, not via a placeholder ship. Pre-0.13.165 the skill wrote incomplete here and Phase 6 shipped without screenshots; that escape valve hid real capability gaps behind a yellow verdict.

Step 3a: Invite the worker the device signed in as (ace#3228)

Skipped when Step 3b (per-run user) ran. Otherwise the device signed in as a pool test worker (ace#2955), which is often NOT ${ACE_E2E_PHONE} — the only phone Phase 4 invites. When worker-1 is held by another session the device is worker-2..N, and without an invite the claim recipe hunts a tile that cannot exist.

  1. Read the slot from the mobile_ensure_avd_running result: test_worker.signed_in names the slot; its phone is the matching test_worker.held[] row (held.find(h => h.slot === signed_in).phone).
  2. connect_list_flw_invites({organization_slug: <PM org>, opportunity_id, phone}).
  3. match === null → connect_send_flw_invite({organization_slug, opportunity_id, phone_numbers: [phone]}), then re-read. Still null → HALT verdict: fail, severity: BLOCKER (call mobile_stop_avd first). status: pending is normal — the claim recipe accepts it.

Recipes get the worker's PHONE/PIN auto-injected (withDeviceIdentity, mcp/mobile/client.ts), so do NOT pass PHONE_LOCAL/PIN to recipes. Record test_worker: {slot, phone, invited_in_step_3a: true|false} in the Step 6 manifest. Verified live on bednet-check-2-visit/20261018-0554 (worker-2, +74260000102).

Step 3b: Per-run demo test user — OFF BY DEFAULT (ace#1289)

Skip this entire step unless the switch is on. Read it once via perRunTestUserEnabled() (lib/per-run-test-user.ts; env var ACE_PER_RUN_TEST_USER, fails CLOSED on anything but an explicit true/1/yes/on). Off → Step 3 booted the AVD exactly as it always has, registering the fixed ${ACE_E2E_PHONE}, and Step 4 follows unchanged.

On, the device side of the invite → register → re-invite sequence runs here, in this order, before any recipe:

  1. Recover the run's minted phone from run_state.yaml.phases.connect-setup.products.connect.ace_test_user.phone (Phase 4 Step 7b wrote it, and it is REQUIRED at that boundary when per_run: true). Cross-check it against derivePerRunTestUser(run_state.yaml.run_id) — they must match; a mismatch means the run id changed under a fork and the recorded value wins, because that is the number Connect actually holds an invite for.

  2. Register it on the device. Pass the credentials to mobile_ensure_avd_running — it takes an OPTIONAL testUser override:

    mobile_ensure_avd_running({
      avdName: '${ACE_AVD_NAME}',
      testUser: { phone, phoneLocal, countryCode, name }   // pin + backupCode stay env-derived
    })
    

    No .env write, and therefore no Claude Code restart. That is the whole reason this is a call argument: every MCP server reads .env at module load, so an .env rewrite would need a full restart to take effect (CLAUDE.md § MCP changes need a full Claude restart). Registration already runs on every dispatch — the cold-boot funnel re-registers unconditionally, ~15-25s — so this changes WHICH number registers, not WHETHER one does.

    Honoured on both the local AVD and the lease backend — the lease registers from the client exactly as local does.

  3. Re-invite, then GATE on the read-back. The first invite (Phase 4) could not link an OpportunityAccess, because Connect creates that link only at invite time and only for a ConnectID user that already exists (opportunity/tasks.py). Now that the user exists, send it again:

    connect_send_flw_invite({ organization_slug, opportunity_id, phone_numbers: [phone] })
    connect_list_flw_invites({ organization_slug, opportunity_id, phone })
    

    Classify the result with classifyPerRunPostRegistrationGate(match) and honour halt: true — do not walk. It requires connect_user_id !== null: Connect's mobile opp-list endpoint filters opportunityaccess__user, so an access with a null user matches nothing and the opportunity is invisible on the device forever, without self-healing (connect-id 2bd03c4). Proceeding past this burns a full dispatch discovering zero tiles.

    Record registered_at (ISO) alongside the Phase-4 ace_test_user block.

Why this exists, and why it is off. A fresh user has no accepted invites, so the tile list is O(1) instead of O(list-that-grows-forever) — this is the durable end of the class #1475 and #1532 only bought headroom against (pruning provably cannot reach the accepted In Progress section, see Step 4). It stays off because the fresh-signup branch routes through the photo-capture surface whose 7 camera ids the selector map for the baseline in force records as "deliberately raw pending live calibration", inside a runFlow.when visible: guard that fails SILENTLY. The precondition to flip it on: the 7 camera ids in connect-register-from-otp.yaml are calibrated against a live mobile_capture_ui_dump on the APK baseline in force, and one fresh-signup registration has completed on that same baseline.

The sentence names no APK version deliberately (ace#1289): it said 2.63.2, DEFAULT_APK_VERSION moved to 2.64.0 on 2026-09-06, and every copy stayed behind. At the 2.64.0 baseline both clauses are still open — they are residuals R2 (fresh-signup branch unwalked) and R4 (the rebuilt camera surface unexamined) of docs/mobile-calibration/connect-2.64.0-2026-09-06.md.

Step 4: Run static prerequisite recipes

These set up the AVD to the post-claim state the smoke recipes assume:

  • connect-login.yaml with ${ACE_E2E_PHONE_LOCAL}, ${ACE_E2E_PIN}. (Only when Step 3b ran — ACE_PER_RUN_TEST_USER on, default OFF — substitute the run's minted phoneLocal for ${ACE_E2E_PHONE_LOCAL}; the PIN is unchanged, it is not a per-user secret in the demo range.)
  • connect-claim-opp.yaml with ${OPP_RUN_ID} (the deterministic tile matcher) read verbatim from run_state.yaml.run_id, AND ${OPP_NAME} read verbatim from run_state.yaml.phases.connect-setup.products.connect.opportunity.name (logging / screenshot context only).

OPP_RUN_ID matcher — the deterministic tile finder (#618). Phase 4 (connect-opp-setup) front-prefixes every opp name with the run-id: "<run_id> · <PDD display name>" (e.g. "20260601-0739 · Malaria RDT Performance Sampling"). The run-id (format YYYYMMDD-HHMM, the top-level run_state.yaml.run_id) is unique per run and, as a FRONT prefix, lands on the tile's first line — short and never clipped by the tile's name-wrap. The claim/resume recipes anchor their tile match on it as text: ".*${OPP_RUN_ID}.*" (Maestro text: is a full-match regex; the .* wrappers substring-match the run-id within the longer tile label). Normally you pass ${OPP_RUN_ID} verbatim from run_state.yaml.run_id. This is what makes finding this run's opp among the test user's accumulated invites deterministic.

Multi-role runs pass the opportunity's tile_token instead (products.connect.opportunity.tile_token, "<run_id> · <Role>" — lib/worker-roles.ts). There the test user is invited to one opportunity per worker role, every one named "<run_id> · <Role> · <display>", so .*<run_id>.* matches all of their tiles; the token adds the role, which follows the run id on the tile's first line. Record it as opp_run_id_matcher. Residual: the token's · (U+00B7) inside a Maestro text: regex has not yet been observed on a device — the name has always carried it, the matcher has not; a miss shows up as the claim recipe's tile-not-found, and the fix is the run id alone with the other roles' opportunities not yet invited.

But "unique per run" is an assumption about NAMES, not about run-ids, and it can be false. The anchor is a regex SUBSTRING match, so it matches EVERY still-visible tile whose name contains the token. Two ways that happens:

  1. A capture-only opp created for a re-shoot (see Step 2.7). If you name it with the same run-id as the run it belongs to, and the superseded opp is still active and still invited to the test user, .*<run_id>.* matches both tiles and the walk is non-deterministic. Live on turmeric-market-study/20260828-1108: opp d5959a5c stayed active carrying 20260828-1108, so the capture opp was named 20260901-1352 · … and OPP_RUN_ID=20260901-1352 was passed at dispatch instead of the run_id.
  2. A run-id that is a prefix of another. There is no such collision in the YYYYMMDD-HHMM format today, but a hand-authored or truncated token can reintroduce one.

So: OPP_RUN_ID is the token that uniquely identifies the tile you want, which is usually — not always — run_state.run_id. Whenever you point capture at an opp other than the run's primary one, pass that opp's own token and record it in the manifest as opp_run_id_matcher so the choice is auditable. Before dispatching, sanity-check that no OTHER active opp invited to the test user shares the token.

OPP_NAME — context only (no longer the matcher). Phase 4 also writes the exact tile name Connect renders into phases.connect-setup.products.connect.opportunity.name. Pass that string verbatim as ${OPP_NAME} for logging and screenshot context. Do NOT reassemble from slug pieces. It is no longer the recipe matcher — ${OPP_RUN_ID} (above) is.

Fallback (legacy runs only). If opportunity.name is missing from run_state.yaml (rare — only pre-0.13.x runs predating the name-field write), fall back to connect_get_opportunity({org_slug, opportunity_id}).name and log [WARN] OPP_NAME read from connect_get_opportunity fallback — current run_state.yaml lacks phases.connect-setup.products.connect.opportunity.name. Never compose from slug pieces.

This insulates Phase 6 from Connect's name-formatting changes — we read what Phase 4 wrote when it created the opp, not what the Connect UI happens to render today. Caught live on malaria-itn-fgd run 20260515-1645 Phase 6 attempt 9, where slug-based composition produced "ai-demo-space - malaria-itn-fgd (run 20260515-1645)" but the live tile text was "Malaria ITN FGD — malaria-itn-fgd (run 20260515-1645)"; the subagent self-corrected mid-dispatch by calling connect_get_opportunity, wasting a recipe cycle.

OPP_NAME uniqueness assumption (durable note) — resolved by the run-id FRONT prefix (#618). Because every /ace:run Phase 3 invites the same ACE test user (${ACE_E2E_PHONE}) to a fresh Connect opp, the test user's in-app invite list grows unboundedly across runs. Every prior opp the test user was ever invited to is still listed, and there is no per-run cleanup atom yet (TBD) — the list still grows.

What changed (#618): connect-opp-setup now front-prefixes the opp name with the run-id — "<run_id> · <display name>" (e.g. "20260601-0739 · Malaria RDT Performance Sampling") — and the claim/resume recipes anchor their match on the run-id token via text: ".*${OPP_RUN_ID}.*". This resolves the two fragilities the old run-id-SUFFIX form carried:

  1. Discriminator no longer clipped. The old name put the run-id at the END of a long name that wraps to 2-3 lines on the tile — the most-clipped position, so the only disambiguating token was the one most likely below the fold. The run-id is now a FRONT prefix on the tile's first line, which is never clipped by name-wrap.
  2. No reliance on "newest near top." The old tapOn:text matcher depended on Connect ordering by invited_at descending so the right tile sorted near the top. The recipes now scrollUntilVisible on the run-id regex and assert it, so the match is positional-order- independent: it finds this run's tile wherever it sits in the list.

PRUNE THE LIST BEFORE THE WALK (ace#1289). The run-id match makes finding the right tile deterministic, but it does not make it reachable: unclaimed opps render under a "New Opportunities" header that sits BELOW the entire "In Progress" section, so on the dogfood device the target tile is ~6 full viewport scrolls down and both tile-finding scrolls time out. A bigger scroll budget is not a fix — the list grows every run, so any fixed budget expires again.

So bound the list. Before Step 5:

  1. connect_list_invites / connect_list_flw_invites for the ACE test user (${ACE_E2E_PHONE});
  2. selectPrunableInvites from lib/invite-pruning.ts with currentOpportunityId set to THIS run's opp and testUserPhone set to the test user;
  3. connect_delete_unaccepted_flw_invites once per returned opportunity (the atom is per-opp — byOpportunity is already grouped for it).

currentOpportunityId is required and the helper throws without it. Pruning this run's own invite would break the very run doing the pruning — a far worse failure than the slow scroll. Accepted invites are excluded too (they represent real workers), and every exclusion comes back with a reason: log them, so "we left 3 alone" is a statement with evidence.

What pruning provably CANNOT reach — record this so nobody re-proposes it. lib/invite-pruning.ts:88-91 excludes status: 'accepted', so the pruner only ever shortens the New Opportunities section (~5 tiles of a ~25-tile steady-state list). The ~20-card In Progress section — which supplies most of the scroll depth — is structurally unprunable, and soft-deactivating prior opps does not remove tiles either (Connect's mobile opp-list endpoint has no active=True filter — see skills/sweep-connect). Bounding the list is a real but bounded win; it is not the class fix.

The class fix is Step 3b, and it is built but switched off. A per-run demo phone gives a user with zero accepted invites, so depth is O(1) by construction rather than O(a list nothing can shorten). It is gated on ACE_PER_RUN_TEST_USER (default OFF) pending one calibration — see Step 3b.

The recipes' own scroll budgets are already recalibrated and pinned — do not re-litigate them. PR #1475 put speed: 40 / timeout: 120000 / visibilityPercentage: 30 on the tile scroll in both connect-claim-opp.yaml and connect-resume-opp.yaml (live-proven on bednet-check-2-visit/20260814-0357), and test/mcp/mobile/static-recipe-invariants.test.ts § "tile-discovery scroll budgets stay in lockstep" fails if either recipe drifts off that shared constant. Each recipe additionally carries a fallback re-hunt after its primary tile scroll (claim-opp re-anchors on the home-new-opportunities section header first, making the second pass O(section) instead of O(list)); the same suite pins that too. The residual tracked on #1289 is the unbounded list itself, not the budget.

Step 4.9: Resolve each leg's previews folder (output previews contract v1)

A preview of an output lives in the folder of the phase that built the output, whoever captured it (ace-web docs/specs/2026-09-29-output-previews-design.md; helpers in lib/output-previews.ts, pinned by test/lib/output-previews.test.ts). Phase 3 built the Learn and Deliver apps, so this skill — although it runs in Phase 6 — writes each smoke leg's PNGs into Phase 3's folder:

<run>/3-commcare/previews/<app-output-slug>/_previews.yaml   # § Step 6.2
<run>/3-commcare/previews/<app-output-slug>/<NN>-<step>.png

Resolve, per leg, before anything is uploaded:

  1. The app — from the authoritative binding, never from a name. The leg's smoke journey carries app: learn | deliver in 3-commcare/app-test-cases.yaml (the field Step 2 already groups by). Every frame the leg uploads belongs to that app — including the frames of the connect-resume-opp dispatch that opens the Deliver leg. Do not infer it from a journey id, a recipe file name or a substring match.
  2. The output key — from THIS run's run_state.yaml, never assumed. Read phases.commcare-setup.products and call resolveAppOutputKey(products, app). The declared shape (lib/phase-products-schema.ts, sole writer app-deploy) is apps.learn / apps.deliver, but runs have drifted to apps.learn_app or a flat learn_app; the helper returns the key the run ACTUALLY carries, preferring the declared one. The index must name that exact key — it is how ace-web attaches the frames to the app. present: false (no app entry at all) → use the returned declared key and log a WARN in auto_surfaced; it means Phase 3's products.apps write-back is missing, which is Phase 3's defect, not a reason to skip previews.
  3. The folder. previewsFolderPath('3-commcare', key) → 3-commcare/previews/<slug> (apps.learn → apps-learn). Create it with drive_create_folder (find-or-create) under the run's 3-commcare folder: previews/ first, then <slug>/ inside it.
  4. Clear it — per leg, immediately before that leg's first upload. A re-capture REPLACES: drive_list_folder the leg's folder and drive_trash_file every child (PNGs and the old _previews.yaml), so two captures never mix. Clear only the folder of the leg you are about to run, and only then — never both folders at dispatch start. Learn completion is one-way per (test user, opportunity) (Step 2.7), so a Deliver-only re-run that wiped apps-learn/ would destroy previews nothing can recapture.
  5. Names. previewFileName(n, step) → <NN>-<step>.png, with n counting from 1 across ALL of the leg's dispatches in capture order (the Deliver leg's connect-resume-opp frames first, then journey-deliver's). The prefix keeps a name-ordered listing in capture order; the index is still what a reader follows.

One writer per output folder. Only this skill writes 3-commcare/previews/apps-*/. /ace:qa-deep's deep journeys, forensics, ui-dumps and videos do NOT go there (see Step 5's upload rule); and nothing about previews is ever written under phases.<phase>.products — ace-web reads any mapping there with a file_id as an output, so a pointer would show up as a bogus one.

Step 5: Run the smoke recipes — two independent legs

Palette-composition footguns (jjackson/ace#592) — read before composing recipes:

  1. mobile_run_recipe does NOT resolve ${SELECTOR:...} placeholders inside runFlow'd palette files. It resolves selectors in the top-level recipe body it's handed, but a palette piece pulled in via runFlow: runs verbatim — Maestro receives the literal ${SELECTOR:foo} token and (because ${...} is also Maestro's env-var syntax) resolves it to NaN (e.g. Assert that "NaN" is visible). When composing static palette pieces, either (a) run mobile_resolve_selectors on the composed body first, or (b) inline literal selectors. Do NOT assume nested runFlow inherits the parent's selector resolution.
  2. runFlow relative paths resolve against Maestro's temp chunk dir, not the recipe-file dir. runFlow: ../../../../mcp/... fails with Flow file does not exist: /var/folders/.../mcp/.... Always use absolute paths when composing palette pieces.
  3. Login credentials are easy to mis-split. COUNTRY_CODE is 7 (the dialing code), PHONE_LOCAL is 4260000101, and the PIN is the full ${ACE_E2E_PIN} (111111, 6 digits). The +7426 demo prefix in the docs is CC 7 + local starting 4260000101 — do NOT mis-split it as CC 7426.
  4. Mid-session re-login surface (tracked recipe gap). Between the Learn and Deliver legs the Connect session can drop to a signed-out-but-registered screen — screen_login_main with welcome_msg "Welcome ACE Test!" and a login_button "LOGIN WITH PERSONALID". connect-login.yaml today only branches on first-start (str_setup_message) and already-signed-in (connect_fragment_jobs_list); it halts on this intermediate screen. The branch (tap login_button → lockPassword → PIN) is documented but not yet live-validated — calibrate it the next time this surface is reached on-device, then add the branch + a static-recipe-invariants assertion. Until then, a re-login drop between legs is a known cause of a Deliver-leg halt.

HARD RULE — screenshots come ONLY from a status: pass execution in THIS run (jjackson/ace#756). If mobile_run_recipe returns status != pass for a required journey, that leg's screenshots DO NOT EXIST: record the leg fail with the recipe failure (failureClass, stderr, failed step) inline in the verdict, and NEVER source PNGs from the screenshot dir, a prior run, or any imagined "replay cache" — no such mechanism exists, and any file not in the returned screenshots[] of this dispatch is untraceable. The shallow verdict MUST NOT be pass if any required journey's recipe failed (per the Step 9 table). Halt loud per CLAUDE.md § Phase preconditions — don't ship placeholders. (Since #756 mobile_run_recipe also wipes the screenshot dir at execution start, so stale PNGs are structurally absent — but the rule stands regardless of what's on disk.)

Output dirs — pass ONE run-scoped ROOT; the MCP namespaces per dispatch (dimagi-internal/ace#1130). Pass the same screenshotDir: "$TMPDIR/ace-screenshots/<opp>/<run-id>" to every mobile_run_recipe call in the phase — both legs, every recipe.

Use $TMPDIR, not /tmp (ace#1456). /tmp is SHARED across macOS accounts: whichever account runs ACE first on a machine owns /tmp/ace-screenshots at mode 755, and every other account is then locked out of exactly the path this skill used to tell it to use — a bare EACCES: permission denied, mkdir mid-dispatch. $TMPDIR is per-user and correct on both single- and multi-user machines. Observed on bednet-check-2-visit/20260814-2019, where /tmp/ace-screenshots was owned by the other account on the box; it cost a dispatch cycle to diagnose. Same shared-/tmp family as ace#1046, which was the read-side sibling.

The path must be under $TMPDIR/ace-screenshots/ (or the legacy /tmp/ace-screenshots/, still accepted so an in-flight run's artifacts keep resolving, or ACE_SCREENSHOT_ROOT if the operator relocated it). mobile_run_recipe wipes its output dir at execution start, and before ace#1111 the only guard was a shape denylist — so any path with two or more segments (~/Documents, ~/.ssh, ~/Library) was destroyed irrecoverably by a single argument. It is now containment: anything outside the allow-listed root is refused before the wipe.

mobile_run_recipe writes each dispatch's artifacts into <root>/<recipeId>/ and the start-of-run wipe (#756) targets ONLY that subdirectory, so one leg can no longer destroy another leg's finished captures. It used to: on bednet-spot-check/20260731-1353 the Learn leg passed and produced a full screenshot set + video, then the Deliver dispatch — handed the same dir — wiped them (journey_learn: {count: 1, status: lost-not-recapturable}, UX judge verdict: incomplete). That loss is unrecoverable: Learn completion is one-way per (test user, opportunity) (#568/#570), so re-capture costs a whole fresh /ace:run. Two rules follow:

  • Read every artifact back from result.screenshotsDir / result.screenshots[].path — never by globbing the root you passed. The root now holds one subdirectory per recipe (journey-learn/, connect-resume-opp/, journey-deliver/, …), so a glob over the root spans journeys and re-opens exactly the confusion #756 closed.
  • Upload each dispatch's artifacts to Drive right after that dispatch returns, from its own returned paths. Don't batch uploads to the end of the phase off a directory listing. Two destinations, split by WHAT the file is, not by which recipe made it:
    • PNGs of a status: pass dispatch → the leg's previews folder from § Step 4.9, 3-commcare/previews/<app-output-slug>/<NN>-<step-name>.png (shareAnyoneWithLink: true, mimeType: image/png).
    • Everything else — each PNG's sibling <step-name>.xml ui-dump, the *-FAILURE.{png,xml} forensics of a failed dispatch, 00-postlearn-landing.xml — stays Phase 6 forensics in 6-qa-and-training/screenshots/<recipe-base>/. A preview folder holds previews only; a failure frame is never one.

Do NOT hand-roll per-leg roots to work around the old collision — the namespacing is structural now, and a per-leg root would only bury the per-recipe subdirs one level deeper.

Capture is split into a Learn leg and a Deliver leg. The legs are graded independently; a Deliver failure never suppresses Learn capture.

Learn leg (always runs first). Clear the Learn previews folder (§ Step 4.9 step 4), then run journey-learn.yaml against the AVD. Upload every captured screenshot to 3-commcare/previews/<learn-output-slug>/<NN>-<step-name>.png (normally apps-learn/; shareAnyoneWithLink: true, mimeType: image/png); upload any sibling <step-name>.xml ui-dump to 6-qa-and-training/screenshots/journey-learn/<step-name>.xml with mimeType: application/xml. Record the Learn leg outcome (pass iff the recipe status is pass AND every screenshot is non-zero bytes). A Learn failure records the Learn sub-verdict and does NOT abort the dispatch — but the Deliver leg then cannot run (Connect gates Deliver behind Learn completion), so it is recorded blocked-by-learn.

Learn-completion GATE — verify against Connect (authoritative), not the device screen (jjackson/ace#897). A green on-device recipe + a "Congratulations — you passed" assessment screen do NOT prove Learn is complete on Connect. Connect gates Deliver on learn_progress == 100% — i.e. a CompletedModule for every connect.learn_module marker — which is a separate signal from the assessment pass (commcare_connect/opportunity/models.py::learn_progress; form_receiver/processor.py::update_completed_learn_date sets the completion date only at 100.0%). A walk that finalizes the assessment but leaves one content form's module unregistered lands at e.g. 80% (4/5 modules), assessment Passed, Deliver locked — and the on-device recipe still reads "pass." The canonical incident: hh-poverty-targeting/20260722-1341, where the device said "passed / ready to deliver" but Connect showed 80% modules, and the Deliver-leg block got mis-diagnosed as a Connect platform failure (#897) and a recipe differentiator bug (#893). Neither was the cause — the Learn walk simply never drove Learn to 100%.

So after the Learn recipe passes, and BEFORE attempting the Deliver leg, call connect_get_learn_progress({ domain, opportunity_id }) and find the ACE test user's row. Assert learn_complete == true (equivalently modules_completed_pct >= 100 / completed_learning_date present). Branch on the result:

  • learn_complete == true → Learn genuinely completed on Connect; proceed to the Deliver leg (it is unlocked).
  • learn_complete == false → the Learn leg did NOT reach 100% on Connect regardless of the device screen. Record the Learn sub-verdict as incomplete-on-connect with the exact modules_completed_pct and assessment_status (e.g. "Connect: 80% modules (4/5), assessment Passed — the Learn walk did not finalize every module form"), and record the Deliver leg blocked-by-learn-incomplete (do NOT attempt it — Connect will keep the app on "Continue Learning" and the Deliver gate cannot open). Surface this as the precise, actionable failure (which specific module didn't register + re-run the walk to finalize it), NOT a Connect-platform or recipe-differentiator theory. This is the "close the loop to the source of truth" check — the device is not authoritative about learn completion; Connect is.
  • If connect_get_learn_progress is unavailable (older MCP not yet rebound) fall back to the PM UI (/opportunity/<id>/workers/learn/ — the Modules completed column) and apply the same 100% assertion; do not silently skip the gate.

The same gate applies to Deliver — and for the same reason (dimagi-internal/ace#1066). journey-deliver returning pass proves only that the form walked and finalized on the device. A plain Deliver form finalizes through form-submit.yaml's nav_btn_next auto-finalize branch, which writes to the local outbox; only the score-gated branch asserts anything about the server. Observed live on bednet-spot-check/20260729-1239: status: pass, then Daily Visits 0/5 and last synced: never — the visit had not reached Connect at all. An opportunity whose Deliver→Connect path was completely broken would still have produced a green Deliver leg.

So after the Deliver recipe passes, call connect_get_deliver_progress({ domain, opportunity_id }) and find the ACE test user's row.

Measure the walk elapsed FIRST — the gate below branches on it. Take it from this run's own frames: mobile_run_recipe returns screenshots[].takenAt (ISO 8601) on every capture, so

walk_elapsed_seconds =
  takenAt("journey-deliver-submitted")           # the post-submit frame
  − takenAt("deliver-form-walk-form-question")   # the FIRST Deliver form-question frame

Those are the two frames on the canonical run (hh-poverty-targeting/20260824-1404). Both labels are caller-bound, so resolve them per run rather than assuming these literals: deliver-form-walk.yaml captures deliver-form-walk-form-question${WALK_LABEL} (so the emitted name carries the leg's WALK_LABEL suffix), and form-submit.yaml captures the caller-supplied ${SCREENSHOT_NAME_POST_SUBMIT} that the per-run journey-deliver recipe binds. The rule is therefore: first Deliver form-question frame → post-submit frame, and record which two names you measured between.

Then read the opportunity's duration floor — the minimum visit duration Connect will accept for this deliver unit, in deliver_unit_checks[].duration_minutes (MINUTES; × 60 for duration_floor_seconds). There is no read-back atom for it, and you must not invent one: grep docs/atom-schemas.md — connect_set_verification_flags is the only atom that touches the field and it is write-only, and connect_get_opportunity reads the opportunity edit form merged with the dashboard, neither of which renders the verification-flags config. So read the value this run itself wrote in Phase 4: the cs-verification-flags row in the run's decisions.yaml, corroborated by 4-connect/connect-opp-setup_summary.md (connect-opp-setup § Step 5 is the sole writer, and it sets a duration only when the PDD states a minimum delivery duration). If neither records one, the opportunity has no duration floor: duration_floor_seconds = 0.

And establish the opportunity's start_date — with the read's own honesty flag, because an absent value is NOT a fact. The counts branch on it before anything else, so establishing it is a precondition of grading rather than a convenience. Two sources, in this order:

  1. Live — connect_get_opportunity({organization_slug, opportunity_id}). start_date comes only off the opportunity DASHBOARD (ace#1550), and every dashboard-only field degrades to undefined when that page does not render, so read dashboard_read FIRST. Per that atom's own contract (grep docs/atom-schemas.md rather than paraphrasing it), only dashboard_read: 'ok' licenses reading start_date's value — or its absence — as a fact about the opportunity. The other four statuses (setup_incomplete, no_cards, not_a_dashboard, not_fetched) mean the page did not answer. setup_incomplete is the one retrying can never fix. OBSERVED, not predicted (ace#1637, measured live on ai-demo-space 2026-09-06): Connect refuses to render a dashboard for an opportunity whose setup is unfinished — 11 of 11 such rows returned 302 → …/payment_units/create against 6 of 6 ok rows returning 200, and OpportunityDashboard.get redirects on is_setup_complete, which is paymentunit_set.exists() and total_budget and start_date and end_date. start_date is itself one of those four, so the field the gate needs is precisely the field whose absence suppresses the page that carries it.
  2. Stored — phases.connect-setup.products.connect.opportunity.start_date, which Phase 4 wrote at create time. Use it when the live read did not answer, and record that you fell back to it: it is a snapshot, and connect_update_opportunity can move the date after Phase 4 wrote it.

No new atom is needed either way. Compare the established date against the date the walk ran.

Now assert against the counts. start_date must be ESTABLISHED before any count is graded, and among the count branches the not-started branch is evaluated FIRST, because it is the one opportunity configuration under which no count can move:

  • start_date could not be established — the live read's dashboard_read was not 'ok' AND the run carries no stored Phase-4 value → record start-date-unreadable. This is not a pass, and it is not not-delivered-on-connect either: nothing has shown the Deliver→Connect path to be broken: the gate simply cannot tell which branch applies, because the one input that separates "Connect withheld credit by design" from "the visit never arrived" is unread. Record the dashboard_read status verbatim alongside the delivered/approved/rejected counts, and name the remedy: re-read connect_get_opportunity live, and if dashboard_read is setup_incomplete that opportunity renders no dashboard at all, so the stored Phase-4 value corroborated by 4-connect/connect-opp-setup_summary.md is the only source there will ever be. Do not default an unread start_date to "already started" and fall through to the hard assertion. A degraded dashboard read returns start_date: undefined, which compares as "not in the future" and silently skips the branch below — reproducing the exact ace#2187 misgrade (a working Deliver path graded not-delivered-on-connect) on the one input the branch was added to catch.

  • The opportunity has not started (start_date is in the future) and delivered == 0 → record not-started-as-designed: a PASS-with-note, NOT not-delivered-on-connect. Connect credits nothing before an opportunity's start date, so delivered >= 1 is structurally unreachable and no action available to Phase 6 can produce a delivery. The device says so itself — the Deliver home renders a red "The job has not started yet. You will not earn any progress for work." banner over a Daily Visits 0/N chip, and the sync recipe's terminal frame captures it. Record start_date, the run date, the delivered/approved/rejected counts and that frame as evidence; keep the leg pass, since the walk proved exactly what Phase 6 set out to prove. Note this is not the rejected shape — nothing was submitted-then-refused, so rejected == 0 and none of the branches below describes it. Do not relax the gate generally, and do not move the opportunity's start date to make the walk payable: that edits the programme's own contract to satisfy a test, which is the same error rejected-by-duration-floor-as-designed exists to prevent. Live: poverty-graduation/20260915-1518 — Deliver walk pass (43 screenshots, 0 failures), connect_get_deliver_progress {delivered: 0, approved: 0, rejected: 0}, connect_get_opportunity start_date: "2026-11-01" against a run date of 2026-09-16 (ace#2427).

  • delivered >= 1 — HARD, unconditional. The visit reached Connect. This is the real end-to-end proof of the Deliver→Connect path and the floor of the gate; a Deliver leg that cannot clear it is not-delivered-on-connect, never pass. It is reachable on every started opportunity, floor or no floor, so it is never conditional on the duration floor or on the counts; the sole configuration that can make it unreachable is the not-started branch above, which is read from the opportunity's own start_date rather than inferred from a zero count.

  • approved >= 1 — CONDITIONAL. A payment unit actually registered. It is strictly stronger than delivered (a delivery can be submitted and then rejected by verification), so assert it only when the walk could physically satisfy the opportunity's own verification rules — i.e. when duration_floor_seconds == 0 or walk_elapsed_seconds >= duration_floor_seconds. Asserting it unconditionally is structurally unreachable on any opportunity whose floor exceeds a machine-speed Maestro walk, and Connect is correct to reject such a visit (ace#1667). This is NOT a criterion app-test-cases.yaml declares — the sentence that used to claim so was false. A Deliver journey's structural_pass_criteria are device- and sync-level (app_boots, no_crash, submission_confirmed, and per-opp additions of the same kind such as daily_visits_counter_non_zero_after_sync); approved and the phrase "one payment unit registers" appear nowhere in the producer skill or in any emitted artifact. Verified — grep -rn -i "payment unit registers\|approved" skills/app-test-cases/SKILL.md test/fixtures/ACE-Test-001/3-commcare/app-test-cases.yaml → no matches (exit 1), and the same grep over the hh-poverty-targeting/20260824-1404 artifact matched only comments (ace#1667). So approved >= 1 is an extra payability check this gate adds, which is exactly why it may be skipped when the opportunity's own configuration makes it unreachable — no journey criterion is being waived.

  • rejected > 0 and approved == 0 and walk_elapsed_seconds < duration_floor_seconds → record rejected-by-duration-floor-as-designed: a PASS-with-note, NOT delivered-but-rejected. The verification wiring is working exactly as the PDD specified — an automated walk is faster than a human visit, so it lands under the floor and Connect rejects it by design. Record both numbers (walk_elapsed_seconds, duration_floor_seconds) plus the counts in the Deliver leg outcome, keep the leg pass, and do not file a finding, do not retry the walk, and above all do not relax or remove the duration floor — that floor is the PDD control this branch exists to protect. Corroborating evidence that the floor is evaluated rather than merely stored: hh-poverty-targeting/20260824-1404 walked in 287 s against a 360 s floor and Connect returned {delivered: 1, approved: 0, rejected: 1}.

  • rejected > 0 and approved == 0 with NO applicable duration floor (duration_floor_seconds == 0, or the walk already cleared it) → record delivered-but-rejected with the counts; the visit reached Connect and then failed verification for some other reason. That is a real finding about the opportunity's verification wiring, NOT a recipe defect — do not retry the walk to "fix" it.

The device-side deliver-sync.yaml gate (the server-derived Daily Visits counter) stays as the cheap in-recipe check, but it is still the device reporting; this atom is the authoritative read. Same lesson as jjackson/ace#897 on the Learn side: the device is not authoritative about completion — Connect is.

Deliver leg (runs second; depends on the Learn leg). Only attempt if the Learn leg reached completion AND the Connect learn-completion gate above passed (learn_complete == true). journey-deliver.yaml resumes from the now-unlocked state in the same device session (no re-login). Clear the Deliver previews folder (§ Step 4.9 step 4) before the leg's first upload, then upload its PNGs to 3-commcare/previews/<deliver-output-slug>/<NN>-<step-name>.png (normally apps-deliver/) and its ui-dumps to 6-qa-and-training/screenshots/journey-deliver/. Record the Deliver leg outcome independently.

FIRST, before any Deliver recipe — capture the post-Learn landing (unconditional, #618 ground truth). The instant the Learn leg completes, mobile_capture_ui_dump and upload it — into the Phase 6 forensics folder 6-qa-and-training/screenshots/, never a previews folder — to journey-deliver/00-postlearn-landing.xml (drive_upload_binary, mimeType: "application/xml", shareAnyoneWithLink: true; then § Step 5.8's readback — ace#1831). This is captured BEFORE the Deliver leg runs (and BEFORE any failure remediation) precisely because the device's post-Learn state is fragile — see the next rule. The dump tells you which post-Learn pathway you're on: nsv_home_screen → Path A (device on the CommCare Learn home, inside CommCare — the #618 landing, now HEALED in connect-resume-opp.yaml via the #711 nav-drawer return branch); connect_fragment_jobs_list → Path B (Connect jobs list). Both paths are handled by connect-resume-opp.yaml — the first recipe in the Deliver leg: its notVisible: home-jobs-list guard opens the nav drawer → taps "Opportunities" to return from Path A to the jobs list, then proceeds to the target tile's CTA; on Path B the guard no-ops and it goes straight to the CTA. So just run the Deliver leg normally — do NOT special-case Path A or divert to a calibration probe. The dump above is still captured unconditionally as #618 ground truth (and feeds atlas drift detection). Write the local copy into the run ROOT you pass as screenshotDir (see the output-dirs rule in Step 5) — the root itself is never wiped, and the wipe that does run is confined to the dispatching recipe's own <root>/<recipeId>/ (#1130). Belt and braces: that wipe is also selective (#1034) — 00-* and *-FAILURE.* files survive Deliver-leg retries, so this dump and prior-attempt forensics are NOT destroyed when the Deliver recipe re-runs.

NEVER cold-boot / re-bootstrap to "recover" a Deliver-leg failure. A Deliver landing on the CommCare home (nsv_home_screen) is the #618 nav gap — an expected, characterized condition, NOT a device-infra failure. mobile_ensure_avd_running (cold-boot) and /ace:mobile-bootstrap destroy the post-Learn state and reset the device PIN, which makes recovery impossible and wedges the rest of the run on a system auth prompt (observed live, run 20260605-2138: a cold-boot after the Deliver handoff failed reset the PIN → every subsequent recipe failed on com.android.systemui → the leg gave up after re-register attempts). On a Deliver-leg failure: do NOT escalate to a boot/bootstrap. Capture the dump (above), let connect-resume-opp.yaml run (it heals Path A via the nav-drawer return branch), and only if the recipe still genuinely fails, record the Deliver leg incomplete with the auto-captured *-FAILURE.{png,xml} forensics and STOP. Re-bootstrap is ONLY for a Learn-leg CommCareSetupActivity/PersonalID-wipe failure (the table row that names it), never for a post-Learn Deliver landing.

Do NOT halt the dispatch on a single leg failure. Run both legs (or record why the Deliver leg couldn't run), then write the per-app verdict in Step 9. The recipe-error → failure-mode table below is the per-leg classifier — apply it to whichever leg failed.

  • CRITICAL: the shareAnyoneWithLink: true flag is required. Slides' createImage (used by training-deck-render downstream) fetches PNGs via Google's image-import service, which doesn't carry the SA's auth — so an SA-only file gets "image cannot be reached" and the deck slide comes out blank. Setting anyone-with-link at upload time avoids a class of "deck builds without errors but slides are empty" bugs. The standalone drive_set_anyone_with_link atom exists for retroactively sharing a file that was uploaded without the flag. Verified live 2026-05-02 via scripts/test-screenshot-to-slides-e2e.ts.
  • NEW (0.13.229): each ScreenshotEntry may now carry a sibling uiDumpPath pointing at a <step-name>.xml file containing the Android uiautomator dump output captured at the same moment as the PNG. When present, upload it to ACE/<opp>/runs/<run-id>/6-qa-and-training/screenshots/<recipe-base>/<step-name>.xml (the Phase 6 forensics folder — NOT beside its PNG, which is a preview in 3-commcare/previews/<app-output-slug>/; the manifest row joins the two by step) via drive_upload_binary with mimeType: "application/xml" and shareAnyoneWithLink: true. The old rationale here — "not needed, XMLs aren't consumed by Slides" — asked the wrong question: Slides is one consumer, and the manifest publishes these as run artifacts a reviewer may be sent (ace#1831). Then run § Step 5.8's readback, which is what tells you an .xml in particular may come back as a warning page no amount of sharing fixes. The dumps are produced automatically by MaestroBackend.runRecipeWithDumps whenever the caller passes a serial to mobile_run_recipe (the default path for local-AVD invocations since 0.13.229). Absence is fine — pre-0.13.229 recipes have no XMLs and the upload step becomes a no-op. The dumps unlock atlas auto-maintenance (every Phase 6 dispatch leaves a complete record of resource-IDs at every surface the recipe visited) and selector drift detection (diff current dump vs prior passing run's dump). See docs/learnings/2026-05-14-atlas-side-channel-capture.md for the underlying problem and mcp/mobile/recipe-splitter.ts for the splitting logic.

#618 Path-A heal lives in the recipe — the calibration probe is retired. The Path-A return-nav (CommCare Learn home → Connect jobs list) was captured live and encoded as the guarded notVisible: home-jobs-list branch in connect-resume-opp.yaml (#711: open the nav drawer → tap "Opportunities"), followed by a pre-scroll jobs-list sync (#765: the just-completed opp's card can be absent from a stale list). So Path A no longer needs a separate probe-and-mark-incomplete step — running the Deliver leg exercises the heal directly, and the unconditional 00-postlearn-landing.xml dump above is the ground-truth capture. The old connect-learn-home-deliver-probe.yaml calibration recipe was removed (jjackson/ace#618).

If the recipe's Path-A branch ever fails on a fresh run, the auto-captured connect-resume-opp-FAILURE.{png,xml} forensics (plus 00-postlearn-landing.xml) are the characterization signal — re-calibrate the nav-drawer route against that live dump rather than re-introducing a probe (the "close the loop to the source of truth — one live dump beats a guess" rule). Because Learn completion is one-way per (test user, opp), the recipe's nav-drawer heal + #765 sync can only be validated end-to-end on a fresh /ace:run Phase 6 Deliver leg.

Before consulting this table, READ the failure screenshot. Two sources, in priority order:

  1. result.failureForensics (auto-captured, cross-backend — since 0.13.538). On every recipe status: 'fail', mobile_run_recipe captures the device state at the moment of failure — uiDumpPath (an <recipe-id>-FAILURE.xml element tree: resource-ids/text/bounds, the highest-signal artifact for selector + nav debugging) plus screenshotPath (<recipe-id>-FAILURE.png of the offending screen) — and returns them on the result. Both land in that dispatch's own output dir (<screenshotDir>/<recipeId>/, #1130), so they get uploaded + provenance-stamped alongside the smoke PNGs. This works on the lease backend too (Maestro's own debug-bundle screenshot is local-only). Read failureForensics first — the ui-dump usually shows the exact screen + the resource-ids present, which resolves "wrong selector" vs "wrong screen" immediately. On the local backend it also carries appCrash when CommCare crashed mid-recipe (ace#2584) — read that before the screen, because then the screen is not the cause (first table row).
  2. Maestro's debug-bundle screenshot (local AVD only). Written on every halt; path appears in the maestro.log Failed: line as screenshot-❌-<timestamp>-(<recipe>.yaml).png, under ~/.maestro/tests/<timestamp>/.

The image/dump often names the failure mode literally — "Logged out of PersonalID", "Enter Code", "Failed to start learning", "App not found" — making package / process / driver probing redundant. Image-read first, infer second. Skipping this step produced an inverted-conclusion bug live in 2026-05-13 (turmeric run 20260513-0616): the agent saw org.commcare.dalvik absent of a connect-named sibling package and concluded "Connect not installed" — when the failure screenshot showed "Logged out of PersonalID — Lost PersonalID configuration with server, please recover your PersonalID account and retry" with a Reconfigure → CTA in the nav drawer. Reading the screenshot would have produced the right answer in one step.

Recognized failure modes (surface a [BLOCKER] in auto_surfaced naming the specific failure + remediation rather than a generic "smoke recipe failed" message):

Recipe error containsFailure mode + root causeRemediation
Check this row FIRST, whatever the error string says: result.failureForensics.appCrash is present (local backend; the crash block is in failureForensics.crashLogPath, <recipe-id>-FAILURE-crash.txt)CommCare crashed during the recipe. org.commcare.dalvik threw an uncaught exception after the recipe started (a new FATAL block in logcat -b crash; system-app crashes and crashes from before the recipe are excluded). CommCare restarted onto whatever screen it came back on, so the failure screen, the ui-dump and any recipe label (e.g. claim-START-HANDOFF-WEDGED-issue629) are consequences, not the cause — do not read them as a selector, nav or #629 handoff defect. Live case: spark-facilitator/20261001-2208, IllegalArgumentException at ConnectJobIntroFragment.java:161 ~2s after the Start tap, reported as an nsv_home_screen assert failure (ace#2584).Quote appCrash in the verdict and attach the crash file. Check whether the server side completed anyway (connect_list_flw_invites, connect_get_learn_progress); if so, a plain re-dispatch of the leg is the recovery — no cold-boot needed. The crash itself is CommCare's (dimagi/commcare-android): if the same frame recurs, report it upstream with the crash file rather than adapting the recipe around it.
Failed to start learningReleased Learn CCZ has Nova <module xmlns="…connect…"> + <assessment xmlns="…connect…"> wrappers that the AVD's CommCare runtime can't launch. Confirmed live 2026-05-07 against leep-paint-collection: turmeric Learn (working) has 0 wrapper refs; LEEP Learn (broken) has 16. Tracking: voidcraft-labs/nova-plugin#7, jjackson/ace#115 finding 1.As of 0.13.66 Phase 3's Step 2.8 invokes commcare-form-patch automatically — re-run /ace:run <opp> and Phase 6 should pick up the patched Learn release. For an in-flight opp that already shipped Phase 3 without the patch: /ace:step commcare-form-patch <opp> then re-run Phase 6. Diagnostic probe: node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/probe-connect-learn-handoff.ts" <opp_uuid> (define ACE_ROOT per the Step 6.5 block) + adb logcat.
deviceInfo … UNAVAILABLE / MaestroDriverErrorMaestro driver app on the AVD is installed but its gRPC server isn't responding. Symptom of a wedged driver process or a stale install whose runtime state diverged from the CLI's expectations. Reproduced live 2026-05-11 against leep run 20260511-0507 Phase 6 (port 7001 refusing connections, every recipe stalling on deviceInfo).Since 0.13.165 mobile_ensure_avd_running auto-heals (force-stop + uninstall both halves of dev.mobile.maestro, then re-probe to trigger the CLI's auto-reinstall). If this error still surfaces, the heal exhausted — run /ace:mobile-bootstrap to re-baseline the AVD + driver, then /ace:step app-screenshot-capture <opp>/<run-id>. Logcat probe: `adb -s logcat
extendedWaitUntil timeout on connect_fragment_jobs_listClaim flow didn't reach jobs list. LLO program-application not ACCEPTED, or Connect session expired on the AVD.Re-run connect-login.yaml and verify connect_get_opportunity returns the expected opp.
assertVisible(text: ${OPP_NAME}) failure AND focused activity is CommCareSetupActivity AND/OR failure screenshot shows "Logged out of PersonalID" / "Lost PersonalID configuration" / "Reconfigure" / "Enter Code" / "Welcome to CommCare"AVD's per-user state was wiped — no ApplicationDocument configured (CCHQ app never pulled OR app db was wiped) and/or PersonalID account de-registered from the device. Important: org.commcare.dalvik IS the Connect-enabled CommCare client (no separate Connect package); presence of org.commcare.dalvik in pm list packages does NOT imply a usable Connect home. Should have been caught by Step 2.5; if it surfaces here, Step 2.5's probe missed a signal worth adding. Live in 2026-05-13 (turmeric run 20260513-0616) with both states stacked: setup activity foregrounded + PersonalID drawer banner./ace:mobile-bootstrap — re-installs APK if needed, registers the ACE test user (${ACE_E2E_PHONE}) via the Connect registration API, pulls the CCHQ app for this run's HQ ids, saves a clean snapshot. After bootstrap returns clean: /ace:step app-screenshot-capture <opp>/<run-id>. If this state recurs after a successful bootstrap, the issue is state-loss between sessions (snapshot revert, AVD cold-boot, server-side PersonalID de-registration); file a class-level issue on mobile_ensure_avd_running's state-persistence contract rather than re-bootstrapping.
assertVisible(text: ${OPP_NAME}) failure AND failure screenshot shows the Connect "New Opportunities" / opp list home (the user IS logged in, app IS configured, just no matching tile)Right opp card not on screen for the OPP_NAME being matched. Wrong OPP_NAME env var (typically slug-reassembled instead of read verbatim from run_state.yaml), OR opp not yet claimed by the test user, OR ${OPP_NAME} collides ambiguously with another invite further up the test user's accumulated invite list (see Step 4 "OPP_NAME uniqueness assumption").Confirm OPP_NAME was read verbatim from run_state.yaml.phases.connect-setup.products.connect.opportunity.name (NOT composed from slug pieces — see Step 4 "OPP_NAME source" and #115 finding 4). If the field is present and still mismatches the tile, scroll the invite list or use the connect_get_opportunity fallback.
assertVisible(nsv_home_screen) failure AND a claim-START-HANDOFF-WEDGED-issue629 screenshot is present (Start was tapped on the opp-detail Job Card; neither the Learn home nor the Deliver download gate ever rendered within 180s)Inert btn_start handoff — Connect's POST /users/start_learn_app/ never fired or 500-ed, so the Learn CCZ download never began and nsv_home_screen never appeared. This is the Connect-platform half of #629, downstream of ACE — the released CCZ already passed Phase 3's commcare-cli play install gate, so it is NOT a Nova/recipe/marker defect. connect-claim-opp.yaml now captures the labeled artifact + fails loud at this exact point instead of a generic 180s timeout.Run connect_preflight_learn_app_user({org_slug, opportunity_id}) to classify: an auth/domain/conflict failure there confirms the server-side start_learn_app block (fix Connect-side / re-invite the test user); a clean preflight points at a device-side wedge (re-run after mobile_ensure_avd_running cold-boot). Do NOT mark the Learn leg pass on placeholder screenshots; record the wedge verdict and surface #629.
Deliver leg: deliver-home-job-card wait/assert fails AND a deliver-launch-download-server-error screenshot is present (the recipe reached + tapped the Deliver download gate, then the failure surface is the PersonalID re-login screen — connect_login_button "LOGIN WITH PERSONALID" / "Couldn't connect to the server" / welcome_msg "Welcome <user>!")Transient server-connectivity Deliver-CCZ download failure — the async CCZ download could not reach the server, so the app de-authed to PersonalID re-login. Downstream of ACE: the released CCZ already passed Phase 3's commcare-cli play install gate, so it is NOT a Nova/recipe/marker/selector defect — the recipe navigated correctly to the gate. deliver-launch.yaml now captures this labeled forensic (#772) instead of a bare 180s timeout. Surfaced live on bednet-spot-check run 20260612-2347.Transient — re-attempt, do NOT cold-boot. Record the Deliver leg incomplete (transient), not fail. Re-run the Deliver smoke via /ace:qa-deep <opp> once backend connectivity is stable — that (app-ux-eval), not Phase 6, is the actual Phase-9 llo-launch quality gate. Do NOT mobile_ensure_avd_running cold-boot / /ace:mobile-bootstrap — the post-Learn state is fragile and a boot resets the device PIN (#618/#629). If the drop persists across retries, the backend (CommCare HQ / Connect CCZ-serve) is genuinely unreachable — escalate that, not the recipe.
Two capture steps in one journey hash byte-identical (surfaced by the Step 5.5 distinctness check, not a recipe error string)Recipe screenshots before/after a no-op transition, or the transition screen doesn't exist on this opp (e.g. Connect hands off directly into an installed app, so the intermediate screen never renders). Live instances: hh-poverty-targeting/20260702-1456 connect-resume-opp-tile ≡ connect-resume-opp-list-synced, journey-deliver-submitted ≡ journey-deliver-final (dimagi-internal/ace#866).Adjust the recipe to capture a genuinely distinct surface (e.g. the job-detail/View-Info screen) or drop the step; downstream consumers must treat duplicate_of steps as one moment.

Manual-debug fallback — when the table doesn't resolve it. If a recipe error matches NO row above (a novel nav defect, an uncharacterized inter-leg handoff like #618, a selector that resolved on a sibling APK but not this one), do NOT halt blind. Open failureForensics.screenshotPath and read failureForensics.uiDumpPath before writing the verdict:

  1. The .png tells you which screen the device died on (Learn home? Connect job card? a dialog?).
  2. The -FAILURE.xml ui-dump lists the resource-ids/text actually present on that screen — diff that against the selector the recipe reached for. "Selector absent from the dump" → wrong screen or APK selector drift; "selector present but tap was a no-op" → a childOf/scoping or timing issue.
  3. If that names a concrete recipe/selector fix, gh issue create against jjackson/ace with the dump excerpt + the screen it characterizes (the "close the loop to the source of truth" rule — one live dump beats another plausible guess), then encode the fix only after a live mobile_capture_ui_dump → candidate-tap → re-dump confirms it navigates. Until confirmed live, halt loud with the characterization in the verdict — never ship an unvalidated nav guess (the #618 / #591 selector-drift class).

Step 5.5: Distinctness check — byte-identical frames are ONE moment (dimagi-internal/ace#866)

After each journey's captures land (and BEFORE assembling the Step 6 manifest), compute a content hash over every PNG in the journey:

# One dir per dispatch (#1130) — hash the dispatch's own output dir,
# i.e. the `screenshotsDir` it returned.
md5 -q <screenshotDir>/<recipe-base>/*.png    # or: shasum <...>/*.png

Any byte-identical pair within a journey means the two "steps" observed the SAME frame — the recipe screenshotted before/after a no-op transition, or the transition screen doesn't exist on this opp (e.g. Connect hands off directly into an installed app, so the intermediate Connect screen never renders). Handling:

Do not hand-apply the rule — call assignCanonicalDuplicates from lib/capture-manifest.ts. Pass every frame as {step, md5, takenAt}; it returns the same frames with duplicate_of filled in. Then:

  • Mark each later duplicate in the manifest with duplicate_of: <canonical-step-name> instead of listing it as a distinct capture.
  • Log every duplicate pair in the structural verdict's auto_surfaced list (severity: WARN, naming both step names + the shared hash).

The rule it applies is recipe order — the first frame to observe a state owns it — except that an auto-named frame always yields to a meaningfully-named twin, however much earlier it was taken. Maestro names an unnamed chunk boundary step-<index>-<command>-<args>, and those land in the harvest alongside real takeScreenshot output. On turmeric-market-study/20260828-1108 step-010-assertCondition-org.commcare.dalvikid_vi was byte-identical to deliver-launch-download-gate and led it by 0.28s, so first-in-recipe-order alone made the opaque string canonical — and downstream prose cites the canonical step's NAME, so the training deck would have captioned a slide with it. That exception is exactly the kind of thing that has to be re-derived by hand every run if it lives only in prose here; it was missed once and caught on re-read. It is now pinned by test/lib/capture-manifest.test.ts.

Never present two identical frames as different moments. Downstream consumers (training-deck-generate, training-flw-guide) treat a duplicate_of step as the same moment as its canonical step — never as a distinct panel or slide. Surfaced live on hh-poverty-targeting/20260702-1456, where connect-resume-opp-tile ≡ connect-resume-opp-list-synced and journey-deliver-submitted ≡ journey-deliver-final shipped as distinct steps and the training deck then claimed distinct moments over duplicate frames (dimagi-internal/ace#866).

Step 5.6: Pixel review — OPEN the claim-bearing frames and record what they show

Every check up to here treats a screenshot as an id. The manifest proves a frame exists, is distinct, and resolves. None of that says what is IN the picture, and a caption can only ever contradict the picture.

Measured on turmeric-market-study/20260828-1108: an FLW guide and a 50-slide deck passed schema validation, 100% file_id resolution, zero duplicate citations, visual coverage 1.00 and 49 inline images verified against an anonymous reader — and two of the first four frames anyone opened did not show what the prose said. journey-learn-posttest-result was captioned "your certification result" in both artifacts and is actually the lesson menu with a transient "1 form sent to server!" toast: no score anywhere on it. The recipe had submitted straight past the pass/fail label, so no frame of the score existed at all — while the released CCZ proved the app renders one. Right claim, wrong picture, every gate green.

So: open the frames, and write down what you see.

You do not have to open all of them. Open, at minimum:

  • every terminal / outcome frame — results, confirmations, "you passed", post-submit landings. Prose wants to assert over these most, and they are the most likely to actually be a menu with a toast.
  • every frame a downstream artifact will make a negative or design claim about ("there is no gallery option", "the cap is shown here").
  • the first screen a worker sees (job card / opp detail). It renders operator-authored text — the opportunity description is worker-visible — so run scaffolding leaks onto it. On the run above the job card read "…a capture vehicle for Phase 6 walkthrough screenshots", and both artifacts cited that frame.

For each opened frame add a one-line shows: to its manifest entry — what is actually on screen, not the step name restated:

  - step: journey-learn-posttest-result
    file_id: …
    shows: "Post-test lesson tile with a '1 form sent to server!' toast. NO score."

shows is the only field in the manifest that is evidence rather than bookkeeping, and consumers gate on it: framesCitedWithoutShows (lib/capture-manifest.ts) returns every step an artifact cites without one. A producer either records a shows or drops the claim.

If a frame you need does not exist, say so and fix the recipe — do not promote a nearby frame into the role. The fix for the case above is a takeScreenshot between the last answer tap and FINISH in journey-learn.yaml.

Step 5.7: Upload session videos + sweep the device-video spool

Every mobile_run_recipe call records an mp4 of the run (on-device screenrecord), on BOTH backends: locally since 0.13.700 (around each attempt), and on the lease via the image's ace-mobile record start|stop around the whole chunked walk — downloaded into the same <screenshotsDir>/<recipeId>.mp4, returned in the same result.videos[], spooled the same way. A lease machine whose CLI predates record walks unrecorded (logged once) — that is the "absent is not a failure" case below. See playbook/integrations/mobile-integration.md § Recording.

The client spools EVERY video it records — including the ones it also returns in result.videos[]. So the spool and result.videos[] OVERLAP. Upload each video exactly once, in this order:

5.7a — This dispatch's journey videos. Take RecipeRunResult.videos[] from each Step 5 leg. Upload each to ACE/<opp>/runs/<run-id>/6-qa-and-training/videos/<recipe-base>.mp4 via drive_upload_binary with mimeType: "video/mp4" and shareAnyoneWithLink: true (so the run-summary page can embed it without an auth round-trip). A recipe that needed a driver heal contributes two entries — <recipe-base>.mp4 (pre-crash) and <recipe-base>-attempt2.mp4; upload both.

While uploading, build a set of the recipeId + attempt pairs you uploaded. You need it for 5.7b.

The spool is SESSION-scoped (ppid), not run-scoped (ace#1084). If two capture skills run in one Claude Code session, whichever sweeps first files the other's heal/registration clips under its OWN Drive prefix — the clips are not lost, but they are filed against the wrong leg. Sweep as late as you can, and if both legs ran in one session, check the <recipeId> in each spool filename against the leg you are uploading for rather than assuming everything in the spool is yours.

5.7b — The REMAINING spool entries only. Call mobile_list_session_videos (no arguments — the atom resolves this session's spool itself; never hand-resolve ~/.ace/mobile-videos/<ppid>/, and never glob mobile-videos/*/, which would read a CONCURRENT session's spool). It returns { spoolDir, videos: [<absolute path>, …] }.

Spool filenames are <epoch-ms>-<recipeId>[-attemptN].mp4. For each returned path:

  • Parse recipeId (everything after the first -, minus any -attemptN suffix and the .mp4) and attempt (N from -attemptN, else 1).
  • SKIP it if that recipeId + attempt pair is already in the set you built in 5.7a. It is the same recording; uploading it again doubles the run's video footprint in Drive for zero information.
  • Known limit of that key (ace#1084): it is recipeId + attempt, not file identity. If one leg is re-run in the same session — manual debugging, typically — the spool holds two distinct-epoch files for that pair and BOTH get skipped, dropping one forensic clip. When you have deliberately re-run a leg, compare the <epoch-ms> prefixes and upload the one 5.7a did not already cover.
  • Otherwise upload it to 6-qa-and-training/videos/_device/<filename> (same mimeType, and shareAnyoneWithLink: true — "forensic, not presentational" was the wrong test: the manifest publishes these as run artifacts and a reviewer can be handed one, ace#1831).

What legitimately survives that filter is exactly what 5.7a can't see: the heal and registration recipes that run inside mobile_ensure_avd_running, and the static prerequisite recipes from Step 4 — recipes whose callers are atoms, not uploading skills.

5.7c — Clear the spool. Call mobile_clear_session_videos (again no arguments; it is scoped to this session's ppid by construction and cannot touch another session's spool). It returns { spoolDir, cleared } — log cleared.

videos[] absent or the spool empty is not a failure: recording is best-effort by contract, and ACE_MOBILE_RECORD=off disables it entirely. Log the count and continue — never halt a dispatch over a missing video.

Step 5.8: Read every upload back anonymously — by BYTES, never by 200

An upload that returns a fileId proves the SERVICE ACCOUNT wrote something. It says nothing about what a reviewer receives. Before Step 6 writes the manifest, fetch each file this dispatch uploaded with no credentials and judge the response:

curl -sSL -D - -o body.bin --max-time 60 \
  "https://drive.google.com/uc?export=download&id=<fileId>"

Pass the file name, the status, the Content-Type and the first bytes to classifyUploadReadback (lib/upload-readback.ts). Four verdicts, three of them failures, and they do NOT share a remedy:

verdictwhat the reviewer getsremedy
okthe artifactnone
gateda sign-in page, or 401/403drive_set_anyone_with_link({fileId}), then re-read
interstitiala Google warning page at HTTP 200sharing will NOT help — see below
wrong-typebytes that are not the file's typere-upload; a truncated or empty write

Record the verdict per artifact in the Step 9 per_item[].note, and do not list a failing artifact in the manifest as if a reader could open it. A readback failure is not a reason to halt the dispatch — the walk already happened and its evidence is real — but it IS a reason the manifest must not claim otherwise.

Why bytes and not 200. Measured anonymously on hh-poverty-targeting/20260828-0702, 2026-09-06 — all six artifacts answered 200, and two of them were unreadable:

journey-learn.mp4            200 video/mp4               ftypisom               OK
journey-deliver.mp4          200 video/mp4               ftypisom               OK
journey-deliver-FAILURE.png  200 image/png               PNG                    OK
atlas-report.yaml            200 application/octet-stream "version:"            OK
journey-deliver-FAILURE.xml  200 text/html               "Google Drive - Virus scan warning"
00-postlearn-landing.xml     200 text/html               "Google Drive - Virus scan warning"

A 200/HEAD check — the one ace#1831 originally proposed — certifies all six.

The .xml case, and why shareAnyoneWithLink is not its fix. Those two files answer 200 to an anonymous fetch, so they are already reachable; the warning page is what that reachability gets you. Sharing them again changes nothing (sharingWouldNotHelp: true). What works: upload the same dump under a .txt name alongside it, or leave the .xml out of the reviewer-facing link set and keep it as an internal forensic. Do NOT loop on drive_set_anyone_with_link — that is the wrong remedy applied to the right symptom, and it reads as progress while changing nothing.

The auditor half of this class is already live: classifyLink in lib/run-surface-audit.ts returns INTERSTITIAL for the same shape (ace#1868 / PR #2040). This step is the PRODUCER half, and it exists because the auditor only probes links the run-summary payload emits — these artifacts live in the capture manifest, so no audit reaches them.

Step 6: Write 6-qa-and-training/app-screenshot-capture_manifest.yaml

Compose the manifest to a LOCAL FILE first, then write it with drive_upload_binary({name, mimeType: 'text/yaml', parentFolderId, localFilePath}) — do not emit it inline (ace#1918). The server reads the bytes off disk, so the write costs ~zero context regardless of size. This manifest is one row per captured step across every journey and was measured at 43,778 chars in the 2026-09-02 corpus (four runs re-sampled 2026-09-06: 6k–19k, so it scales with journey count).

Not drive_create_file (ace#2490). That atom always creates a native Google Doc, and a Doc export of this manifest comes back with every \n as \r\n\r\n\r\n — the canonical example in skills/_training-template.md § Machine-parsed artifacts must not be written as Google Docs is this very file. Every consumer parses it, so it must be stored as real bytes. The same rule covers the two Step 9 verdict YAMLs.

Link each captured PNG back to (a) its journey id (a meaningful slug like journey-learn-pass / journey-deliver-submit from app-test-cases.yaml), (b) its takeScreenshot: step label, (c) its Drive path. This is the input shape the per-artifact training skills (training-flw-guide, training-deck-generate) consume. Steps flagged by the Step 5.5 distinctness check carry duplicate_of: <first-step-name> instead of being listed as distinct captures.

Write the container and key EXACTLY as below — the consumers key off both names. Until ace#2224 this step specified the manifest's content and never its container, and the only worked example spelled the key step_name; a structurally reasonable manifest (screenshots: at the top level, keyed by step_name) then satisfied NEITHER consumer, and verify_caption_backing returned unknown-id for all 9 cited frames on bednet-check-2-visit/20260907-1126 until it was rewritten. Write:

captures:
  - journey_id: journey-learn-pass          # slug from app-test-cases.yaml
    step: journey-learn-posttest-result     # the takeScreenshot: label
    drive_path: 3-commcare/previews/apps-learn/07-journey-learn-posttest-result.png
    file_id: <drive fileId>
    md5: <content hash from Step 5.5>
    shows: "…"                              # Step 5.6, on every frame you opened
  - journey_id: journey-learn-pass
    step: journey-learn-final
    file_id: <drive fileId>
    duplicate_of: journey-learn-posttest-result   # Step 5.5 alias — NOT a distinct moment

One flat captures: list, one row per captured step across every journey, with journey_id on the row. The journey-grouped variants (journeys[].steps[], journeys[].screenshots[] + journeys[].duplicates[]), a top-level screenshots[], and step_name as an alias for step are all still READ — collectCaptureEntries in lib/capture-manifest.ts is the one reader every consumer goes through — but write the shape above so there is one thing to read. Any other container name is invisible to every consumer, and a consumer that finds nothing reports clean, not empty.

Then assert the manifest reads back, before you write it (ace#2236). This is a required step, not advice — the prose above has been correct since ace#2224 and the very next run wrote 60 frames under a top-level screenshots: anyway (spark-facilitator/20260907-1120), where collectCaptureEntries saw zero. Nothing reported it, because the asymmetry here is vicious: a malformed manifest costs nothing at write time and produces reassuring output downstream. Run this over the object you are about to serialise:

import { assertManifestReadable } from '../../lib/capture-manifest';
const readback = assertManifestReadable(manifest, { expectedCount: framesCaptured });

ok: false is a halt, not a warning. Two reasons:

reasonmeaningfix
unread-containerpath names a container holding capture-shaped rows that no consumer readsmove those rows into captures: — do NOT ask for the reader to be widened again
count-mismatchthe reader sees a different number of frames than this dispatch capturedfind the rows that went missing; every consumer sees the reader's number

Rows the reader deliberately ignores — journeys[].superseded_artifacts[] (forensics from a failed dispatch, ace#1571) and the videos: block — are not findings.

The manifest also carries a videos: block so the training skills and the run-summary page can find the recordings without re-listing Drive:

videos:
  - journey_id: journey-learn-pass
    recipe_base: journey-learn
    attempt: 1
    drive_path: 6-qa-and-training/videos/journey-learn.mp4
    file_id: <drive fileId>
  - journey_id: journey-deliver-submit
    recipe_base: journey-deliver
    attempt: 1
    drive_path: 6-qa-and-training/videos/journey-deliver.mp4
    file_id: <drive fileId>

Omit the block entirely when no videos were captured.

And a drive_folders: block naming the folders this skill writes into (ace#3238). The illustrated training guides pass these ids straight to scripts/embed-doc-screenshots.ts --screenshots; without them every author has to find the folders by hand (connect-fi/20261018-1440 Phase 6). Since output previews contract v1 there is no single "screenshots folder" — passing frames are split per app, forensics sit elsewhere — so the block names each one:

drive_folders:
  previews_learn: <folderId of 3-commcare/previews/apps-learn/>      # § Step 4.9
  previews_deliver: <folderId of 3-commcare/previews/apps-deliver/>  # § Step 4.9
  forensics: <folderId of 6-qa-and-training/screenshots/>            # ui-dumps, *-FAILURE.*

Take each id from the drive_create_folder (find-or-create) call that resolved it — never a path you typed. Record BOTH previews folders even on a one-leg re-run: find-or-create the other leg's folder (do NOT clear it — the § Step 4.9 clear rule is per leg and still holds) so a Deliver-only re-run still hands the guides the Learn frames it kept. The slug follows the run's output key (previewsFolderPath), so a drifted key (apps-learn-app) is still recorded under previews_learn / previews_deliver — the key names the app, the id names the folder. The cross-opp Connect pool (ACE/_common/connect-screenshots/<v>/) is NOT this skill's output and is not recorded here; the guides resolve it themselves. Enforced: test/skills/capture-manifest-drive-folders.test.ts — every drive_folders.<key> a consumer names must be a key this block declares.

Step 6.2: Write each app's _previews.yaml — the index ace-web reads

After the manifest is written and read back, write ONE index per app into the folder § Step 4.9 resolved. The index is authoritative: ace-web shows exactly its items, in order, as the app's preview, and replays them at this skill's beat (captured_by), not at the app's.

Which frames. The leg's manifest captures[] rows, in capture order, and only when the leg's FINAL status is pass — i.e. after the Connect learn-completion / deliver gates in Step 5, not merely the recipe's status (the Step 5 hard rule, jjackson/ace#756). Leave out duplicate_of aliases (Step 5.5 — one moment is one preview) and any frame whose Step 5.8 readback was not ok. Caption each from its shows: line (Step 5.6) where it has one. buildPreviewsIndex also drops every opportunity-LIST frame (claim-opp-list*, claim-opp-new-tile, connect-resume-opp-list*, connect-resume-opp-tile — showsOtherOpportunities in lib/opp-title-frames.ts): the +7426 test worker is a shared pool, so its list names other ACE programs, other partners' included, and reviewers outside Dimagi read these previews (ace#3030). The frames stay in the manifest and the folder; they are only not previews.

import {
  buildPreviewsIndex, serializePreviewsIndex, assertPreviewsIndexReadable,
  PREVIEWS_INDEX_NAME, PREVIEWS_INDEX_MIME, outputSlug,
} from '../../lib/output-previews';

const index = buildPreviewsIndex({
  phase: 'commcare-setup',                // the phase that OWNS the output
  outputKey,                              // § Step 4.9 — exactly as run_state writes it
  capturedBy: 'app-screenshot-capture',
  capturedPhase: 'qa-and-training',
  capturedAt: new Date().toISOString(),
  frames: legPassed ? legCaptures : [],   // manifest rows: {file_id, name, step, shows, duplicate_of}
});
// write serializePreviewsIndex(index) to a LOCAL file, then:
// drive_upload_binary({name: PREVIEWS_INDEX_NAME, mimeType: PREVIEWS_INDEX_MIME,
//                      parentFolderId: <the leg's previews folder>, localFilePath})

Real bytes, never a Google Doc. drive_upload_binary with mimeType: 'text/yaml' — the same rule as the manifest (ace#2490): a Doc's export comes back with every \n as \r\n\r\n\r\n and the reader parses nothing.

Then read it back and assert (required, not advice). drive_read_file the uploaded id and run the TEXT through:

const rb = assertPreviewsIndexReadable(text, {
  folderSlug: outputSlug(outputKey), phase: 'commcare-setup', outputKey,
  capturedBy: 'app-screenshot-capture', expectedCount: index.items.length,
});

ok: false is a halt for that app's previews: fix and re-write, and record the finding in the structural verdict's auto_surfaced. An index the reader cannot parse is an app with no pictures, and nothing downstream reports it.

A leg that did not pass still gets an index — with items: []. Frames may already sit in its folder (the Deliver leg's connect-resume-opp frames upload before journey-deliver runs; a Learn walk can pass on the device and then fail the Connect gate). The empty index is what tells a reader to show none of them. The same holds for a leg that never ran this dispatch (blocked-by-learn, blocked-by-learn-incomplete): clear its folder (§ Step 4.9 step 4) and write the empty index, so a stale capture never outlives the verdict that superseded it. Exception: a Learn leg recorded satisfied-by-prior-completion (Step 2.7) did not re-walk and cannot — leave its folder and index exactly as they are.

Nothing is written under phases.commcare-setup.products for this. The index file is found by its folder; a pointer in products would render as a bogus output.

Step 6.5: Harvest selector-drift signal (atlas-drift)

Run the drift harvester over this run's screenshot ROOT (the same screenshotDir you passed to mobile_run_recipe in Step 5). The *-FAILURE.xml / *-FAILURE.png forensics and the per-step runRecipeWithDumps XMLs now sit in per-recipe subdirectories under it (#1130) — the harvester walks recursively, so pointing it at the root picks up every leg in one pass. This closes the consume-loop on selector drift — otherwise the dumps just pile up unread.

ACE_ROOT="${CLAUDE_PLUGIN_ROOT:-$(python3 -c "import json,os; d=json.load(open(os.path.expanduser('~/.claude/plugins/installed_plugins.json'))); print(d['plugins']['ace@ace'][0]['installPath'])")}"
node "$ACE_ROOT/node_modules/tsx/dist/cli.mjs" "$ACE_ROOT/scripts/probe-atlas-drift.ts" <screenshotDir> \
  --out <screenshotDir>/atlas-drift-report.md

No --apk flag, deliberately. The probe reads ACE_CONNECT_APK_VERSION itself and falls back to DEFAULT_APK in scripts/probe-atlas-drift.ts — a pin site lib/apk-pin-sites.ts classifies, so it moves with every baseline bump. This snippet used to pass --apk with its OWN shell-default version literal, a second pin the scanner could not see (its env-fallback form matched the JS || syntax, not the shell :- one), so it sat on the old baseline right through the 2.64.0 upgrade. ACE_* vars are normally EMPTY in a shell — they load into MCP subprocesses, not the parent (CLAUDE.md § Gotchas) — so that fallback was the operative value, and Step 6.5 diffed live 2.64.0 dumps against the previous baseline's map: every genuinely-new id read as drift, every dropped one was missed. Do not reintroduce the flag; one pin is the fix (ace#2078).

Best-effort: a probe error never fails the phase — log it and continue. When the report has a "⚠️ Drift suspects on FAILURE screens" section, treat those resource-ids as the priority signal: each is a candidate root cause for a recipe failure in this run (the recipe reached for a logical name whose mapped id: no longer matches what this APK renders — the #591/#618 selector-drift class). Do NOT auto-edit the selector map — surface the candidates in the verdict and, when a candidate is confirmed live (mobile_capture_ui_dump → candidate-tap → re-dump navigates), gh issue create against jjackson/ace proposing the new selectors.<logical-name> row. This is the "close the loop to the source of truth" rule — one live dump beats another plausible guess.

After any leg fails, also re-run the probe with --yaml-out <screenshotDir>/atlas-report.yaml. This classifies a *-FAILURE.xml four ways (mapped / drift / unmapped-surface / matcher-miss) and is the machine-readable sibling of the markdown report above — copy its classification value verbatim into the failing step's entry in per_item[].note on the Step 9 verdict, so the classification survives past this run instead of sitting unread in a report file (the #811/#893 fate — 4 of the 24 known issues cited a FAILURE dump nobody opened). A matcher-miss means the recipe reached for an anchor that IS on screen — the selector map is not the problem, and a new selector row must NOT be authored for it; the fix is correcting the recipe. Only unmapped-surface (needs_tier2: true in the yaml) means the surface itself has no map coverage and is a real candidate for a new logical-selector row. Getting this backwards — authoring a selector for a matcher-miss, or shrugging off a genuinely unmapped surface as "just a bad selector" — is exactly the #811/#893 failure this file exists to prevent.

When the classification IS unmapped-surface, the next step has a name: run skills/selector-map-heal (dimagi-internal/ace#1256), pointed at this atlas-report.yaml + the FAILURE dump. It authors candidate selector rows from the dump, proves them by re-running the blocked leg on-device, and ships them only on green — so record the classification AND the heal dispatch (or the operator instruction to run it) in the step verdict, not just the coverage gap. This applies to unmapped-surface only; matcher-miss and drift have the different remedies stated above.

Two more values are NON-verdicts — record them and move on, never heal on them (dimagi-internal/ace#1571):

  • superseded — the dump is from an attempt a later dispatch of the same recipe already replaced. *-FAILURE.* deliberately survives the per-dispatch wipe (#1034), so a leg that fails once and then passes leaves stale forensics sitting next to the passing retry's captures.
  • non-app-surface — every node belongs to the home screen or system chrome, so the recipe was not in the app at all. The real finding is "the app was not foregrounded" (force-stopped, crashed, backgrounded by a heal); note that instead, and never author a selector row for it.

Both used to surface as unmapped-surface with needs_tier2: true, and the routing above is scoped to unmapped-surface only — so neither may reach selector-map-heal. On hh-poverty-targeting/20260819-1435 a passing Deliver leg asked for a heal against the Android launcher. The probe now filters both and names the dumps it dropped in a skipped: block; a skipped: entry is context for the verdict note, not work.

Step 7: Thin UX smoke judge

For each smoke recipe (Learn + Deliver), assemble the captured screenshot set into a single LLM-as-Judge call:

Prompt: "These screenshots are from a smoke run of the {{app}} app. The target FLW persona (from PDD) is: {{persona_summary}}. Looking at these screenshots in order, would this person be able to complete the journey without confusion? Rate 0-3 + one-line reason. 0 = a typical persona-matching FLW would get stuck; 3 = obviously usable."

Threshold: ≥ 2/3 per app. Below → halt with verdict.

This is intentionally one LLM call per app (~2 calls total). Deep, per-dimension UX grading is app-ux-eval running from /ace:qa-deep.

Step 8: Self-evaluate (LLM-as-Judge)

For the structural-quality dimensions:

  • Did every smoke recipe complete (status: pass)?
  • Are screenshots of expected count produced (≥ 1 per takeScreenshot step)?
  • Are all screenshots non-zero bytes?

Step 9: Write verdicts

Write the canonical structural verdict to 6-qa-and-training/app-screenshot-capture_verdict.yaml AND the shallow smoke verdict to 6-qa-and-training/app-screenshot-capture_verdict-shallow.yaml. Both shapes conform to lib/verdict-schema.ts so opp-eval can aggregate. Write both as real bytes — drive_upload_binary with mimeType: 'text/yaml' and localFilePath — never as Google Docs (ace#2490; see Step 6).

REQUIRED before either file is written — and again after ANY re-run that changes a leg's status (dimagi-internal/ace#1830). Both verdicts must agree with the manifest's journeys[].status. Run the pure helper over the manifest you just wrote in Step 6 and each verdict you are about to write:

import { checkVerdictManifestAgreement, formatAgreementFindings }
  from '../../lib/verdict-manifest-agreement';
const report = checkVerdictManifestAgreement(manifest, verdict);

report.ok === false means the two files describe different runs. Do not write the verdict until every disagreement is resolved — resolve it by re-scoring, never by editing the manifest to match a verdict. The three kinds:

kindmeanswhat to do
stale-verdictthe leg PASSES in the manifest but the verdict grades it failed/ungradeablethe leg was recovered after this verdict was written — re-run the Step 7 smoke judge over the recovered captures and rewrite per_item, overall_score and verdict. Preserve the prior state in a correction: / superseded_per_item: block; do not erase it.
unsupported-passthe leg did NOT pass in the manifest but the verdict grades it passthe Step 5 hard rule (jjackson/ace#756) — that leg has no screenshots, so nothing was graded. Score it fail/incomplete with the recipe failure inline.
unreported-legthe verdict grades other legs but is silent on this oneadd the missing per_item entry; a mean computed over one leg is not a two-leg verdict.

This applies to BOTH verdict files, every time. The class this closes: on hh-poverty-targeting/20260828-0702 a recovered Deliver leg rewrote the manifest and the structural verdict, and the shallow verdict was left behind for five hours still carrying a [BLOCKER] reading "no Deliver screenshots exist for this run" — while opp-eval aggregates verdict files by directory discovery and Phase 9's llo-launch reads them, so the run misreported itself to every downstream consumer in the direction that keeps a healthy run gated. The contract was being honoured by hand on one file and not the other. Enforced: test/lib/verdict-manifest-agreement.test.ts.

Structural verdict (6-qa-and-training/app-screenshot-capture_verdict.yaml):

skill: app-screenshot-capture
target: <opp-name>
ran_at: <ISO timestamp>
capture_path: 6-qa-and-training/app-screenshot-capture_manifest.yaml

overall_score: 8.5             # 0.0–10.0, weighted across dimensions
verdict: pass | warn | fail | incomplete
# `incomplete` is reserved for *upstream-incomplete* state the skill
# itself cannot remediate: app-test-cases.yaml missing or malformed,
# `3-commcare/recipes/journey-*.yaml` files absent, recipes carrying
# unfilled REPLACE_* selectors. That's "the rubric COULD NOT grade
# because Phase 3 didn't ship its full output set," not "the AVD is
# sick."
#
# AVD / Maestro driver health is **not** incomplete-state — it's
# `verdict: fail` with severity:BLOCKER. Pre-0.13.165 a hung Maestro
# driver wrote `incomplete` and the phase shipped placeholders; that
# escape valve let real Phase 6 capability problems hide behind a
# benign-looking yellow verdict run after run. Since 0.13.165
# `mobile_ensure_avd_running` includes an auto-heal (force-stop +
# uninstall + reinstall of `dev.mobile.maestro`), so a healthy
# workstation almost never needs to surface AVD failure to this
# rubric. When the heal exhausts, we want the BLOCKER, not the soft
# fail — the operator runs `/ace:mobile-bootstrap` and the run
# retries with screenshots. NEVER use `verdict: blocked`
# (off-schema; not in lib/verdict-schema.ts).

dimensions:
  coverage:           { score: 9.0, weight: 0.30 }   # both smoke journeys covered
  execution:          { score: 8.5, weight: 0.30 }   # every smoke recipe status == pass
  artifact_quality:   { score: 9.0, weight: 0.20 }   # every screenshot is a valid PNG, non-zero bytes
  manifest_integrity: { score: 8.0, weight: 0.20 }   # manifest.yaml lists every screenshot actually present in Drive

per_item:
  - ref: learn
    score: 9.0
    verdict: pass
    note: "journey-learn walked to completion; 6 screenshots, all PNG"
  - ref: deliver
    score: 0
    verdict: incomplete       # or fail / pass
    note: "journey-deliver.yaml missing — Phase 3 deferred it"

Top-level verdict from the two legs:

LearnDelivertop-level verdictphase proceeds clean?
passpasspassyes
passfail (ran, broke)failno — blocks
passincomplete (recipe missing/scaffold)incompleteno — blocks
failblocked-by-learnfailno — blocks
incompleteincompleteincompleteno — blocks

A non-pass verdict still ships the Learn screenshots it captured. Operator-authorized whole-step skip remains the separate explicit escape (unchanged).


auto_surfaced:
  - severity: WARN
    message: "Recipe X timed out at step Y; partial screenshots captured"

Shallow smoke verdict (6-qa-and-training/app-screenshot-capture_verdict-shallow.yaml):

skill: app-screenshot-capture
target: <opp-name>
mode: shallow
ran_at: <ISO timestamp>
capture_path: 6-qa-and-training/app-screenshot-capture_manifest.yaml

overall_score: 8.3             # mean per-app smoke score, NORMALIZED to 0–10: mean(0–3) × 10 / 3
verdict: pass | fail | incomplete
# pass iff every per-app smoke-judge score >= 2/3.
smoke_raw_mean: 2.5            # the same mean on the judge's own 0–3 scale

dimensions:
  ux_smoke:           { score: 8.3, weight: 1.00, raw_scale: "0-3", raw: 2.5 }
# per_item scores stay on the judge's 0–3 scale (they are the judge's answers).
# overall_score and dimension scores MUST be 0–10 — lib/verdict-schema.ts
# `overall_score: min(0).max(10)` is the uniform shape ace-web and opp-eval read;
# writing the raw 0–3 mean there rendered a good 2.5/3 as a red "25/100"
# (spark-facilitator/20260925-1536).

per_item:
  - ref: "learn"
    score: 3
    verdict: pass
    note: "Field labels clear, submission confirmation visible"
  - ref: "deliver"
    score: 2
    verdict: pass
    note: "Mostly clear; one screen has a developer-named field but recoverable"

auto_surfaced: []

Incomplete-mode shape (when blocked before grading):

skill: app-screenshot-capture
target: <opp-name>
ran_at: <ISO timestamp>
capture_path: phase5-block.md

overall_score: 0          # required by schema; not meaningful when incomplete
verdict: incomplete
live_state_verified: false

dimensions:
  coverage:           { score: 0, weight: 0.30 }
  execution:          { score: 0, weight: 0.30 }
  artifact_quality:   { score: 0, weight: 0.20 }
  manifest_integrity: { score: 0, weight: 0.20 }

auto_surfaced:
  - severity: PLATFORM
    message: "ACE_E2E_PHONE family unset in .env; mobile_register_test_user cannot run"
  - severity: PLATFORM
    message: "app-test-cases.yaml missing or no is_smoke: true entries; re-run /ace:step app-test-cases <opp>"

CSRF token handling (when seeding ace-web prod sessions)

If a sub-step here needs to POST to a Django-backed endpoint (the ACE web app's /api/ingest/upload, an OCS / Connect write, etc.) from inside a Playwright browser context, never use page.request.post(). The Playwright request context has its own cookie jar that doesn't share the page's session cookies, so Django rejects the write with HTTP 403 ("CSRF cookie not set" or "CSRF token missing or incorrect"). This is the same class of bug mcp/connect/backends/playwright.ts and mcp/ocs-server.ts already work around in production (search those files for X-CSRFToken for the canonical examples).

The two-step pattern is: read the CSRF cookie from document.cookie, then issue the fetch from inside the page so the browser's own cookie jar travels with the request:

// Inside the Playwright session, after a GET has warmed csrftoken_ace:
const csrf = await page.evaluate(() => {
  const m = document.cookie.match(/(?:^|;\s*)csrftoken_ace=([^;]+)/);
  return m ? m[1] : null;
});
if (!csrf) throw new Error('csrftoken_ace not set — GET base URL first to warm it');

const response = await page.evaluate(
  async ([url, csrf, body]) => {
    const r = await fetch(url, {
      method: 'POST',
      headers: { 'X-CSRFToken': csrf as string, 'Content-Type': 'application/json' },
      credentials: 'same-origin',
      body: JSON.stringify(body),
    });
    return { status: r.status, body: await r.text() };
  },
  [endpoint, csrf, payload],
);

Notes:

  • Cookie name is csrftoken_ace on the ACE web app (ace-web); on the OCS / Connect Django backends it's plain csrftoken. Pick the right one for the host you're targeting.
  • The cookie isn't set until some view rendered by CsrfViewMiddleware has been hit — a single await page.goto(baseUrl) first is enough to warm it. See skills/upload-transcript/SKILL.md § Shell reference for the curl-cookie-jar equivalent of the same pattern.
  • The header MUST be X-CSRFToken (capital T-O-K-E-N), not X-CSRF-Token. Django's CsrfViewMiddleware only honors the former.

MCP Tools Used

  • ace-gdrive: drive_read_file, drive_upload_binary, drive_create_file, drive_list_folder, drive_create_folder, drive_trash_file (§ Step 4.9 clears a leg's previews folder before re-capture).
  • ace-mobile: mobile_ensure_avd_running, mobile_install_apk, mobile_run_recipe.

Mode Behavior

  • Auto: Run end-to-end, write artifacts, proceed.
  • Review: Pause after Step 2 (smoke-recipe selection) so an operator can confirm which two journeys will run; resume on approval.

Dry-Run Behavior

  • Read inputs and resolve smoke recipes normally.
  • Skip AVD boot and mobile_run_recipe calls.
  • Write empty manifest with dry_run: true flag.
  • Skip the UX smoke judge (no screenshots to grade).
  • State tracks as dry-run-success.

LLM-as-Judge Rubric

DimensionPass criteria
Coverageboth legs attempted; Learn always; Deliver iff Learn completed
Executionevery smoke recipe status: pass
Artifact qualityevery screenshot is a valid PNG with non-zero bytes
Manifest integritymanifest.yaml lists every screenshot path actually present in Drive
UX smoke (shallow verdict only)per-app score ≥ 2/3 from the thin smoke judge

Change Log

DateChangeAuthor
2026-10-18## Inputs are parameters with resolvers (journeys doc, test-case catalog, HQ domain, org, opportunity, opp_run_id tile matcher, output folder); the run_state write-back is skipped outside a run.ACE (#3148 batch 3)
2026-09-29Output previews contract v1 — smoke-leg PNGs move to the phase that BUILT the app. New § Step 4.9 resolves each leg's folder 3-commcare/previews/<app-output-slug>/ from the journey's app: binding and the output key THIS run's phases.commcare-setup.products actually carries (resolveAppOutputKey — apps.learn declared, drifted spellings honoured), and clears it per leg before upload (never the other leg's — Learn is one-way). New § Step 6.2 writes one authoritative _previews.yaml per app (real bytes, read back through assertPreviewsIndexReadable; items: [] for a leg that did not pass). 6-qa-and-training/screenshots/ keeps Phase 6's forensics only (ui-dumps, *-FAILURE.*, post-Learn landing). Manifest unchanged in shape; each row's Drive path now points at the preview folder. Helpers: lib/output-previews.ts. Spec: ace-web docs/specs/2026-09-29-output-previews-design.md.ACE team
2026-08-29Step 9 required check: the verdicts must agree with the manifest's journeys[].status (dimagi-internal/ace#1830). Both verdict files are now gated on checkVerdictManifestAgreement (lib/verdict-manifest-agreement.ts) before they are written, and again after any re-run that changes a leg's status. It names three kinds — stale-verdict (manifest passes, verdict says failed/ungradeable: the ace#1830 recovery direction), unsupported-pass (manifest failed, verdict says pass: the Step 5 hard rule, jjackson/ace#756, which this skill has asserted in prose since 2026-06-12 with nothing enforcing it), and unreported-leg. Surfaced by hh-poverty-targeting/20260828-0702, where a Deliver leg recovered at 15:02Z rewrote the manifest and the structural verdict while app-screenshot-capture_verdict-shallow.yaml sat at 3.0 / fail for five hours still carrying a [BLOCKER] reading "no Deliver screenshots exist for this run" — opp-eval aggregates verdict files by directory discovery and Phase 9's llo-launch reads them, so the run misreported itself to every downstream consumer in the direction that keeps a healthy run gated. The guard keys on data that already exists on both sides rather than on a new staleness stamp, because a stamp needs every writer to remember it — the same by-hand honouring that failed here. Enforced: test/lib/verdict-manifest-agreement.test.ts, calibrated against both real states of that run (the 14:10Z stale file as the positive control, the 19:30Z re-graded 2.5/pass as the negative).ACE team
2026-08-29Step 2.6 gains the input-without-erase recipe-sanity class (dimagi-internal/ace#1844). Maestro's inputText appends at the cursor, so an inputText with no eraseText before it concatenates onto whatever the field already held. Live on spark-facilitator/20260828-0703: 40 typed into a field holding a preload submitted 140, the form's cross-field constraint refused to advance, and the leg died two screens later on an unrelated scroll. Pure recipe shape — runs unconditionally, never gated on fields.ACE team
2026-07-31Step 5 output-dir contract: one run-scoped ROOT, MCP-namespaced per dispatch (dimagi-internal/ace#1130). Pass the SAME screenshotDir root to every mobile_run_recipe call in the phase; the MCP writes each dispatch into <root>/<recipeId>/ and confines #756's start-of-run wipe to that subdir, so the Deliver leg can no longer destroy the Learn leg's finished captures. Read artifacts back from result.screenshotsDir / result.screenshots[].path (never a glob over the root) and upload per dispatch as it returns. Step 5.5's hash command, Step 6.5's atlas-drift invocation (recursive — point it at the root), and the 00-postlearn-landing.xml local-copy location updated to match. Surfaced by bednet-spot-check/20260731-1353, where a PASSING Learn leg's screenshots + video were wiped by the Deliver dispatch and could not be re-captured (Learn completion is one-way per test user + opportunity, #568/#570).ACE team
2026-07-13Step 5.5 distinctness check (dimagi-internal/ace#866). After each journey's captures land and before the manifest is assembled, hash every PNG in the journey (md5 -q / shasum); any byte-identical pair means two "steps" observed the same frame. The FIRST step stays canonical; later duplicates are marked duplicate_of: <first-step> in the manifest (not listed as distinct captures) and logged in the verdict's auto_surfaced list. Matching row added to the Step 5 recipe-error table. Surfaced by hh-poverty-targeting/20260702-1456, where two byte-identical pairs shipped as distinct steps and deck slides claimed distinct moments over duplicate frames.ACE team
2026-06-12Step 5 hard rule: screenshots only from a status: pass execution in THIS run (jjackson/ace#756). A status != pass mobile_run_recipe result for a required journey means that leg has NO screenshots — record the leg fail with the recipe failure inline; never source leftover PNGs (no "replay cache" exists); the shallow verdict must not be pass if any required journey's recipe failed. Pairs with the MCP-side fix: mobile_run_recipe now wipes the screenshot dir at execution start so stale PNGs are structurally absent. Surfaced by bednet-spot-check run 20260612-1220, where a failed deliver leg shipped prior-run PNGs under a confabulated "runner replay cache" note.ACE team
2026-06-03Step 6.5 auto-harvests selector drift. End of Phase 6 now runs scripts/probe-atlas-drift.ts over the run's screenshot dir automatically (best-effort), closing the consume-loop on the runRecipeWithDumps + *-FAILURE.xml dumps that previously just accumulated. The harvester is now FAILURE-aware: resource-ids seen on a *-FAILURE.xml screen but absent from the selector map surface as a priority "Drift suspects on FAILURE screens" section (candidate root causes for a recipe failure in this run — the #591/#618 drift class). Pairs with thrown-failure forensics capture (mobile_run_recipe now captures the failure screen on a thrown driver-death too, not just on returned status:'fail'). Canonical contract: playbook/integrations/mobile-integration.md § Failure forensics.ACE team
2026-05-05Path-scheme migration. Inputs repointed to 2-scenarios/pdd-to-app-journeys.md, 3-commcare/app-test-cases.yaml, 3-commcare/app-deploy_summary.md, 3-commcare/recipes/. Outputs repointed to 6-qa-and-training/screenshots/<recipe-base>/<step-name>.png, 6-qa-and-training/app-screenshot-capture_manifest.yaml, 6-qa-and-training/app-screenshot-capture_verdict.yaml, 6-qa-and-training/app-screenshot-capture_verdict-shallow.yaml (per manifest). Both verdict YAML examples' capture_path updated. No behavior change beyond paths.ACE team
2026-05-27Recipe naming convention. Screenshot dirs updated from <journey-id>/ to <recipe-base>/ (journey-learn/, journey-deliver/). Recipe read references updated from J*.yaml to journey-*.yaml. Structural verdict per_item refs changed from ref: "J1.yaml" to ref: learn / ref: deliver. No runtime behavior change. See spec 2026-05-27-phase6-learn-deliver-decoupling.ACE team
2026-05-31Meaningful journey ids. The manifest's journey-id reference is now a meaningful slug (learn-happy-path / deliver-yes) from app-test-cases.yaml instead of J<n>. Pairs with the descriptive recipe filenames / screenshot dirs already in place. See skills/app-test-cases/SKILL.md § Journey id convention.ACE team
2026-05-31journey- prefix. The manifest's journey-id reference examples now carry the journey- prefix (journey-learn-pass / journey-deliver-submit) to match the amended id convention. See skills/app-test-cases/SKILL.md § Journey id convention.ACE team
2026-05-31Tolerate recipe: deferred (lazy deep recipes, #605). Phase 3 now authors recipe files only for the two is_smoke: true journeys; deep journeys carry recipe: deferred and have no file (generated on demand by /ace:qa-deep). Step 2's pre-flight is scoped explicitly to the is_smoke: true subset, and a deferred deep journey with no recipe file is no longer a missing-recipe failure. This skill already only walked the smokes — this change makes the deferred-deep state explicitly non-fatal.ACE team
2026-05-27Two-leg capture split. Step 5 rewritten as independent Learn leg + Deliver leg. Learn always runs first; a Deliver failure (or missing recipe) no longer suppresses Learn screenshots. Per-app per_item verdict mapping table added (pass/fail/incomplete/blocked-by-learn outcomes). Coverage rubric updated: "both legs attempted; Learn always; Deliver iff Learn completed." See spec 2026-05-27-phase6-learn-deliver-decoupling.ACE team
2026-05-06Step 2 input-completeness pre-flight — restructured the post-Step-1 logic into an explicit failure-mode table that distinguishes upstream Phase 3 incomplete output (master yaml without recipes) from smoke-flag malformation. Each failure halts with a named PLATFORM auto_surfaced message + the exact /ace:step remediation command, and writes verdict: incomplete (not fail — upstream gaps aren't smoke failures). Surfaced by leep-paint-collection run 20260506-1440 where a Phase 3 dispatch paraphrased the app-test-cases SKILL contract and elided the per-journey recipe outputs; app-screenshot-capture halted correctly but the operator-facing message conflated the failure mode with general "missing input" diagnostics. See jjackson/ace#106 finding #3 + #16.ACE team
2026-05-07Step 5 anyone-with-link via drive_upload_binary({shareAnyoneWithLink: true}) — replaces the previous unfulfillable contract (the SKILL named drive.permissions.create but no MCP atom implemented it). The new flag sets role: reader, type: anyone atomically at upload time, eliminating the "deck builds without errors but slides are empty" failure mode. Standalone drive_set_anyone_with_link({fileId}) atom also added for retroactive sharing. See jjackson/ace#115 finding #3.ACE team
2026-08-01Migrated the recipe-sanity-probe's per-form field read to uuid addressing (ace#1132). The probe's optional fields input was spelled as a nova_get_form read taking the module/form index pair; Nova has accepted no index param since its 2026-07-31 redeploy, so the call would have been rejected and the probe silently degraded to its field-blind behaviour — re-introducing the #858 false positives on label-heavy Learn apps. Now nova_get_form({app_id, moduleUuid, formUuid}), uuids off the nova_get_app({app_id}) responses the probe already collects. Enforced by test/skills/nova-uuid-addressing.test.ts.ACE team
2026-09-06Stop routing concerns to a gate brief that does not exist (dimagi-internal/ace#1884). 0.13.116 removed the per-skill gate-brief file class and the ace#1880 sweep removed the remaining *.md PATHS, but prose directives naming the gate brief as a DESTINATION survived in 15 files — a concern "surfaced in the gate brief" is surfaced nowhere. Repointed at the verdict YAML's auto_surfaced block, which is what the orchestrator actually renders the pause summary from. Gated by the new destination check in test/skills/gate-brief-removal-complete.test.ts.ACE team
2026-09-06Large artifacts are composed to a LOCAL FILE and written with localFilePath (dimagi-internal/ace#1918). The manifest was measured at 43,778 chars in the 2026-09-02 Drive corpus. Enforced: test/skills/large-artifact-localfilepath.test.ts.ACE team
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 197kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall