Walsworth-Group/physkit / skills/fork-campaign
fork-campaign
VerifiedFresh-ledger sibling when a bug invalidates a campaign's findings.
fork-campaign — the invalidation clean break
A campaign's ledger is only as trustworthy as the infrastructure it was scored under. Some changes invalidate or make misleading the findings a campaign accumulated: a forward-model bug that scored a decoder on wrong test data, a metric error that inverted the rankings, a corrected dataset that changes which approaches win. A version bump that carries the tainted journal forward is not a permitted response to such a change. The required response is a clean break: a fresh-ledger campaign, the source archived read-only, and only the findings that survive re-verification carried across. Fork-and-archive is a first-class move so a journal built on a known-bad model can never silently steer the next effort, the memory-contamination failure at campaign scale.
First: is a fork actually the right move?
One question drives the decision: does this change invalidate or make misleading the campaign's prior findings?
- (a) Findings stay valid — a small fix, or a metric refinement that preserves the rankings. Not a fork. Do a
version bump within the campaign (
metric_version/data_version), re-score affected rows under the comparability guard, and log a one-line change-record in the lineage. The normal loop andcampaign-statushandle this, not this skill. - (b) Findings are invalidated or made misleading — approaches may have been wrongly discarded or promoted on bad infrastructure. Fork-and-archive. This skill.
- (c) A materially new approach (new architecture, objective, or outputs), with no inheritance beyond shared
data and artifacts. A new campaign (
new-campaign), not a fork: there is nothing tainted to break from.
If (a) or (c), stop and route there. Proceed below only for (b). When unsure between (a) and (b): if you cannot show the change preserves the rankings, treat it as (b). The cost of a wrongly-carried-forward tainted finding exceeds the cost of a clean re-verification.
Preconditions
- The source campaign sits in its project:
campaigns/<src>/with a../../PROJECT.mdcarrying the lineage (PROJECT.md§Lineage). - A campaign without a project home (imported or hand-built) is lifted into a project first (
new-campaignStep 0.1); the archive and lineage machinery this skill needs exists nowhere else. - Know what broke and which findings it taints before forking. A
debugorverifysession that exposed the invalidating bug supplies that diagnosis. Do not fork on suspicion; fork on a diagnosed cause.
Procedure
1. Diagnose & record the invalidation
Write down, precisely: what changed (the bug or correction); why it invalidates the prior findings (which scored quantities were wrong, and how); and which specific findings and promotions are now suspect. The suspect list covers every DISCOVERY, every current best, and every written-off approach decided on the bad infrastructure. This is the fork's justification and goes into the lineage. A finding is suspect until proven still-valid, not the reverse.
2. Archive the source (frozen, read-only, still indexed)
- Move
campaigns/<src>/tocampaigns/_archive/<src>/, creating_archive/if absent. Nothing is deleted: the archived campaign stays for provenance. - Mark it frozen and read-only. Drop a
FROZEN.mdat its root stating that it was archived by a fork on<date>, the invalidation reason, and "read-only — do not resume; superseded bycampaigns/<new>/." - Filesystem read-only permissions are unreliable on a shared drive. The
FROZEN.mdmarker and the lineage status are the enforcement, not permissions. - Its
results/—JOURNAL.md,scoreboard.tsv(itsfindingcells included), the logbook — do not become working documents of the new effort. They are read-only history. The new campaign starts from a clean ledger, never by copying these forward.
3. Scaffold the fresh-ledger campaign (compose new-campaign)
Invoke new-campaign in the project to create a new sibling campaigns/<NN>_<slug>/ with a clean scoreboard
and a fresh JOURNAL (its template sections, empty). Carry over, deliberately and explicitly:
- the corrected infrastructure (the fixed forward model, metric, or dataset), validated before it is re-frozen.
Run
correctness-suite: the sign and convention round-trip plus the symmetry self-tests that would have caught the original bug. Add the check that failed as a standing regression gate. - the shared data and promoted artifacts from the project level (
../../data,../../models/, …), unchanged: referenced, never copied. - the FROZEN metric spec and levers, re-examined for whether the bug touched them. (A metric that was
merely wrong, with the save-sets still valid and re-scorable, is a metric epoch inside the source
campaign —
campaign.md§The ONE metric — not a reason to fork; fork when the artifacts are tainted.)
Do not carry over the old ledger, the old current-best config as "best", or any conclusion. Those are re-earned in step 4.
4. Carry forward ONLY what survives RE-verification (the honesty gate)
For each prior finding the user wants to keep, re-verify it from scratch under the corrected setup: re-run
verify, which spawns an independent refuter, recompute every number on the fixed infrastructure, and grade it
again. Then:
- A finding that returns CONFIRMED under the corrected setup carries forward as its new row's
findingcell and under## Where we standin the new JOURNAL, tagged with its provenance: "carried from<src>and re-verified after the fork on<date>, verdict CONFIRMED" and citing the NEW verdict artifact. A finding never carries forward on the strength of the old verdict. - A finding that does not survive is left in the archive, not carried. If it had been promoted, note in the lineage that the fork retracted it.
- No claim-splitting, no silent inheritance. A number is inseparable from the infrastructure it was scored under. If that infrastructure was the bug, the number is refuted-until-re-earned.
5. Record the fork in the lineage
Update the project index so the history is complete and honest:
- Set the archived campaign's status to
archived(forked). Leave its row in place: provenance never disappears. - Append the new campaign's row with status
active, with what changed from predecessor set to the fork reason, and a note linking it to the archived source. - Record, in the row or a fork note, what was invalidated and what carried over re-verified, with the verify verdicts. Anyone reading the lineage must be able to see that a fork happened, why, and which prior conclusions did and did not survive.
Verify (before you hand off)
- The source is in
campaigns/_archive/, carries itsFROZEN.md, and its lineage status isarchived(forked). - The new campaign has a clean ledger (no copied scoreboard or JOURNAL rows) and passes
validate_campaign.py. - Every carried-forward finding shows a post-fork CONFIRMED verdict with provenance; nothing rode in on an old verdict.
- The lineage records the fork, the invalidation reason, and the carried and dropped findings.
Composition & boundaries
Composes new-campaign (the fresh scaffold) and verify (the re-validation gate). Pairs with debug (which
diagnoses the invalidating bug) and correctness-suite (which re-validates the corrected infrastructure before
re-freezing). A metric epoch keeps the ledger because the save-sets are valid and re-scorable; a fork breaks it because the artifacts themselves are tainted.
Not for scaffolding a brand-new investigation (new-project), a routine fix (a version bump), or a new
approach (new-campaign).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |