panjose/Co-Scientist / skills/hypothesis-ranking-pipeline
hypothesis-ranking-pipeline
VerifiedUpdate ranking artifacts for one reviewed hypothesis using canonical placement-opponent selection, ranked-frontier selection, tournament judgments, and Elo updates.
hypothesis-ranking-pipeline
Goal:
- Update ranking artifacts for one reviewed hypothesis using canonical placement-opponent selection, ranked-frontier selection, tournament judgments, and Elo updates.
Inputs:
- candidate
hypotheses/<id>/HYPOTHESIS.json - current viable hypothesis frontier
- optional
state/PROXIMITY_GRAPH.json - current tournament artifacts
- numeric ranking settings from
state/RESOLVED_RUN_CONFIG.json
Outputs:
- updated
tournaments/*.json - updated candidate and opponent
hypotheses/<id>/HYPOTHESIS.jsonartifacts with Elo and match references - updated
state/PIPELINE_STATE.json - updated
state/CURRENT_STAGE.json
Sub-skills:
hypothesis-placement-tournamenthypothesis-ranked-tournamentranking-elo-update
Context Loading:
- Open
skills/shared-references/schema-index.md. - Read
packages/agent_contracts/hypothesis.pyand confirm the exactHypothesisContractshape before writing any touchedHYPOTHESIS.jsonartifact. - Read
packages/agent_contracts/ranking.pyand confirm the exactTournamentMatchContractplusHypothesisMatchupContractshapes. - Read
packages/agent_contracts/resolved_config.pyforplacement_match_countandtournament_top_k. - Read
packages/agent_contracts/pipeline_runtime.pybefore updatingstate/PIPELINE_STATE.jsonorstate/CURRENT_STAGE.json. - If
state/PROXIMITY_GRAPH.jsonexists, readpackages/agent_contracts/state.pyand load the exactProximityGraphContract. - Read
packages/agent_contracts/proximity.pybefore interpretingstate/proximity_receipts/<hypothesis_id>.jsonorstate/PROXIMITY_STATUS.json. - Treat a missing or candidate-incomplete proximity graph as a valid upstream state only when the candidate has a bridge receipt/status proving that the embedding provider was skipped, disabled, unavailable, invalid, or failed. Do not interpret graph absence as permission to fabricate similarity state.
Execution Contract:
- This pipeline skill may orchestrate LLM tournament judgments, but all frontier-selection mechanics must use the canonical deterministic helper surface from
tools. - Use
from tools import sync_pipeline_stage_artifactsas the canonical paired write surface for entering theRankingsubstage. - On entry, call
tools.sync_pipeline_stage_artifacts(run_dir, current_phase="Ranking", current_skill="hypothesis-ranking-pipeline"). - Use
from tools import select_placement_opponentsas the canonical placement-opponent selector whenstate/PROXIMITY_GRAPH.jsonis available. - Use
from tools import select_fallback_placement_opponentsonly for receipt-gated placement fallback when usable proximity state is unavailable. - Use
from tools import get_top_k_hypothesesto derive the canonical current top-k frontier. - Use
from tools import should_run_ranked_tournamentto decide whether the candidate must enter ranked play. - Use
from tools import select_ranked_opponentsto derive the canonical ranked-tournament opponents when ranked play is warranted. - The helper implementations live in
packages/agent_mechanics/top_k_select.py. - Use
ranking-elo-updateafter tournament winners are already decided. It is the canonical Elo and match-reference update step. - Apply the placement batch Elo update before ranked-top-k gating; ranked tournament eligibility must be computed from the post-placement frontier, not stale pre-placement ratings.
ranking-elo-updatemust close the batch by callingtools.apply_and_persist_elo_updates(...)before convergence or downstream routing reads the frontier again.- The ranking closeout helper updates touched canonical hypothesis artifacts without rewriting standalone
REVIEW/*.jsonstage artifacts and writes aRankingUpdateReceiptContract. - Match refs are not sufficient closeout proof; each completed placement or ranked batch must have a persisted ranking update receipt.
- Ranking closeout must use the receipt-writing helper exactly once per completed batch. If the same stable receipt already exists, treat the helper's returned persisted receipt as the closeout proof and do not reapply Elo deltas or append duplicate match refs.
- Do not replace the helper logic with ad hoc manual opponent choice when the canonical artifacts needed by the helpers are available.
- If
state/PROXIMITY_GRAPH.jsonis missing, invalid, or does not contain usable state for the candidate, first checkstate/proximity_receipts/<candidate_id>.jsonandstate/PROXIMITY_STATUS.json. - Receipt-gated placement fallback is allowed only when those bridge artifacts explicitly record a skipped, disabled, failed, or provider-unavailable proximity outcome for the candidate. If no such receipt/status exists, return to
hypothesis-proximity-updateinstead of choosing opponents manually. - When receipt-gated placement fallback is used, call
tools.select_fallback_placement_opponents(candidate, hypotheses, placement_match_count)and write an explicit trace/rationale that names the candidate receipt status and states that placement was not embedding-, proximity-, or similarity-informed.
Execution Steps:
- Open
skills/shared-references/schema-index.md, then read the exact ranking, hypothesis, resolved-config, optional proximity, and pipeline-runtime contracts before writing artifacts. - Before selecting opponents or running tournaments, call
tools.sync_pipeline_stage_artifacts(run_dir, current_phase="Ranking", current_skill="hypothesis-ranking-pipeline"). - Load the candidate hypothesis, the current viable frontier, and
state/RESOLVED_RUN_CONFIG.json. - If
state/PROXIMITY_GRAPH.jsonis available and contains usable proximity state for the candidate, calltools.select_placement_opponents(graph, candidate, hypotheses, placement_match_count)to derive the placement opponents. - If
state/PROXIMITY_GRAPH.jsonis unavailable, invalid, or candidate-incomplete, inspect the candidate proximity receipt/status. If the bridge artifacts record a skipped, disabled, failed, or provider-unavailable outcome, record an explicit receipt-gated fallback rationale in the trace, name the fallback status, state that placement was not embedding-, proximity-, or similarity-informed, and calltools.select_fallback_placement_opponents(candidate, hypotheses, placement_match_count). If no such bridge artifact exists, stop ranking and return tohypothesis-proximity-update. Do not treat fallback selection as equivalent totools.select_placement_opponents(...)when the graph exists. - Run
hypothesis-placement-tournamentfor each placement matchup and persist the resultingTournamentMatchContractartifacts. - Run
ranking-elo-updateexactly once for the placement batch immediately after placement winners are decided, and confirm it writes every touchedhypotheses/<id>/HYPOTHESIS.jsonartifact plus a ranking update receipt before ranked gating. Match ref arrays must remain duplicate-free. - Reload the candidate and viable frontier from the persisted hypothesis artifacts.
- Call
tools.get_top_k_hypotheses(hypotheses, tournament_top_k)to derive the post-placement frontier andtools.should_run_ranked_tournament(candidate, top_k_hypotheses, tournament_top_k)to decide whether ranked play is required. - If ranked play is required, call
tools.select_ranked_opponents(candidate, hypotheses, tournament_top_k)and runhypothesis-ranked-tournamentfor each resulting matchup. If ranked play is required but no ranked opponents exist because the frontier has no other viable hypotheses, record that as a traceable no-op. - If ranked tournament matches were created, run
ranking-elo-updateexactly once for the ranked batch and confirm it writes every touchedhypotheses/<id>/HYPOTHESIS.jsonartifact plus a ranking update receipt again. Do not copy opponent-side ranked refs into the current child's evolution-round receipt unless the child washypothesis_1_idfor that ranked match. - Persist the touched tournament artifacts and touched
hypotheses/<id>/HYPOTHESIS.jsonartifacts. - Validate the updated ranking artifacts before declaring completion.
Artifact Rules:
packages/agent_mechanics/top_k_select.pyis the canonical frontier-selection implementation; the pipeline skill must not fork its semantics in prompt prose.- Placement-opponent choice, ranked-entry gating, and ranked-opponent choice are deterministic mechanics, not freeform policy decisions.
- When canonical similarity state exists, placement opponents must come from
tools.select_placement_opponents(...). - When canonical similarity state is unavailable and a receipt-gated fallback is used, placement opponents must come from
tools.select_fallback_placement_opponents(...), and ranking artifacts and traces must describe the fallback as receipt-gated deterministic placement, not as proximity-informed ranking. - Ranked-tournament entry must come from
tools.should_run_ranked_tournament(...), not from an ad hoc narrative threshold. - Ranked opponents must come from
tools.select_ranked_opponents(...), not from a manually re-sorted custom frontier.
Completion Rule:
- This skill is complete only when the canonical selection helpers have been applied where their required artifacts exist, all resulting tournament winners have been serialized,
ranking-elo-updatehas persisted the touched hypothesis artifacts and ranking update receipt, and the run validates for downstream evolution or overview work.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |