Skip to content

amazon-science/ammo / ai_cli_session/.codex/skills/ammo

Profile and optimize GPU kernels for vLLM inference on NVIDIA GPUs. Use when targeting a specific model, hardware, dtype, parallelism, and workload to improve latency.

FileKindSize
RUNTIME.mdmarkdown4.9 KB
SKILL.mdskill md18.5 KB
agents/ammo-auditor.mdmarkdown9.1 KB
agents/ammo-champion.mdmarkdown9.6 KB
agents/ammo-delegate.mdmarkdown3.2 KB
agents/ammo-implementer.mdmarkdown13.8 KB
agents/ammo-investigator.mdmarkdown4.0 KB
agents/ammo-report.mdmarkdown2.9 KB
agents/ammo-researcher.mdmarkdown10.3 KB
agents/ammo-resolver.mdmarkdown4.4 KB
agents/ammo-transcript-monitor.mdmarkdown7.8 KB
agents/openai.yamlexecutablescript313 B
data/ATTRIBUTION.mdmarkdown1.4 KB
data/gsm8k_full.jsonexecutablescript735.5 KB
data/gsm8k_subset.jsonexecutablescript116.8 KB
eval/EVAL-SKILL.mdexecutablescript21.8 KB
eval/agents/transcript_grader.mdexecutablescript18.0 KB
eval/causal/__init__.pyexecutablescript137 B
eval/causal/agents/causal_analyzer.mdexecutablescript10.5 KB
eval/causal/build_dag.pyexecutablescript21.6 KB
eval/causal/diff_versions.pyexecutablescript38.5 KB
eval/causal/extract_events.pyexecutablescript36.3 KB
eval/causal/generate_postmortem.pyexecutablescript17.0 KB
eval/causal/score_nodes.pyexecutablescript21.5 KB
eval/causal/templates/narrative_template.mdexecutablescript472 B
eval/causal/templates/viz_template.htmlexecutablescript9.5 KB
eval/causal/tests/__init__.pyexecutablescript151 B
eval/causal/tests/conftest.pyexecutablescript448 B
eval/causal/tests/fixtures/sample_session.jsonlexecutablescript10.4 KB
eval/causal/tests/fixtures/sample_session_data.jsonexecutablescript3.4 KB
eval/causal/tests/fixtures/sample_snapshot.jsonexecutablescript2.5 KB
eval/causal/tests/test_build_dag.pyexecutablescript2.7 KB
eval/causal/tests/test_diff_versions.pyexecutablescript13.5 KB
eval/causal/tests/test_extract_events.pyexecutablescript3.7 KB
eval/causal/tests/test_generate_postmortem.pyexecutablescript3.1 KB
eval/causal/tests/test_score_nodes.pyexecutablescript3.3 KB
eval/scripts/aggregate_versions.pyexecutablescript15.2 KB
eval/scripts/archive_run.pyexecutablescript9.1 KB
eval/scripts/codex_session.pyexecutablescript10.6 KB
eval/scripts/generate_dashboard.pyexecutablescript5.3 KB
eval/scripts/parse_artifacts.pyexecutablescript48.1 KB
eval/scripts/parse_session_logs.pyexecutablescript51.3 KB
eval/scripts/rescore_archived.pyexecutablescript5.5 KB
eval/scripts/score_campaign.pyexecutablescript43.8 KB
eval/scripts/snapshot_changes.pyexecutablescript11.1 KB
eval/scripts/verify_citations.pyexecutablescript8.8 KB
eval/tests/test_dashboard_bugs.pyexecutablescript8.8 KB
eval/tests/test_parse_session_logs.pyexecutablescript3.9 KB
eval/tests/test_parse_session_logs_round_inference.pyexecutablescript11.0 KB
eval/tests/test_score_campaign_diluted.pyexecutablescript9.4 KB
eval/viewer/dashboard_template.htmlexecutablescript15.1 KB
instrumentation/sitecustomize.pyexecutablescript3.6 KB
orchestration/audit-protocol.mdmarkdown9.3 KB
orchestration/debate-protocol.mdmarkdown11.2 KB
orchestration/integration-logic.mdmarkdown10.3 KB
orchestration/parallel-tracks.mdmarkdown8.9 KB
references/artifact-layout.mdexecutablescript22.4 KB
references/audit-invariants.mdmarkdown10.6 KB
references/champion-common-patterns.mdmarkdown3.4 KB
references/crossover-probing.mdexecutablescript4.7 KB
references/cudagraph-safety.mdmarkdown2.7 KB
references/debate-rules.mdmarkdown4.7 KB
references/debate-scoring-rubric.mdexecutablescript16.7 KB
references/e2e-delta-math.mdexecutablescript17.9 KB
references/e2e-latency-guide.mdexecutablescript9.6 KB
references/fusion-feasibility-heuristics.mdmarkdown5.6 KB
references/gpu-pool.mdexecutablescript5.4 KB
references/impl-track-rules.mdexecutablescript13.8 KB
references/nsys-profiling-guide.mdmarkdown9.8 KB
references/optimization-categories.mdexecutablescript11.0 KB
references/shutdown-protocol.mdexecutablescript3.1 KB
references/technology-selection.mdmarkdown5.0 KB
references/torch-compile-contract.mdmarkdown13.8 KB
references/validation-defaults.mdexecutablescript34.2 KB
references/workload-invocation.mdexecutablescript4.9 KB
references/writing-style.mdexecutablescript6.9 KB
scripts/ammo_state.pyexecutablescript142.0 KB
scripts/artifact_layout.jsonconfig16.3 KB
scripts/check_projection_accuracy.pyexecutablescript11.7 KB
scripts/create_evidence_template.pyexecutablescript9.7 KB
scripts/create_worktree_with_build.shexecutablescript10.2 KB
scripts/generate_validation_report.pyexecutablescript35.7 KB
scripts/gpu_reservation.pyexecutablescript27.1 KB
scripts/hook_cmd_classify.pyscript13.3 KB
scripts/mine_trace.pyscript29.4 KB
scripts/monitor_queue_ack.pyexecutablescript6.1 KB
scripts/ncu_sanity.shexecutablescript5.2 KB
scripts/ncu_sanity_driver.pyexecutablescript8.5 KB
scripts/new_target.pyexecutablescript22.0 KB
scripts/preflight_vllm_api.pyexecutablescript12.3 KB
scripts/reconcile_track_state.pyexecutablescript15.8 KB
scripts/remove_worktree_cleanup.shexecutablescript1.1 KB
scripts/resolve_codex_transcript.pyexecutablescript3.6 KB
scripts/run_vllm_bench_latency_sweep.pyexecutablescript249.6 KB
scripts/transcript_filter.pyexecutablescript38.4 KB
scripts/transitions.jsonexecutablescript1.5 KB
scripts/verify_validation_gates.pyexecutablescript88.8 KB
tests/agents/test-champion.mdexecutablescript20.4 KB
tests/agents/test-impl-champion.mdexecutablescript14.3 KB
tests/agents/test-implementer.mdexecutablescript15.8 KB
tests/agents/test-orchestrator.mdexecutablescript33.8 KB
tests/agents/test-researcher.mdexecutablescript21.0 KB
tests/agents/test-transcript-monitor.mdexecutablescript13.4 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.mdmarkdown1.1 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.summary.jsonconfig582 B
tests/fixtures/diluted_pass/legacy_no_phase.jsonconfig400 B
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.mdmarkdown2.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.summary.jsonconfig1.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.jsonconfig626 B
tests/fixtures/mining/real_campaign_bottleneck_analysis.mdmarkdown20.7 KB
tests/run_all.shexecutablescript1.6 KB
tests/scripts/test_transcript_filter.pyexecutablescript41.1 KB
tests/test_agent_runtime_contracts.pyexecutablescript1.8 KB
tests/test_ammo_state.pyexecutablescript71.7 KB
tests/test_artifact_layout_manifest.pyscript11.6 KB
tests/test_audit_started.pyscript18.2 KB
tests/test_correctness_comparator.pyscript12.6 KB
tests/test_diluted_pass.pyscript24.4 KB
tests/test_drift_compat.pyscript5.1 KB
tests/test_evidence_freshness.pyexecutablescript5.6 KB
tests/test_fail_closed.pyexecutablescript14.7 KB
tests/test_gpu_reservation.pyexecutablescript24.8 KB
tests/test_gpu_reservation_session_id.pyexecutablescript11.2 KB
tests/test_guidance_semantic_contract.pyscript24.9 KB
tests/test_hook_cmd_classify_parity.pyscript11.7 KB
tests/test_hook_semantics.pyexecutablescript95.3 KB
tests/test_mine_trace.pyscript24.2 KB
tests/test_mining_invalidated_schema.pyexecutablescript9.4 KB
tests/test_new_target_dp_ep.pyexecutablescript7.4 KB
tests/test_new_target_v2_scaffold.pyexecutablescript13.1 KB
tests/test_parse_artifacts.pyexecutablescript3.0 KB
tests/test_policy_convergence.pyscript797 B
tests/test_preflight_vllm_api.pyscript13.3 KB
tests/test_projection_accuracy_v2.pyexecutablescript4.1 KB
tests/test_reconcile_track_state.pyexecutablescript18.2 KB
tests/test_resolve_codex_transcript.pyexecutablescript3.1 KB
tests/test_state_schema_semantics.pyexecutablescript7.6 KB
tests/test_status_set_sync.pyexecutablescript9.1 KB
tests/test_sweep_baseline_from_arm.pyscript11.0 KB
tests/test_sweep_decode_capture.pyexecutablescript31.8 KB
tests/test_sweep_dp.pyexecutablescript45.4 KB
tests/test_sweep_fresh_cache.pyscript5.5 KB
tests/test_sweep_profiler_guard.pyscript22.6 KB
tests/test_sweep_real_prompts.pyscript20.1 KB
tests/test_sweep_refactor.pyexecutablescript37.6 KB
tests/test_sweep_round_slot.pyexecutablescript7.4 KB
tests/test_validation_report_phase_triage.pyexecutablescript10.0 KB
tests/test_validation_report_v2.pyexecutablescript6.2 KB
tests/test_verify_validation_gates.pyexecutablescript52.9 KB
tests/test_workload_matrix.pyexecutablescript19.3 KB

references/audit-invariants.md

View on GitHub
# Audit Invariants

This is the Phase 2 coverage floor delivered to `ammo-auditor` only after its
independent Phase 1 reconstruction. It names semantic properties, not a frozen
row count. Mechanical validators own schema, enum, arithmetic, completeness,
and transition checks; verify their inputs, then challenge the meaning,
provenance, and unexplained discrepancies.

Apply **Pre-Check**, the assigned stage section, and **Cross-Artifact Checks**.
Record a non-applicable property with its reason. `BLOCKING` means the
stage claim is unsafe; `HIGH` means investigate and explain before acceptance.
Severity must scale with campaign impact: a finding that cannot change the
stage verdict or a shipped number is not `BLOCKING`, and a finding with no
material impact is reported as a note, not a repair-cycle trigger. The lead
rejects a `BLOCKING` rating that names no campaign-impact mechanism.
Phase 2 cannot erase a Phase 1 blocker.

## Pre-Check

- **Primary artifacts exist (BLOCKING).** Resolve canonical round-scoped paths
  from `state.json` and `artifact-layout.md`. A state pointer or rendered view is
  not evidence when its primary file is absent.
- **Mechanical validation is green (BLOCKING).** Run the applicable schema,
  transition, evidence, and validation-gate commands. Failed sweep legs, null or
  non-finite metrics, placeholders, partial buckets, or state/report disagreement
  invalidate the affected claim.
- **Identity and provenance agree (BLOCKING).** Target, round, `op_id`, source
  tree/patch, environment, workload, topology, and artifact timestamps describe
  the same run. Do not silently combine adjacent runs.
- **Frozen target is honored (BLOCKING).** Model revision, dtype, TP/DP/EP,
  compile/graph mode, batch/workload matrix, serving knobs, and comparator match
  the campaign contract unless an explicit authorized override exists.
- **Claims are cited (HIGH).** Non-trivial architecture, dispatch, phase, and
  performance assertions resolve to source, model config, trace, or runner data.

## After Stage 1 — `T_AUDIT_S1`

- **Clean timing is authoritative (BLOCKING).** Baseline timing comes from the
  profiler-free `baseline` slot with successful legs and the frozen workload.
  Profiling or golden-capture latency is never substituted.
- **Profiling is separate and representative (BLOCKING).** Nsys captures use the
  same production path and topology, cover the claimed phases/steps, and are
  clearly marked non-timing evidence. Rank/device coverage is sufficient for the
  topology.
- **Baseline values are physically and statistically plausible (HIGH).** Check
  units, iteration aggregation, variance/outliers, and scaling across workload
  buckets. Investigate implausible values rather than enforcing a universal
  latency constant.
- **Golden comparator is usable (BLOCKING).** The capture is complete for the
  declared correctness task and bound to the same baseline source/environment.
- **Constraints reflect evidence (BLOCKING).** Architecture facts match the
  actual model config; production dispatch, correctness invariants, and workload
  characterization match source and traces.

## After Stage 2 — `T_AUDIT_S2`

- **Phase accounting inputs are canonical (BLOCKING).** `scripts/mine_trace.py`
  owns the arithmetic and fails loud, so verify its inputs instead of
  recomputing `decode_busy`, the shares, closure, or per-row `f_e2e`:
  - `rounds/{N}/mining/mined.json` exists with schema `mine_trace/1`, and the
    pasted `tables.md` numbers in `bottleneck_analysis.md` match it. A
    hand-edited table is a blocker.
  - `config_echo` names the canonical round-scoped traces, arm, and
    `e2e_results`, and its `provenance` records the kernel table and nsys
    versions actually used. No FATAL is suppressed and every `warnings[]` entry
    is addressed in the artifact prose.
  - The delimiter policy is defensible: `separation_ratio` is not marginal and
    `first_kernel_after` fits a steady-state step boundary. State
    the capture's `--nsys-output-len` — a reduced-OSL capture can ground prefill
    attribution but not decode step counts or decode shares.
  - The family partition is semantically right: labels match their symbols,
    `partition_coverage` is complete, and any `residual_pct` is small and
    disclosed. `step_count_source` is the strongest rung the artifacts support.
- **Opportunity accounting uses `f_e2e` (BLOCKING).** Top components are ranked
  by production E2E share, not decode-only share. Their total does not exceed the
  measured phase budget; prefill-active and inter-kernel slices are not erased.
- **Bottlenecks map to production source (BLOCKING).** Kernel aliases, occurrence
  counts, phase membership, dispatch paths, and TP-rank behavior support each
  named bottleneck. Trace order overrides architecture intuition.
- **Bounds are physical, not candidate promises (HIGH).** Removable-work and
  hardware ceilings state their boundary and assumptions. Host gaps are not
  counted as recoverable without production-boundary evidence.
- **Mining remains proposal-neutral (HIGH).** The artifact supplies facts and
  bounds; it does not pre-select a mechanism or inflate attainable speedup.

## After Stages 4-5 — `T_AUDIT_S45`

- **Every selected track is terminal and reconciled (BLOCKING).** Typed state,
  `evidence.json`, generated summary, human verdict, monitor record, source
  commit, and gate artifacts identify the same track and outcome.
- **Correctness covers the production path (BLOCKING).** Applicable component
  tests and the binding full-model comparator pass at the declared precision.
  Lossy work includes its required task-quality evidence. A component test may be
  skipped only with a valid mechanism-specific reason; full-model correctness
  may never be skipped.
- **The optimized mechanism actually runs (BLOCKING).** Claim-appropriate
  profiler artifacts prove the expected production path, buckets, and fallback.
  Nsys is normally sufficient for activation/ordering; counters are required
  only for counter-dependent claims.
- **Clean E2E is paired and complete (BLOCKING).** Baseline and optimized arms
  use the same source identity where required, environment, venv, launch,
  topology, workload, cache policy, and profiler-free timing method. Apply the
  canonical per-bucket verdicts and significance rules.
- **Projection and observation are reconcilable (HIGH).** Recompute the
  occurrence-weighted, non-overlapping production-boundary estimate using
  `f_e2e`. A material gap requires a causal explanation; proxy magnitude cannot
  retroactively justify an E2E claim.
- **Gating is proved, not inferred (BLOCKING when used).** The implemented
  boundary, exact eligible set/threshold, fallback, rollback control, compile
  behavior, and pre/post results support `GATED_PASS`. Non-monotonic results use
  exact-set gating.
- **Diluted ship is independently re-derived (BLOCKING when used).** Recompute
  every current `validation-defaults.md` DILUTED_PASS condition from raw sweep
  JSON. The generated `diluted_pass` block, `diluted:true`, PASS status, phase
  significance, and campaign accounting must agree. Do not double-count its
  phase win as another E2E contribution.
- **Monitor findings are resolved (HIGH/BLOCKING by consequence).** Continuous
  coverage spans the implementer's lifetime, and unresolved accuracy, scope,
  provenance, or activation findings remain visible to the final audit.

## After Stage 6-7 — `T_AUDIT_S67`

For EXHAUSTED, skip SHIP-only properties but verify the terminal integration
record and structured exhaustion memory.

- **The measured composition is the shipped composition (BLOCKING).** Selected
  commits/patches, any independently reviewed conflict resolution, final source
  tree, accepted environment, and integration evidence describe one exact set.
  Failed, exploratory, or unselected flags are not promoted.
- **Combined validation is production-valid (BLOCKING).** Full-model
  correctness, clean paired E2E, all required buckets, and separate activation
  profiling pass for the exact composition. A single-track short-circuit is
  valid only when its accepted patch and environment are unchanged.
- **No interaction is hidden (BLOCKING).** Merge residue, dispatch ordering,
  graph/compile behavior, return shapes, flag collisions, gating interactions,
  and original fallbacks are reviewed. Individual PASS results do not prove a
  multi-track composition.
- **Bookkeeping derives from primary results (BLOCKING).** Integration status,
  selected/shipped identities, dilution annotations, commit, and environment
  agree across state and artifacts. Cumulative improvement is the direct Round-1
  baseline/latest-integrated ratio, never multiplied round-local gains.
- **Baseline continuity is sound (BLOCKING).** Round 1 remains immutable. After
  SHIP, fresh golden references are captured from the promoted mainline and
  accepted environment for the next round.
- **Continuation belongs to Stage 7 (BLOCKING).** Stage 6 finalized the current
  round only. Threshold comparison, terminal token, mining invalidation, and any
  next-round creation follow the Stage 7 state/transition authorities.
- **Exhaustion is specific (HIGH).** Failed component/technology/shape/failure
  tuples retain evidence and expiry. They inform the next debate without acting
  as a blanket technology ban.

## Cross-Artifact Checks

- **Causality.** Stage timestamps, artifact mtimes, source commits, profiler
  captures, and verdict writes form a plausible order. Later evidence cannot
  validate earlier code without explicit rebinding.
- **Same-run pairing.** Flag different sessions, worktrees, venvs, cache modes,
  or stale baseline reuse. Require a paired rerun when the difference can explain
  the measured effect.
- **No contaminated authority.** Profiler, warmup, debug, partial, failed, and
  archived runs remain diagnostic and cannot supply clean timing or final
  correctness.
- **Round-to-round continuity.** Shipped identities and accepted environment are
  monotonic and complete; regressions or drift are investigated from direct
  measurements, not smoothed away.
- **Independent reconstruction wins.** When a renderer, state summary, delegate,
  or report disagrees with primary evidence, preserve the disagreement, fix the
  producer, and re-audit. Do not choose the convenient copy.

## Mechanical Authorities

- schema and state transitions: `.codex/schemas/state.schema.json`,
  `scripts/transitions.json`, and `scripts/ammo_state.py`
- structured validation: `scripts/reconcile_track_state.py` and
  `scripts/verify_validation_gates.py`
- verdict arithmetic and special paths: `validation-defaults.md`
- E2E projection: `e2e-delta-math.md`
- canonical paths and evidence precedence: `artifact-layout.md`
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall