Skip to content

amazon-science/ammo / ai_cli_session/.codex/skills/ammo

Profile and optimize GPU kernels for vLLM inference on NVIDIA GPUs. Use when targeting a specific model, hardware, dtype, parallelism, and workload to improve latency.

FileKindSize
RUNTIME.mdmarkdown4.9 KB
SKILL.mdskill md18.5 KB
agents/ammo-auditor.mdmarkdown9.1 KB
agents/ammo-champion.mdmarkdown9.6 KB
agents/ammo-delegate.mdmarkdown3.2 KB
agents/ammo-implementer.mdmarkdown13.8 KB
agents/ammo-investigator.mdmarkdown4.0 KB
agents/ammo-report.mdmarkdown2.9 KB
agents/ammo-researcher.mdmarkdown10.3 KB
agents/ammo-resolver.mdmarkdown4.4 KB
agents/ammo-transcript-monitor.mdmarkdown7.8 KB
agents/openai.yamlexecutablescript313 B
data/ATTRIBUTION.mdmarkdown1.4 KB
data/gsm8k_full.jsonexecutablescript735.5 KB
data/gsm8k_subset.jsonexecutablescript116.8 KB
eval/EVAL-SKILL.mdexecutablescript21.8 KB
eval/agents/transcript_grader.mdexecutablescript18.0 KB
eval/causal/__init__.pyexecutablescript137 B
eval/causal/agents/causal_analyzer.mdexecutablescript10.5 KB
eval/causal/build_dag.pyexecutablescript21.6 KB
eval/causal/diff_versions.pyexecutablescript38.5 KB
eval/causal/extract_events.pyexecutablescript36.3 KB
eval/causal/generate_postmortem.pyexecutablescript17.0 KB
eval/causal/score_nodes.pyexecutablescript21.5 KB
eval/causal/templates/narrative_template.mdexecutablescript472 B
eval/causal/templates/viz_template.htmlexecutablescript9.5 KB
eval/causal/tests/__init__.pyexecutablescript151 B
eval/causal/tests/conftest.pyexecutablescript448 B
eval/causal/tests/fixtures/sample_session.jsonlexecutablescript10.4 KB
eval/causal/tests/fixtures/sample_session_data.jsonexecutablescript3.4 KB
eval/causal/tests/fixtures/sample_snapshot.jsonexecutablescript2.5 KB
eval/causal/tests/test_build_dag.pyexecutablescript2.7 KB
eval/causal/tests/test_diff_versions.pyexecutablescript13.5 KB
eval/causal/tests/test_extract_events.pyexecutablescript3.7 KB
eval/causal/tests/test_generate_postmortem.pyexecutablescript3.1 KB
eval/causal/tests/test_score_nodes.pyexecutablescript3.3 KB
eval/scripts/aggregate_versions.pyexecutablescript15.2 KB
eval/scripts/archive_run.pyexecutablescript9.1 KB
eval/scripts/codex_session.pyexecutablescript10.6 KB
eval/scripts/generate_dashboard.pyexecutablescript5.3 KB
eval/scripts/parse_artifacts.pyexecutablescript48.1 KB
eval/scripts/parse_session_logs.pyexecutablescript51.3 KB
eval/scripts/rescore_archived.pyexecutablescript5.5 KB
eval/scripts/score_campaign.pyexecutablescript43.8 KB
eval/scripts/snapshot_changes.pyexecutablescript11.1 KB
eval/scripts/verify_citations.pyexecutablescript8.8 KB
eval/tests/test_dashboard_bugs.pyexecutablescript8.8 KB
eval/tests/test_parse_session_logs.pyexecutablescript3.9 KB
eval/tests/test_parse_session_logs_round_inference.pyexecutablescript11.0 KB
eval/tests/test_score_campaign_diluted.pyexecutablescript9.4 KB
eval/viewer/dashboard_template.htmlexecutablescript15.1 KB
instrumentation/sitecustomize.pyexecutablescript3.6 KB
orchestration/audit-protocol.mdmarkdown9.3 KB
orchestration/debate-protocol.mdmarkdown11.2 KB
orchestration/integration-logic.mdmarkdown10.3 KB
orchestration/parallel-tracks.mdmarkdown8.9 KB
references/artifact-layout.mdexecutablescript22.4 KB
references/audit-invariants.mdmarkdown10.6 KB
references/champion-common-patterns.mdmarkdown3.4 KB
references/crossover-probing.mdexecutablescript4.7 KB
references/cudagraph-safety.mdmarkdown2.7 KB
references/debate-rules.mdmarkdown4.7 KB
references/debate-scoring-rubric.mdexecutablescript16.7 KB
references/e2e-delta-math.mdexecutablescript17.9 KB
references/e2e-latency-guide.mdexecutablescript9.6 KB
references/fusion-feasibility-heuristics.mdmarkdown5.6 KB
references/gpu-pool.mdexecutablescript5.4 KB
references/impl-track-rules.mdexecutablescript13.8 KB
references/nsys-profiling-guide.mdmarkdown9.8 KB
references/optimization-categories.mdexecutablescript11.0 KB
references/shutdown-protocol.mdexecutablescript3.1 KB
references/technology-selection.mdmarkdown5.0 KB
references/torch-compile-contract.mdmarkdown13.8 KB
references/validation-defaults.mdexecutablescript34.2 KB
references/workload-invocation.mdexecutablescript4.9 KB
references/writing-style.mdexecutablescript6.9 KB
scripts/ammo_state.pyexecutablescript142.0 KB
scripts/artifact_layout.jsonconfig16.3 KB
scripts/check_projection_accuracy.pyexecutablescript11.7 KB
scripts/create_evidence_template.pyexecutablescript9.7 KB
scripts/create_worktree_with_build.shexecutablescript10.2 KB
scripts/generate_validation_report.pyexecutablescript35.7 KB
scripts/gpu_reservation.pyexecutablescript27.1 KB
scripts/hook_cmd_classify.pyscript13.3 KB
scripts/mine_trace.pyscript29.4 KB
scripts/monitor_queue_ack.pyexecutablescript6.1 KB
scripts/ncu_sanity.shexecutablescript5.2 KB
scripts/ncu_sanity_driver.pyexecutablescript8.5 KB
scripts/new_target.pyexecutablescript22.0 KB
scripts/preflight_vllm_api.pyexecutablescript12.3 KB
scripts/reconcile_track_state.pyexecutablescript15.8 KB
scripts/remove_worktree_cleanup.shexecutablescript1.1 KB
scripts/resolve_codex_transcript.pyexecutablescript3.6 KB
scripts/run_vllm_bench_latency_sweep.pyexecutablescript249.6 KB
scripts/transcript_filter.pyexecutablescript38.4 KB
scripts/transitions.jsonexecutablescript1.5 KB
scripts/verify_validation_gates.pyexecutablescript88.8 KB
tests/agents/test-champion.mdexecutablescript20.4 KB
tests/agents/test-impl-champion.mdexecutablescript14.3 KB
tests/agents/test-implementer.mdexecutablescript15.8 KB
tests/agents/test-orchestrator.mdexecutablescript33.8 KB
tests/agents/test-researcher.mdexecutablescript21.0 KB
tests/agents/test-transcript-monitor.mdexecutablescript13.4 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.mdmarkdown1.1 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.summary.jsonconfig582 B
tests/fixtures/diluted_pass/legacy_no_phase.jsonconfig400 B
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.mdmarkdown2.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.summary.jsonconfig1.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.jsonconfig626 B
tests/fixtures/mining/real_campaign_bottleneck_analysis.mdmarkdown20.7 KB
tests/run_all.shexecutablescript1.6 KB
tests/scripts/test_transcript_filter.pyexecutablescript41.1 KB
tests/test_agent_runtime_contracts.pyexecutablescript1.8 KB
tests/test_ammo_state.pyexecutablescript71.7 KB
tests/test_artifact_layout_manifest.pyscript11.6 KB
tests/test_audit_started.pyscript18.2 KB
tests/test_correctness_comparator.pyscript12.6 KB
tests/test_diluted_pass.pyscript24.4 KB
tests/test_drift_compat.pyscript5.1 KB
tests/test_evidence_freshness.pyexecutablescript5.6 KB
tests/test_fail_closed.pyexecutablescript14.7 KB
tests/test_gpu_reservation.pyexecutablescript24.8 KB
tests/test_gpu_reservation_session_id.pyexecutablescript11.2 KB
tests/test_guidance_semantic_contract.pyscript24.9 KB
tests/test_hook_cmd_classify_parity.pyscript11.7 KB
tests/test_hook_semantics.pyexecutablescript95.3 KB
tests/test_mine_trace.pyscript24.2 KB
tests/test_mining_invalidated_schema.pyexecutablescript9.4 KB
tests/test_new_target_dp_ep.pyexecutablescript7.4 KB
tests/test_new_target_v2_scaffold.pyexecutablescript13.1 KB
tests/test_parse_artifacts.pyexecutablescript3.0 KB
tests/test_policy_convergence.pyscript797 B
tests/test_preflight_vllm_api.pyscript13.3 KB
tests/test_projection_accuracy_v2.pyexecutablescript4.1 KB
tests/test_reconcile_track_state.pyexecutablescript18.2 KB
tests/test_resolve_codex_transcript.pyexecutablescript3.1 KB
tests/test_state_schema_semantics.pyexecutablescript7.6 KB
tests/test_status_set_sync.pyexecutablescript9.1 KB
tests/test_sweep_baseline_from_arm.pyscript11.0 KB
tests/test_sweep_decode_capture.pyexecutablescript31.8 KB
tests/test_sweep_dp.pyexecutablescript45.4 KB
tests/test_sweep_fresh_cache.pyscript5.5 KB
tests/test_sweep_profiler_guard.pyscript22.6 KB
tests/test_sweep_real_prompts.pyscript20.1 KB
tests/test_sweep_refactor.pyexecutablescript37.6 KB
tests/test_sweep_round_slot.pyexecutablescript7.4 KB
tests/test_validation_report_phase_triage.pyexecutablescript10.0 KB
tests/test_validation_report_v2.pyexecutablescript6.2 KB
tests/test_verify_validation_gates.pyexecutablescript52.9 KB
tests/test_workload_matrix.pyexecutablescript19.3 KB

tests/test_mining_invalidated_schema.py

View on GitHub
#!/usr/bin/env python3
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: CC-BY-NC-4.0
"""Z2-T1 / P4#7: schema legality of `mining_invalidated` (+ reason) round fields.

Pins the schema gap fix: the round object is `additionalProperties:false`, so
before P4#7 the engine reads `mining_invalidated` (ammo_state.advance, :1219)
that no schema-legal write could ever persist. This file adds the two optional
round properties and proves:

  1. a round carrying BOTH fields (boolean `true` + reason string) validates and
     round-trips through `advance --outcome EXHAUSTED` to 2_bottleneck_mining;
  2. the load-bearing boolean requirement holds at BOTH layers — the schema
     REJECTS the string form `"true"` (the `"type":"boolean"` declaration makes
     the misuse impossible to persist), and the in-process `advance` on a dict
     carrying the string routes to 3_debate (because `is True` is False for a
     string).

ADDITIVE: this is a NEW file. It does NOT import, edit, mutate, or weaken the
frozen tests/test_ammo_state.py (TestAdvance, test_exhausted_mining_invalidated_at_mining,
test_exhausted_without_invalidation_goes_to_debate). Helpers are mirrored locally
on purpose so this file is self-contained.

Run: python3 -m pytest tests/test_mining_invalidated_schema.py -q
"""

from __future__ import annotations

import copy
import json
import subprocess
import sys
from pathlib import Path

_SCRIPTS_DIR = Path(__file__).resolve().parent.parent / "scripts"
_ENGINE = _SCRIPTS_DIR / "ammo_state.py"
_SCHEMA = Path(__file__).resolve().parents[3] / "schemas" / "state.schema.json"

sys.path.insert(0, str(_SCRIPTS_DIR))
import ammo_state  # noqa: E402


# ─────────────────────────────────────────────
# Local fixtures (mirrored — NO import from the frozen test_ammo_state.py)
# ─────────────────────────────────────────────

def _round_skel(round_id, max_rounds=4):
    return {
        "round_id": round_id, "status": "IN_PROGRESS", "team_name": None,
        "profiling_baseline_path": None,
        "baseline": {"started_at": None, "completed_at": None, "e2e_latency": None, "per_bs_verdict": None},
        "bottleneck_mining": {"started_at": None, "completed_at": None, "top_bottleneck_share_pct": None},
        "debate": {"started_at": None, "completed_at": None, "candidates": [],
                   "rounds_completed": 0, "max_rounds": max_rounds, "selected_winners": []},
        "parallel_tracks": {"started_at": None, "completed_at": None, "tracks": {}},
        "integration": {"started_at": None, "completed_at": None, "status": "pending",
                        "passing_candidates": [], "failed_candidates": [], "selected_candidates": [],
                        "conflict_analysis": None, "combined_patch_branch": None, "combined_e2e_result": None,
                        "e2e_latency_combined": None, "per_bs_verdict": None, "commit_sha": None,
                        "final_decision": None, "resolver_invoked": None, "resolver_outcome": None,
                        "conflicting_tracks": None},
        "campaign_eval": {"started_at": None, "completed_at": None},
        "audit": {},
        "shipped": [], "dropped": [],
        "cumulative_speedup_after": None, "combined_e2e_speedup_x": None,
        "combined_e2e_delta_pp": None, "note": None, "round_summary": None,
    }


def _base_state():
    """Canonical valid v4.1 state — mirrors the bash harness base_v2 shape."""
    return {
        "target": {
            "model_id": "test-model", "hardware": "H100", "dtype": "bf16",
            "tp": 1, "dp": 1, "ep": 1, "component": "auto",
        },
        "session_id": None,
        "gpu_resources": {
            "gpu_count": 1, "gpu_model": "NVIDIA H100",
            "memory_total_gib": 80.0, "cuda_visible_devices": "0",
        },
        "campaign": {
            "schema_version": "4.1",
            "status": "active",
            "current_round": 1,
            "current_stage": "1_baseline",
            "config": {
                "min_e2e_improvement_pct": 1.0,
                "noise_tolerance_pct": 0.5,
                "catastrophic_regression_pct": 5.0,
            },
            "cumulative_speedup_vs_round1": 1.0,
            "round_1_baseline_latency_s": None,
            "shipped_optimizations": [],
            "agent_costs": [],
            "rounds": [_round_skel(1)],
        },
    }


def _audited_round1_state():
    """Round 1 at 7_campaign_eval with a full audit chain (incl stage_67) so the
    new-round-start gate is satisfied after advance creates round 2. Mirrored
    locally; NOT imported from the frozen test."""
    st = _base_state()
    st["campaign"]["current_stage"] = "7_campaign_eval"
    st["campaign"]["rounds"][0]["audit"] = {
        "stage_1": {"passed_at": "2026-06-08T14:56:31Z"},
        "stage_2": {"passed_at": "2026-06-08T15:48:49Z"},
        "stage_45": {"passed_at": "2026-06-08T18:26:35Z"},
        "stage_67": {"passed_at": "2026-06-08T18:42:23Z"},
    }
    return st


def _write_state(tmp_path, state):
    """Write a state.json inside a .codex/schemas-walkable artifact dir."""
    root = tmp_path
    (root / ".git").mkdir(exist_ok=True)
    (root / ".codex" / "schemas").mkdir(parents=True, exist_ok=True)
    (root / ".codex" / "schemas" / "state.schema.json").write_text(
        _SCHEMA.read_text(encoding="utf-8"), encoding="utf-8")
    art = root / "kernel_opt_artifacts" / "t"
    art.mkdir(parents=True, exist_ok=True)
    sp = art / "state.json"
    sp.write_text(json.dumps(state, indent=2), encoding="utf-8")
    return sp


def _run(*argv):
    return subprocess.run([sys.executable, str(_ENGINE), *argv],
                          capture_output=True, text=True)


def _schema_validates(state):
    """In-process schema check using the same Draft202012Validator the engine uses."""
    schema = json.loads(_SCHEMA.read_text(encoding="utf-8"))
    errs = ammo_state.schema_errors(state, schema)
    # schema_errors returns None only when jsonschema is unavailable (not here)
    assert errs is not None, "jsonschema must be present to run this test"
    return errs == []


# ─────────────────────────────────────────────
# Tests
# ─────────────────────────────────────────────

def test_round_with_both_fields_validates(tmp_path):
    """A round carrying mining_invalidated=true + reason string is schema-legal
    and passes the engine `validate` verb (exit 0 / PASS)."""
    st = _base_state()
    st["campaign"]["rounds"][0]["mining_invalidated"] = True
    st["campaign"]["rounds"][0]["mining_invalidated_reason"] = "wrong component attribution"
    sp = _write_state(tmp_path, st)
    r = _run("validate", "--state", str(sp))
    assert r.returncode == 0, r.stdout + r.stderr
    assert "PASS" in r.stdout, r.stdout


def test_advance_exhausted_mining_invalidated_file_roundtrip(tmp_path):
    """advance --outcome EXHAUSTED on an audited round-1 state carrying
    mining_invalidated=true persists (validates before write) and routes the new
    round to 2_bottleneck_mining (re-mine). The written doc still validates."""
    st = _audited_round1_state()
    st["campaign"]["rounds"][0]["mining_invalidated"] = True
    st["campaign"]["rounds"][0]["mining_invalidated_reason"] = "diagnosis was wrong"
    sp = _write_state(tmp_path, st)
    r = _run("advance", "--state", str(sp), "--outcome", "EXHAUSTED")
    assert r.returncode == 0, r.stdout + r.stderr
    doc = json.loads(sp.read_text())
    assert doc["campaign"]["current_stage"] == "2_bottleneck_mining", doc["campaign"]["current_stage"]
    assert doc["campaign"]["rounds"][0]["status"] == "EXHAUSTED"
    # the written doc (now carrying the optional fields) re-validates clean
    assert _schema_validates(doc), "advanced doc with mining_invalidated must validate"


def test_string_true_is_rejected_and_does_not_trigger_remine(tmp_path):
    """Boolean-sensitivity guard (team-lead ruling 3), two assertions:

    (i)  SCHEMA rejects the string form: mining_invalidated="true" (string) fails
         validate — the "type":"boolean" declaration makes the misuse impossible
         to persist (the strongest guarantee).
    (ii) ENGINE `is True` branch: advance(<dict-with-string-true>, EXHAUSTED)
         routes to 3_debate, NOT 2_bottleneck_mining, because
         `cur.get("mining_invalidated") is True` (ammo_state.py :1219) is False
         for a string.
    """
    # (i) schema rejects the string form
    st = _base_state()
    st["campaign"]["rounds"][0]["mining_invalidated"] = "true"  # string, not bool
    sp = _write_state(tmp_path, st)
    r = _run("validate", "--state", str(sp))
    assert r.returncode == 1, r.stdout + r.stderr
    assert "PASS" not in r.stdout, r.stdout
    assert not _schema_validates(st), "string 'true' must FAIL boolean type validation"

    # (ii) in-process advance with the string routes to 3_debate (is True is False)
    st2 = _base_state()
    st2["campaign"]["current_stage"] = "7_campaign_eval"
    st2["campaign"]["rounds"][0]["mining_invalidated"] = "true"  # string
    transitions = ammo_state.load_transitions()
    _, new_stage = ammo_state.advance(copy.deepcopy(st2), "EXHAUSTED", transitions)
    assert new_stage == "3_debate", new_stage
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall