Skip to content

amazon-science/ammo / ai_cli_session/.codex/skills/ammo

Profile and optimize GPU kernels for vLLM inference on NVIDIA GPUs. Use when targeting a specific model, hardware, dtype, parallelism, and workload to improve latency.

FileKindSize
RUNTIME.mdmarkdown4.9 KB
SKILL.mdskill md18.5 KB
agents/ammo-auditor.mdmarkdown9.1 KB
agents/ammo-champion.mdmarkdown9.6 KB
agents/ammo-delegate.mdmarkdown3.2 KB
agents/ammo-implementer.mdmarkdown13.8 KB
agents/ammo-investigator.mdmarkdown4.0 KB
agents/ammo-report.mdmarkdown2.9 KB
agents/ammo-researcher.mdmarkdown10.3 KB
agents/ammo-resolver.mdmarkdown4.4 KB
agents/ammo-transcript-monitor.mdmarkdown7.8 KB
agents/openai.yamlexecutablescript313 B
data/ATTRIBUTION.mdmarkdown1.4 KB
data/gsm8k_full.jsonexecutablescript735.5 KB
data/gsm8k_subset.jsonexecutablescript116.8 KB
eval/EVAL-SKILL.mdexecutablescript21.8 KB
eval/agents/transcript_grader.mdexecutablescript18.0 KB
eval/causal/__init__.pyexecutablescript137 B
eval/causal/agents/causal_analyzer.mdexecutablescript10.5 KB
eval/causal/build_dag.pyexecutablescript21.6 KB
eval/causal/diff_versions.pyexecutablescript38.5 KB
eval/causal/extract_events.pyexecutablescript36.3 KB
eval/causal/generate_postmortem.pyexecutablescript17.0 KB
eval/causal/score_nodes.pyexecutablescript21.5 KB
eval/causal/templates/narrative_template.mdexecutablescript472 B
eval/causal/templates/viz_template.htmlexecutablescript9.5 KB
eval/causal/tests/__init__.pyexecutablescript151 B
eval/causal/tests/conftest.pyexecutablescript448 B
eval/causal/tests/fixtures/sample_session.jsonlexecutablescript10.4 KB
eval/causal/tests/fixtures/sample_session_data.jsonexecutablescript3.4 KB
eval/causal/tests/fixtures/sample_snapshot.jsonexecutablescript2.5 KB
eval/causal/tests/test_build_dag.pyexecutablescript2.7 KB
eval/causal/tests/test_diff_versions.pyexecutablescript13.5 KB
eval/causal/tests/test_extract_events.pyexecutablescript3.7 KB
eval/causal/tests/test_generate_postmortem.pyexecutablescript3.1 KB
eval/causal/tests/test_score_nodes.pyexecutablescript3.3 KB
eval/scripts/aggregate_versions.pyexecutablescript15.2 KB
eval/scripts/archive_run.pyexecutablescript9.1 KB
eval/scripts/codex_session.pyexecutablescript10.6 KB
eval/scripts/generate_dashboard.pyexecutablescript5.3 KB
eval/scripts/parse_artifacts.pyexecutablescript48.1 KB
eval/scripts/parse_session_logs.pyexecutablescript51.3 KB
eval/scripts/rescore_archived.pyexecutablescript5.5 KB
eval/scripts/score_campaign.pyexecutablescript43.8 KB
eval/scripts/snapshot_changes.pyexecutablescript11.1 KB
eval/scripts/verify_citations.pyexecutablescript8.8 KB
eval/tests/test_dashboard_bugs.pyexecutablescript8.8 KB
eval/tests/test_parse_session_logs.pyexecutablescript3.9 KB
eval/tests/test_parse_session_logs_round_inference.pyexecutablescript11.0 KB
eval/tests/test_score_campaign_diluted.pyexecutablescript9.4 KB
eval/viewer/dashboard_template.htmlexecutablescript15.1 KB
instrumentation/sitecustomize.pyexecutablescript3.6 KB
orchestration/audit-protocol.mdmarkdown9.3 KB
orchestration/debate-protocol.mdmarkdown11.2 KB
orchestration/integration-logic.mdmarkdown10.3 KB
orchestration/parallel-tracks.mdmarkdown8.9 KB
references/artifact-layout.mdexecutablescript22.4 KB
references/audit-invariants.mdmarkdown10.6 KB
references/champion-common-patterns.mdmarkdown3.4 KB
references/crossover-probing.mdexecutablescript4.7 KB
references/cudagraph-safety.mdmarkdown2.7 KB
references/debate-rules.mdmarkdown4.7 KB
references/debate-scoring-rubric.mdexecutablescript16.7 KB
references/e2e-delta-math.mdexecutablescript17.9 KB
references/e2e-latency-guide.mdexecutablescript9.6 KB
references/fusion-feasibility-heuristics.mdmarkdown5.6 KB
references/gpu-pool.mdexecutablescript5.4 KB
references/impl-track-rules.mdexecutablescript13.8 KB
references/nsys-profiling-guide.mdmarkdown9.8 KB
references/optimization-categories.mdexecutablescript11.0 KB
references/shutdown-protocol.mdexecutablescript3.1 KB
references/technology-selection.mdmarkdown5.0 KB
references/torch-compile-contract.mdmarkdown13.8 KB
references/validation-defaults.mdexecutablescript34.2 KB
references/workload-invocation.mdexecutablescript4.9 KB
references/writing-style.mdexecutablescript6.9 KB
scripts/ammo_state.pyexecutablescript142.0 KB
scripts/artifact_layout.jsonconfig16.3 KB
scripts/check_projection_accuracy.pyexecutablescript11.7 KB
scripts/create_evidence_template.pyexecutablescript9.7 KB
scripts/create_worktree_with_build.shexecutablescript10.2 KB
scripts/generate_validation_report.pyexecutablescript35.7 KB
scripts/gpu_reservation.pyexecutablescript27.1 KB
scripts/hook_cmd_classify.pyscript13.3 KB
scripts/mine_trace.pyscript29.4 KB
scripts/monitor_queue_ack.pyexecutablescript6.1 KB
scripts/ncu_sanity.shexecutablescript5.2 KB
scripts/ncu_sanity_driver.pyexecutablescript8.5 KB
scripts/new_target.pyexecutablescript22.0 KB
scripts/preflight_vllm_api.pyexecutablescript12.3 KB
scripts/reconcile_track_state.pyexecutablescript15.8 KB
scripts/remove_worktree_cleanup.shexecutablescript1.1 KB
scripts/resolve_codex_transcript.pyexecutablescript3.6 KB
scripts/run_vllm_bench_latency_sweep.pyexecutablescript249.6 KB
scripts/transcript_filter.pyexecutablescript38.4 KB
scripts/transitions.jsonexecutablescript1.5 KB
scripts/verify_validation_gates.pyexecutablescript88.8 KB
tests/agents/test-champion.mdexecutablescript20.4 KB
tests/agents/test-impl-champion.mdexecutablescript14.3 KB
tests/agents/test-implementer.mdexecutablescript15.8 KB
tests/agents/test-orchestrator.mdexecutablescript33.8 KB
tests/agents/test-researcher.mdexecutablescript21.0 KB
tests/agents/test-transcript-monitor.mdexecutablescript13.4 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.mdmarkdown1.1 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.summary.jsonconfig582 B
tests/fixtures/diluted_pass/legacy_no_phase.jsonconfig400 B
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.mdmarkdown2.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.summary.jsonconfig1.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.jsonconfig626 B
tests/fixtures/mining/real_campaign_bottleneck_analysis.mdmarkdown20.7 KB
tests/run_all.shexecutablescript1.6 KB
tests/scripts/test_transcript_filter.pyexecutablescript41.1 KB
tests/test_agent_runtime_contracts.pyexecutablescript1.8 KB
tests/test_ammo_state.pyexecutablescript71.7 KB
tests/test_artifact_layout_manifest.pyscript11.6 KB
tests/test_audit_started.pyscript18.2 KB
tests/test_correctness_comparator.pyscript12.6 KB
tests/test_diluted_pass.pyscript24.4 KB
tests/test_drift_compat.pyscript5.1 KB
tests/test_evidence_freshness.pyexecutablescript5.6 KB
tests/test_fail_closed.pyexecutablescript14.7 KB
tests/test_gpu_reservation.pyexecutablescript24.8 KB
tests/test_gpu_reservation_session_id.pyexecutablescript11.2 KB
tests/test_guidance_semantic_contract.pyscript24.9 KB
tests/test_hook_cmd_classify_parity.pyscript11.7 KB
tests/test_hook_semantics.pyexecutablescript95.3 KB
tests/test_mine_trace.pyscript24.2 KB
tests/test_mining_invalidated_schema.pyexecutablescript9.4 KB
tests/test_new_target_dp_ep.pyexecutablescript7.4 KB
tests/test_new_target_v2_scaffold.pyexecutablescript13.1 KB
tests/test_parse_artifacts.pyexecutablescript3.0 KB
tests/test_policy_convergence.pyscript797 B
tests/test_preflight_vllm_api.pyscript13.3 KB
tests/test_projection_accuracy_v2.pyexecutablescript4.1 KB
tests/test_reconcile_track_state.pyexecutablescript18.2 KB
tests/test_resolve_codex_transcript.pyexecutablescript3.1 KB
tests/test_state_schema_semantics.pyexecutablescript7.6 KB
tests/test_status_set_sync.pyexecutablescript9.1 KB
tests/test_sweep_baseline_from_arm.pyscript11.0 KB
tests/test_sweep_decode_capture.pyexecutablescript31.8 KB
tests/test_sweep_dp.pyexecutablescript45.4 KB
tests/test_sweep_fresh_cache.pyscript5.5 KB
tests/test_sweep_profiler_guard.pyscript22.6 KB
tests/test_sweep_real_prompts.pyscript20.1 KB
tests/test_sweep_refactor.pyexecutablescript37.6 KB
tests/test_sweep_round_slot.pyexecutablescript7.4 KB
tests/test_validation_report_phase_triage.pyexecutablescript10.0 KB
tests/test_validation_report_v2.pyexecutablescript6.2 KB
tests/test_verify_validation_gates.pyexecutablescript52.9 KB
tests/test_workload_matrix.pyexecutablescript19.3 KB

tests/test_gpu_reservation_session_id.py

View on GitHub
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: CC-BY-NC-4.0
"""Tests for GPU reservation session_id and auto-release fixes.

Bug 1: shared default session_id ("cli") causes auto-eviction between parallel agents
Bug 2: hooks and CLI reservations must agree on the session-id default chain.
"""

import json
import os
import subprocess
import sys
import tempfile
from pathlib import Path
from unittest import mock

import pytest

# The module under test. Live at skills/ammo/scripts/; make it importable
# regardless of the pytest invocation cwd (matches tests/test_sweep_dp.py).
_SCRIPTS_DIR = Path(__file__).resolve().parent.parent / "scripts"
if str(_SCRIPTS_DIR) not in sys.path:
    sys.path.insert(0, str(_SCRIPTS_DIR))
SCRIPT_PATH = _SCRIPTS_DIR / "gpu_reservation.py"


@pytest.fixture(autouse=True)
def isolated_state(tmp_path, monkeypatch):
    """Each test gets its own state directory and a 2-GPU pool."""
    state_dir = tmp_path / "gpu_res"
    state_dir.mkdir()
    monkeypatch.setattr("gpu_reservation.STATE_DIR", state_dir)
    # Suppress real nvidia-smi calls: seed with 2 GPUs
    state = {"gpus": {"0": None, "1": None}, "gpu_count": 2, "audit": []}
    (state_dir / "state.json").write_text(json.dumps(state))
    # Fast lock retries for tests
    monkeypatch.setattr("gpu_reservation.BACKOFF_DELAYS", [0.01])
    return state_dir


# Import after fixture definition so monkeypatch can apply
import gpu_reservation  # noqa: E402
from gpu_reservation import ReservationError, reserve, release_by_session  # noqa: E402


# ---------------------------------------------------------------------------
# Bug 1: Two agents with same default session_id auto-evict each other
# ---------------------------------------------------------------------------

class TestAutoEvictionBug:
    """Reserve auto-releases previous reservations for the same session_id.
    When two agents both use "cli", agent B evicts agent A."""

    def test_same_session_id_causes_eviction(self):
        """Demonstrates the bug: agent A's reservation is silently released."""
        # Agent A reserves GPU 0
        result_a = reserve(num_gpus=1, session_id="cli")
        assert result_a == [0]

        # Agent B reserves GPU — auto-release frees agent A's GPU first
        result_b = reserve(num_gpus=1, session_id="cli")
        # Bug: B gets GPU 0 (A's was freed) instead of GPU 1
        assert result_b == [0], "Same session_id causes eviction — B took A's GPU"

    def test_different_session_ids_no_eviction(self):
        """With unique session_ids, agents don't evict each other."""
        result_a = reserve(num_gpus=1, session_id="session-aaa")
        assert result_a == [0]

        result_b = reserve(num_gpus=1, session_id="session-bbb")
        assert result_b == [1], "Different session_ids should not evict"

    def test_completion_release_for_one_unique_agent_does_not_free_sibling(self):
        """PostToolUse-style release is safe only with per-agent session ids."""
        reserve(num_gpus=1, session_id="campaign-op007-agent-a", auto_release=False)
        reserve(num_gpus=1, session_id="campaign-op007-agent-b", auto_release=False)

        released = release_by_session("campaign-op007-agent-a")

        assert released == [0]
        state = gpu_reservation.read_state()
        assert state["gpus"]["0"] is None
        assert state["gpus"]["1"]["session_id"] == "campaign-op007-agent-b"

    def test_no_auto_release_prevents_eviction(self):
        """--no-auto-release flag should skip the auto-release step."""
        # Agent A reserves GPU 0
        reserve(num_gpus=1, session_id="shared-session")

        # Agent B reserves with no_auto_release=True — should get GPU 1, not 0
        result_b = reserve(
            num_gpus=1, session_id="shared-session", auto_release=False
        )
        assert result_b == [1], "no_auto_release should skip eviction"

    def test_no_auto_release_fails_when_pool_exhausted(self):
        """With no_auto_release, if pool is full, should raise ReservationError."""
        reserve(num_gpus=1, session_id="shared-session")
        reserve(num_gpus=1, session_id="shared-session", auto_release=False)

        # Pool is now full (2 GPUs, both reserved)
        with pytest.raises(ReservationError, match="Not enough free GPUs"):
            reserve(num_gpus=1, session_id="shared-session", auto_release=False)

    def test_auto_release_default_still_works_for_retries(self):
        """Default behavior (auto_release=True) still works for retry scenarios."""
        # Agent reserves GPU 0
        reserve(num_gpus=1, session_id="retry-session")

        # Same agent retries — auto-release frees old reservation first
        result = reserve(num_gpus=1, session_id="retry-session")
        assert result == [0], "Auto-release should still work by default for retries"


# ---------------------------------------------------------------------------
# Bug 2: CLI default session_id should follow the session-id env chain
# ---------------------------------------------------------------------------

class TestCLISessionIdDefault:
    """CLI --session-id default follows $AMMO_SESSION_ID, then $CLAUDE_SESSION_ID."""

    def _run_cli(self, args: list[str], env_override: dict | None = None):
        """Run gpu_reservation.py as a subprocess with controlled env."""
        env = os.environ.copy()
        # Point at the test's isolated state dir
        env["AMMO_GPU_RES_DIR"] = str(gpu_reservation.STATE_DIR)
        if env_override:
            env.update(env_override)
        return subprocess.run(
            [sys.executable, str(SCRIPT_PATH)] + args,
            capture_output=True,
            text=True,
            env=env,
            timeout=10,
        )

    def test_cli_uses_ammo_session_id_env(self):
        """When AMMO_SESSION_ID is set, reserve should use it as session_id."""
        result = self._run_cli(
            ["reserve", "--num-gpus", "1"],
            env_override={"AMMO_SESSION_ID": "agent-xyz-123"},
        )
        assert result.returncode == 0
        gpu_id = result.stdout.strip()
        assert gpu_id == "0"

        # Verify the reservation was stored with the env var session_id
        state = json.loads(
            (gpu_reservation.STATE_DIR / "state.json").read_text()
        )
        assert state["gpus"]["0"]["session_id"] == "agent-xyz-123"

    def test_cli_uses_claude_session_id_when_ammo_unset(self):
        """CLAUDE_SESSION_ID is the runtime fallback when AMMO_SESSION_ID is absent."""
        env = os.environ.copy()
        env["AMMO_GPU_RES_DIR"] = str(gpu_reservation.STATE_DIR)
        env.pop("AMMO_SESSION_ID", None)
        env["CLAUDE_SESSION_ID"] = "claude-fallback-id"
        result = subprocess.run(
            [sys.executable, str(SCRIPT_PATH), "reserve", "--num-gpus", "1"],
            capture_output=True,
            text=True,
            env=env,
            timeout=10,
        )
        assert result.returncode == 0

        state = json.loads(
            (gpu_reservation.STATE_DIR / "state.json").read_text()
        )
        assert state["gpus"]["0"]["session_id"] == "claude-fallback-id"

    def test_ammo_session_id_wins_over_claude_session_id(self):
        """AMMO_SESSION_ID is the portable value the server injects; it wins."""
        result = self._run_cli(
            ["reserve", "--num-gpus", "1"],
            env_override={
                "AMMO_SESSION_ID": "ammo-id",
                "CLAUDE_SESSION_ID": "claude-id",
            },
        )
        assert result.returncode == 0

        state = json.loads(
            (gpu_reservation.STATE_DIR / "state.json").read_text()
        )
        assert state["gpus"]["0"]["session_id"] == "ammo-id"

    def test_cli_falls_back_to_cli_without_env(self):
        """Without either env var, default should still be 'cli'."""
        env = os.environ.copy()
        env["AMMO_GPU_RES_DIR"] = str(gpu_reservation.STATE_DIR)
        env.pop("AMMO_SESSION_ID", None)
        env.pop("CLAUDE_SESSION_ID", None)
        result = subprocess.run(
            [sys.executable, str(SCRIPT_PATH), "reserve", "--num-gpus", "1"],
            capture_output=True,
            text=True,
            env=env,
            timeout=10,
        )
        assert result.returncode == 0

        state = json.loads(
            (gpu_reservation.STATE_DIR / "state.json").read_text()
        )
        assert state["gpus"]["0"]["session_id"] == "cli"

    def test_cli_explicit_session_id_overrides_env(self):
        """Explicit --session-id should override the env var default."""
        result = self._run_cli(
            ["reserve", "--num-gpus", "1", "--session-id", "explicit-id"],
            env_override={"AMMO_SESSION_ID": "agent-xyz-123"},
        )
        assert result.returncode == 0

        state = json.loads(
            (gpu_reservation.STATE_DIR / "state.json").read_text()
        )
        assert state["gpus"]["0"]["session_id"] == "explicit-id"

    def test_cli_no_auto_release_flag(self):
        """CLI --no-auto-release flag should be accepted and skip auto-release."""
        # First reservation
        self._run_cli(
            ["reserve", "--num-gpus", "1"],
            env_override={"AMMO_SESSION_ID": "shared-agent"},
        )

        # Second reservation with --no-auto-release
        result = self._run_cli(
            ["reserve", "--num-gpus", "1", "--no-auto-release"],
            env_override={"AMMO_SESSION_ID": "shared-agent"},
        )
        assert result.returncode == 0
        gpu_id = result.stdout.strip()
        assert gpu_id == "1", "Should get GPU 1, not re-take GPU 0"

        # Both GPUs should be reserved
        state = json.loads(
            (gpu_reservation.STATE_DIR / "state.json").read_text()
        )
        assert state["gpus"]["0"] is not None
        assert state["gpus"]["1"] is not None


# ---------------------------------------------------------------------------
# Hook compatibility: release-session matches the env session id
# ---------------------------------------------------------------------------

class TestHookCompatibility:
    """The PostToolUse hook releases by the session-id env chain.
    After the fix, reserve uses the same value, so release-session matches."""

    def test_hook_release_matches_env_session_id(self):
        """Reserve via CLI with the env session id, then release-session matches."""
        env = os.environ.copy()
        env["AMMO_GPU_RES_DIR"] = str(gpu_reservation.STATE_DIR)
        env["AMMO_SESSION_ID"] = "hook-test-session"

        # Reserve (uses the env session id as default)
        r1 = subprocess.run(
            [sys.executable, str(SCRIPT_PATH), "reserve", "--num-gpus", "1"],
            capture_output=True, text=True, env=env, timeout=10,
        )
        assert r1.returncode == 0

        # Release by session (what the hook does)
        r2 = subprocess.run(
            [sys.executable, str(SCRIPT_PATH), "release-session",
             "--session-id", "hook-test-session"],
            capture_output=True, text=True, env=env, timeout=10,
        )
        assert r2.returncode == 0

        # GPU should be free now
        state = json.loads(
            (gpu_reservation.STATE_DIR / "state.json").read_text()
        )
        assert state["gpus"]["0"] is None, "Hook release should free the GPU"
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall