Skip to content

amazon-science/ammo / ai_cli_session/.codex/skills/ammo

Profile and optimize GPU kernels for vLLM inference on NVIDIA GPUs. Use when targeting a specific model, hardware, dtype, parallelism, and workload to improve latency.

FileKindSize
RUNTIME.mdmarkdown4.9 KB
SKILL.mdskill md18.5 KB
agents/ammo-auditor.mdmarkdown9.1 KB
agents/ammo-champion.mdmarkdown9.6 KB
agents/ammo-delegate.mdmarkdown3.2 KB
agents/ammo-implementer.mdmarkdown13.8 KB
agents/ammo-investigator.mdmarkdown4.0 KB
agents/ammo-report.mdmarkdown2.9 KB
agents/ammo-researcher.mdmarkdown10.3 KB
agents/ammo-resolver.mdmarkdown4.4 KB
agents/ammo-transcript-monitor.mdmarkdown7.8 KB
agents/openai.yamlexecutablescript313 B
data/ATTRIBUTION.mdmarkdown1.4 KB
data/gsm8k_full.jsonexecutablescript735.5 KB
data/gsm8k_subset.jsonexecutablescript116.8 KB
eval/EVAL-SKILL.mdexecutablescript21.8 KB
eval/agents/transcript_grader.mdexecutablescript18.0 KB
eval/causal/__init__.pyexecutablescript137 B
eval/causal/agents/causal_analyzer.mdexecutablescript10.5 KB
eval/causal/build_dag.pyexecutablescript21.6 KB
eval/causal/diff_versions.pyexecutablescript38.5 KB
eval/causal/extract_events.pyexecutablescript36.3 KB
eval/causal/generate_postmortem.pyexecutablescript17.0 KB
eval/causal/score_nodes.pyexecutablescript21.5 KB
eval/causal/templates/narrative_template.mdexecutablescript472 B
eval/causal/templates/viz_template.htmlexecutablescript9.5 KB
eval/causal/tests/__init__.pyexecutablescript151 B
eval/causal/tests/conftest.pyexecutablescript448 B
eval/causal/tests/fixtures/sample_session.jsonlexecutablescript10.4 KB
eval/causal/tests/fixtures/sample_session_data.jsonexecutablescript3.4 KB
eval/causal/tests/fixtures/sample_snapshot.jsonexecutablescript2.5 KB
eval/causal/tests/test_build_dag.pyexecutablescript2.7 KB
eval/causal/tests/test_diff_versions.pyexecutablescript13.5 KB
eval/causal/tests/test_extract_events.pyexecutablescript3.7 KB
eval/causal/tests/test_generate_postmortem.pyexecutablescript3.1 KB
eval/causal/tests/test_score_nodes.pyexecutablescript3.3 KB
eval/scripts/aggregate_versions.pyexecutablescript15.2 KB
eval/scripts/archive_run.pyexecutablescript9.1 KB
eval/scripts/codex_session.pyexecutablescript10.6 KB
eval/scripts/generate_dashboard.pyexecutablescript5.3 KB
eval/scripts/parse_artifacts.pyexecutablescript48.1 KB
eval/scripts/parse_session_logs.pyexecutablescript51.3 KB
eval/scripts/rescore_archived.pyexecutablescript5.5 KB
eval/scripts/score_campaign.pyexecutablescript43.8 KB
eval/scripts/snapshot_changes.pyexecutablescript11.1 KB
eval/scripts/verify_citations.pyexecutablescript8.8 KB
eval/tests/test_dashboard_bugs.pyexecutablescript8.8 KB
eval/tests/test_parse_session_logs.pyexecutablescript3.9 KB
eval/tests/test_parse_session_logs_round_inference.pyexecutablescript11.0 KB
eval/tests/test_score_campaign_diluted.pyexecutablescript9.4 KB
eval/viewer/dashboard_template.htmlexecutablescript15.1 KB
instrumentation/sitecustomize.pyexecutablescript3.6 KB
orchestration/audit-protocol.mdmarkdown9.3 KB
orchestration/debate-protocol.mdmarkdown11.2 KB
orchestration/integration-logic.mdmarkdown10.3 KB
orchestration/parallel-tracks.mdmarkdown8.9 KB
references/artifact-layout.mdexecutablescript22.4 KB
references/audit-invariants.mdmarkdown10.6 KB
references/champion-common-patterns.mdmarkdown3.4 KB
references/crossover-probing.mdexecutablescript4.7 KB
references/cudagraph-safety.mdmarkdown2.7 KB
references/debate-rules.mdmarkdown4.7 KB
references/debate-scoring-rubric.mdexecutablescript16.7 KB
references/e2e-delta-math.mdexecutablescript17.9 KB
references/e2e-latency-guide.mdexecutablescript9.6 KB
references/fusion-feasibility-heuristics.mdmarkdown5.6 KB
references/gpu-pool.mdexecutablescript5.4 KB
references/impl-track-rules.mdexecutablescript13.8 KB
references/nsys-profiling-guide.mdmarkdown9.8 KB
references/optimization-categories.mdexecutablescript11.0 KB
references/shutdown-protocol.mdexecutablescript3.1 KB
references/technology-selection.mdmarkdown5.0 KB
references/torch-compile-contract.mdmarkdown13.8 KB
references/validation-defaults.mdexecutablescript34.2 KB
references/workload-invocation.mdexecutablescript4.9 KB
references/writing-style.mdexecutablescript6.9 KB
scripts/ammo_state.pyexecutablescript142.0 KB
scripts/artifact_layout.jsonconfig16.3 KB
scripts/check_projection_accuracy.pyexecutablescript11.7 KB
scripts/create_evidence_template.pyexecutablescript9.7 KB
scripts/create_worktree_with_build.shexecutablescript10.2 KB
scripts/generate_validation_report.pyexecutablescript35.7 KB
scripts/gpu_reservation.pyexecutablescript27.1 KB
scripts/hook_cmd_classify.pyscript13.3 KB
scripts/mine_trace.pyscript29.4 KB
scripts/monitor_queue_ack.pyexecutablescript6.1 KB
scripts/ncu_sanity.shexecutablescript5.2 KB
scripts/ncu_sanity_driver.pyexecutablescript8.5 KB
scripts/new_target.pyexecutablescript22.0 KB
scripts/preflight_vllm_api.pyexecutablescript12.3 KB
scripts/reconcile_track_state.pyexecutablescript15.8 KB
scripts/remove_worktree_cleanup.shexecutablescript1.1 KB
scripts/resolve_codex_transcript.pyexecutablescript3.6 KB
scripts/run_vllm_bench_latency_sweep.pyexecutablescript249.6 KB
scripts/transcript_filter.pyexecutablescript38.4 KB
scripts/transitions.jsonexecutablescript1.5 KB
scripts/verify_validation_gates.pyexecutablescript88.8 KB
tests/agents/test-champion.mdexecutablescript20.4 KB
tests/agents/test-impl-champion.mdexecutablescript14.3 KB
tests/agents/test-implementer.mdexecutablescript15.8 KB
tests/agents/test-orchestrator.mdexecutablescript33.8 KB
tests/agents/test-researcher.mdexecutablescript21.0 KB
tests/agents/test-transcript-monitor.mdexecutablescript13.4 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.mdmarkdown1.1 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.summary.jsonconfig582 B
tests/fixtures/diluted_pass/legacy_no_phase.jsonconfig400 B
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.mdmarkdown2.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.summary.jsonconfig1.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.jsonconfig626 B
tests/fixtures/mining/real_campaign_bottleneck_analysis.mdmarkdown20.7 KB
tests/run_all.shexecutablescript1.6 KB
tests/scripts/test_transcript_filter.pyexecutablescript41.1 KB
tests/test_agent_runtime_contracts.pyexecutablescript1.8 KB
tests/test_ammo_state.pyexecutablescript71.7 KB
tests/test_artifact_layout_manifest.pyscript11.6 KB
tests/test_audit_started.pyscript18.2 KB
tests/test_correctness_comparator.pyscript12.6 KB
tests/test_diluted_pass.pyscript24.4 KB
tests/test_drift_compat.pyscript5.1 KB
tests/test_evidence_freshness.pyexecutablescript5.6 KB
tests/test_fail_closed.pyexecutablescript14.7 KB
tests/test_gpu_reservation.pyexecutablescript24.8 KB
tests/test_gpu_reservation_session_id.pyexecutablescript11.2 KB
tests/test_guidance_semantic_contract.pyscript24.9 KB
tests/test_hook_cmd_classify_parity.pyscript11.7 KB
tests/test_hook_semantics.pyexecutablescript95.3 KB
tests/test_mine_trace.pyscript24.2 KB
tests/test_mining_invalidated_schema.pyexecutablescript9.4 KB
tests/test_new_target_dp_ep.pyexecutablescript7.4 KB
tests/test_new_target_v2_scaffold.pyexecutablescript13.1 KB
tests/test_parse_artifacts.pyexecutablescript3.0 KB
tests/test_policy_convergence.pyscript797 B
tests/test_preflight_vllm_api.pyscript13.3 KB
tests/test_projection_accuracy_v2.pyexecutablescript4.1 KB
tests/test_reconcile_track_state.pyexecutablescript18.2 KB
tests/test_resolve_codex_transcript.pyexecutablescript3.1 KB
tests/test_state_schema_semantics.pyexecutablescript7.6 KB
tests/test_status_set_sync.pyexecutablescript9.1 KB
tests/test_sweep_baseline_from_arm.pyscript11.0 KB
tests/test_sweep_decode_capture.pyexecutablescript31.8 KB
tests/test_sweep_dp.pyexecutablescript45.4 KB
tests/test_sweep_fresh_cache.pyscript5.5 KB
tests/test_sweep_profiler_guard.pyscript22.6 KB
tests/test_sweep_real_prompts.pyscript20.1 KB
tests/test_sweep_refactor.pyexecutablescript37.6 KB
tests/test_sweep_round_slot.pyexecutablescript7.4 KB
tests/test_validation_report_phase_triage.pyexecutablescript10.0 KB
tests/test_validation_report_v2.pyexecutablescript6.2 KB
tests/test_verify_validation_gates.pyexecutablescript52.9 KB
tests/test_workload_matrix.pyexecutablescript19.3 KB

tests/test_sweep_round_slot.py

View on GitHub
#!/usr/bin/env python3
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: CC-BY-NC-4.0
"""Round-scoped sweep path tests for run_vllm_bench_latency_sweep.py."""

from __future__ import annotations

import json
import sys
from pathlib import Path
from unittest import mock

import pytest

_SCRIPTS_DIR = Path(__file__).resolve().parent.parent / "scripts"
if str(_SCRIPTS_DIR) not in sys.path:
    sys.path.insert(0, str(_SCRIPTS_DIR))


def _make_v2_artifact(tmp_path: Path, current_round: int = 1) -> Path:
    artifact = tmp_path / "artifact"
    (artifact / "rounds" / str(current_round) / "sweeps").mkdir(parents=True)
    (artifact / "state.json").write_text(
        json.dumps({"campaign": {"current_round": current_round, "rounds": []}}),
        encoding="utf-8",
    )
    return artifact


def _make_legacy_artifact(tmp_path: Path) -> Path:
    artifact = tmp_path / "legacy"
    artifact.mkdir(parents=True)
    return artifact


class TestRoundSlotPathResolution:
    def _resolve(self, *, artifact: Path, round_=None, slot=None, out_name="e2e_latency", _out_root=None):
        from run_vllm_bench_latency_sweep import _resolve_sweep_out_name

        ns = mock.Mock(round=round_, slot=slot, out_name=out_name, _out_root=_out_root)
        return _resolve_sweep_out_name(ns, artifact)

    def test_round_and_slot_resolve_to_round_scoped_sweep(self, tmp_path):
        artifact = _make_v2_artifact(tmp_path)

        assert self._resolve(artifact=artifact, round_=1, slot="profiling") == "rounds/1/sweeps/profiling"
        assert self._resolve(artifact=artifact, round_=1, slot="opt/op007") == "rounds/1/sweeps/opt/op007"

    def test_slot_only_reads_current_round_from_state(self, tmp_path):
        artifact = _make_v2_artifact(tmp_path, current_round=3)

        assert self._resolve(artifact=artifact, slot="baseline") == "rounds/3/sweeps/baseline"

    def test_round_without_slot_fails(self, tmp_path):
        artifact = _make_v2_artifact(tmp_path)

        with pytest.raises(SystemExit, match="--round requires --slot"):
            self._resolve(artifact=artifact, round_=2)

    def test_custom_out_name_fails_on_v2_layout(self, tmp_path):
        artifact = _make_v2_artifact(tmp_path)

        with pytest.raises(SystemExit, match="--out-name is removed"):
            self._resolve(artifact=artifact, out_name="profiling")

    def test_legacy_layout_keeps_out_name(self, tmp_path):
        artifact = _make_legacy_artifact(tmp_path)

        assert self._resolve(artifact=artifact, out_name="profiling") == "profiling"


def test_prepare_out_root_archives_v2_slot_under_round_archive(tmp_path):
    from run_vllm_bench_latency_sweep import _prepare_out_root

    artifact = _make_v2_artifact(tmp_path)
    out = artifact / "rounds" / "1" / "sweeps" / "baseline"
    out.mkdir(parents=True, exist_ok=True)
    (out / "old.json").write_text("{}", encoding="utf-8")

    new_out = _prepare_out_root(
        artifact_dir=artifact,
        out_name="rounds/1/sweeps/baseline",
        overwrite=False,
    )

    assert new_out == out
    assert not (out / "old.json").exists()
    archived = list((artifact / "rounds" / "1" / "_archive").glob("baseline_*"))
    assert archived
    assert (archived[0] / "old.json").exists()


def test_all_stage5_sibling_slots_are_gate_slots():
    from run_vllm_bench_latency_sweep import _is_gate_slot

    for slot in (
        "opt/op007",
        "opt_correctness/op007",
        "opt_profiling/op007",
        "integration",
        "integration_profiling",
    ):
        assert _is_gate_slot(slot), slot
    for slot in (None, "baseline", "profiling", "golden_capture"):
        assert not _is_gate_slot(slot), slot


def test_stage5_profiling_dirs_are_track_scoped_while_stage1_stays_round_scoped(tmp_path):
    from run_vllm_bench_latency_sweep import _v2_profiling_dir

    round_root = tmp_path / "artifact" / "rounds" / "2"
    op1 = round_root / "sweeps" / "opt_profiling" / "op001"
    op2 = round_root / "sweeps" / "opt_profiling" / "op002"
    stage1 = round_root / "sweeps" / "profiling"
    post_ship = round_root / "sweeps" / "post_ship_profiling"
    integration = round_root / "sweeps" / "integration_profiling"

    assert _v2_profiling_dir(op1, "nsys") == round_root / "profiling" / "nsys" / "opt" / "op001"
    assert _v2_profiling_dir(op2, "nsys") == round_root / "profiling" / "nsys" / "opt" / "op002"
    assert _v2_profiling_dir(stage1, "nsys") == round_root / "profiling" / "nsys"
    assert _v2_profiling_dir(post_ship, "nsys") == round_root / "profiling" / "nsys" / "post_ship"
    assert _v2_profiling_dir(integration, "nsys") == round_root / "profiling" / "nsys" / "integration"
    assert _v2_profiling_dir(op1, "nsys") != _v2_profiling_dir(op2, "nsys")


def test_v2_profiling_dir_routes_outside_sweep_slot(tmp_path):
    from run_vllm_bench_latency_sweep import _v2_profiling_dir

    out_root = tmp_path / "artifact" / "rounds" / "2" / "sweeps" / "profiling"

    assert _v2_profiling_dir(out_root, "nsys") == tmp_path / "artifact" / "rounds" / "2" / "profiling" / "nsys"


def test_build_child_cmd_omits_out_name():
    from run_vllm_bench_latency_sweep import _build_child_cmd

    cmd = _build_child_cmd(
        python_exe="/venv/bin/python",
        script_path=Path("/sweep.py"),
        run_label="baseline",
        artifact_dir=Path("/art"),
        target_path=Path("/art/target.json"),
        timeout_s=1800,
        out_name="rounds/1/sweeps/baseline",
        out_root=Path("/art/rounds/1/sweeps/baseline"),
        dp=1,
        nproc=1,
        extra_child_flags=[],
    )

    assert "--out-name" not in cmd
    assert cmd[cmd.index("--_out-root") + 1] == "/art/rounds/1/sweeps/baseline"


def test_baseline_slot_rejects_profiling_flags_before_target_load(tmp_path):
    from run_vllm_bench_latency_sweep import main

    artifact = _make_v2_artifact(tmp_path)
    argv = [
        "sweep",
        "--artifact-dir",
        str(artifact),
        "--round",
        "1",
        "--slot",
        "baseline",
        "--labels",
        "baseline",
        "--nsys-profile",
    ]

    with mock.patch("sys.argv", argv), mock.patch("shutil.which", return_value="/usr/bin/nsys"):
        with pytest.raises(SystemExit, match="cannot be combined with --slot baseline"):
            main()


def test_clean_gate_slot_accepts_opt_only_with_stage1_baseline(tmp_path):
    """Stage-1 baseline reuse: opt-only + --baseline-from is the canonical
    Gate 5.3b invocation — the gate-slot guard must NOT reject it (a worktree
    baseline arm can execute the optimized path and contaminate both arms)."""
    from run_vllm_bench_latency_sweep import main

    artifact = _make_v2_artifact(tmp_path)
    imported = artifact / "rounds" / "1" / "sweeps" / "baseline"
    imported.mkdir(parents=True, exist_ok=True)
    argv = [
        "sweep",
        "--artifact-dir",
        str(artifact),
        "--round",
        "1",
        "--slot",
        "opt/OP-001",
        "--labels",
        "opt",
        "--baseline-from",
        str(imported),
    ]

    with mock.patch("sys.argv", argv):
        try:
            main()
        except SystemExit as exc:
            # The invocation proceeds past the label guard; the fixture has no
            # target.json so a later failure is expected — but it must not be
            # the old paired-A/B rejection.
            assert "requires --labels baseline,opt" not in str(exc)
        except Exception:
            pass  # later pipeline failure on the minimal fixture is fine
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall