Skip to content

amazon-science/ammo / ai_cli_session/.codex/skills/ammo

Profile and optimize GPU kernels for vLLM inference on NVIDIA GPUs. Use when targeting a specific model, hardware, dtype, parallelism, and workload to improve latency.

FileKindSize
RUNTIME.mdmarkdown4.9 KB
SKILL.mdskill md18.5 KB
agents/ammo-auditor.mdmarkdown9.1 KB
agents/ammo-champion.mdmarkdown9.6 KB
agents/ammo-delegate.mdmarkdown3.2 KB
agents/ammo-implementer.mdmarkdown13.8 KB
agents/ammo-investigator.mdmarkdown4.0 KB
agents/ammo-report.mdmarkdown2.9 KB
agents/ammo-researcher.mdmarkdown10.3 KB
agents/ammo-resolver.mdmarkdown4.4 KB
agents/ammo-transcript-monitor.mdmarkdown7.8 KB
agents/openai.yamlexecutablescript313 B
data/ATTRIBUTION.mdmarkdown1.4 KB
data/gsm8k_full.jsonexecutablescript735.5 KB
data/gsm8k_subset.jsonexecutablescript116.8 KB
eval/EVAL-SKILL.mdexecutablescript21.8 KB
eval/agents/transcript_grader.mdexecutablescript18.0 KB
eval/causal/__init__.pyexecutablescript137 B
eval/causal/agents/causal_analyzer.mdexecutablescript10.5 KB
eval/causal/build_dag.pyexecutablescript21.6 KB
eval/causal/diff_versions.pyexecutablescript38.5 KB
eval/causal/extract_events.pyexecutablescript36.3 KB
eval/causal/generate_postmortem.pyexecutablescript17.0 KB
eval/causal/score_nodes.pyexecutablescript21.5 KB
eval/causal/templates/narrative_template.mdexecutablescript472 B
eval/causal/templates/viz_template.htmlexecutablescript9.5 KB
eval/causal/tests/__init__.pyexecutablescript151 B
eval/causal/tests/conftest.pyexecutablescript448 B
eval/causal/tests/fixtures/sample_session.jsonlexecutablescript10.4 KB
eval/causal/tests/fixtures/sample_session_data.jsonexecutablescript3.4 KB
eval/causal/tests/fixtures/sample_snapshot.jsonexecutablescript2.5 KB
eval/causal/tests/test_build_dag.pyexecutablescript2.7 KB
eval/causal/tests/test_diff_versions.pyexecutablescript13.5 KB
eval/causal/tests/test_extract_events.pyexecutablescript3.7 KB
eval/causal/tests/test_generate_postmortem.pyexecutablescript3.1 KB
eval/causal/tests/test_score_nodes.pyexecutablescript3.3 KB
eval/scripts/aggregate_versions.pyexecutablescript15.2 KB
eval/scripts/archive_run.pyexecutablescript9.1 KB
eval/scripts/codex_session.pyexecutablescript10.6 KB
eval/scripts/generate_dashboard.pyexecutablescript5.3 KB
eval/scripts/parse_artifacts.pyexecutablescript48.1 KB
eval/scripts/parse_session_logs.pyexecutablescript51.3 KB
eval/scripts/rescore_archived.pyexecutablescript5.5 KB
eval/scripts/score_campaign.pyexecutablescript43.8 KB
eval/scripts/snapshot_changes.pyexecutablescript11.1 KB
eval/scripts/verify_citations.pyexecutablescript8.8 KB
eval/tests/test_dashboard_bugs.pyexecutablescript8.8 KB
eval/tests/test_parse_session_logs.pyexecutablescript3.9 KB
eval/tests/test_parse_session_logs_round_inference.pyexecutablescript11.0 KB
eval/tests/test_score_campaign_diluted.pyexecutablescript9.4 KB
eval/viewer/dashboard_template.htmlexecutablescript15.1 KB
instrumentation/sitecustomize.pyexecutablescript3.6 KB
orchestration/audit-protocol.mdmarkdown9.3 KB
orchestration/debate-protocol.mdmarkdown11.2 KB
orchestration/integration-logic.mdmarkdown10.3 KB
orchestration/parallel-tracks.mdmarkdown8.9 KB
references/artifact-layout.mdexecutablescript22.4 KB
references/audit-invariants.mdmarkdown10.6 KB
references/champion-common-patterns.mdmarkdown3.4 KB
references/crossover-probing.mdexecutablescript4.7 KB
references/cudagraph-safety.mdmarkdown2.7 KB
references/debate-rules.mdmarkdown4.7 KB
references/debate-scoring-rubric.mdexecutablescript16.7 KB
references/e2e-delta-math.mdexecutablescript17.9 KB
references/e2e-latency-guide.mdexecutablescript9.6 KB
references/fusion-feasibility-heuristics.mdmarkdown5.6 KB
references/gpu-pool.mdexecutablescript5.4 KB
references/impl-track-rules.mdexecutablescript13.8 KB
references/nsys-profiling-guide.mdmarkdown9.8 KB
references/optimization-categories.mdexecutablescript11.0 KB
references/shutdown-protocol.mdexecutablescript3.1 KB
references/technology-selection.mdmarkdown5.0 KB
references/torch-compile-contract.mdmarkdown13.8 KB
references/validation-defaults.mdexecutablescript34.2 KB
references/workload-invocation.mdexecutablescript4.9 KB
references/writing-style.mdexecutablescript6.9 KB
scripts/ammo_state.pyexecutablescript142.0 KB
scripts/artifact_layout.jsonconfig16.3 KB
scripts/check_projection_accuracy.pyexecutablescript11.7 KB
scripts/create_evidence_template.pyexecutablescript9.7 KB
scripts/create_worktree_with_build.shexecutablescript10.2 KB
scripts/generate_validation_report.pyexecutablescript35.7 KB
scripts/gpu_reservation.pyexecutablescript27.1 KB
scripts/hook_cmd_classify.pyscript13.3 KB
scripts/mine_trace.pyscript29.4 KB
scripts/monitor_queue_ack.pyexecutablescript6.1 KB
scripts/ncu_sanity.shexecutablescript5.2 KB
scripts/ncu_sanity_driver.pyexecutablescript8.5 KB
scripts/new_target.pyexecutablescript22.0 KB
scripts/preflight_vllm_api.pyexecutablescript12.3 KB
scripts/reconcile_track_state.pyexecutablescript15.8 KB
scripts/remove_worktree_cleanup.shexecutablescript1.1 KB
scripts/resolve_codex_transcript.pyexecutablescript3.6 KB
scripts/run_vllm_bench_latency_sweep.pyexecutablescript249.6 KB
scripts/transcript_filter.pyexecutablescript38.4 KB
scripts/transitions.jsonexecutablescript1.5 KB
scripts/verify_validation_gates.pyexecutablescript88.8 KB
tests/agents/test-champion.mdexecutablescript20.4 KB
tests/agents/test-impl-champion.mdexecutablescript14.3 KB
tests/agents/test-implementer.mdexecutablescript15.8 KB
tests/agents/test-orchestrator.mdexecutablescript33.8 KB
tests/agents/test-researcher.mdexecutablescript21.0 KB
tests/agents/test-transcript-monitor.mdexecutablescript13.4 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.mdmarkdown1.1 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.summary.jsonconfig582 B
tests/fixtures/diluted_pass/legacy_no_phase.jsonconfig400 B
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.mdmarkdown2.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.summary.jsonconfig1.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.jsonconfig626 B
tests/fixtures/mining/real_campaign_bottleneck_analysis.mdmarkdown20.7 KB
tests/run_all.shexecutablescript1.6 KB
tests/scripts/test_transcript_filter.pyexecutablescript41.1 KB
tests/test_agent_runtime_contracts.pyexecutablescript1.8 KB
tests/test_ammo_state.pyexecutablescript71.7 KB
tests/test_artifact_layout_manifest.pyscript11.6 KB
tests/test_audit_started.pyscript18.2 KB
tests/test_correctness_comparator.pyscript12.6 KB
tests/test_diluted_pass.pyscript24.4 KB
tests/test_drift_compat.pyscript5.1 KB
tests/test_evidence_freshness.pyexecutablescript5.6 KB
tests/test_fail_closed.pyexecutablescript14.7 KB
tests/test_gpu_reservation.pyexecutablescript24.8 KB
tests/test_gpu_reservation_session_id.pyexecutablescript11.2 KB
tests/test_guidance_semantic_contract.pyscript24.9 KB
tests/test_hook_cmd_classify_parity.pyscript11.7 KB
tests/test_hook_semantics.pyexecutablescript95.3 KB
tests/test_mine_trace.pyscript24.2 KB
tests/test_mining_invalidated_schema.pyexecutablescript9.4 KB
tests/test_new_target_dp_ep.pyexecutablescript7.4 KB
tests/test_new_target_v2_scaffold.pyexecutablescript13.1 KB
tests/test_parse_artifacts.pyexecutablescript3.0 KB
tests/test_policy_convergence.pyscript797 B
tests/test_preflight_vllm_api.pyscript13.3 KB
tests/test_projection_accuracy_v2.pyexecutablescript4.1 KB
tests/test_reconcile_track_state.pyexecutablescript18.2 KB
tests/test_resolve_codex_transcript.pyexecutablescript3.1 KB
tests/test_state_schema_semantics.pyexecutablescript7.6 KB
tests/test_status_set_sync.pyexecutablescript9.1 KB
tests/test_sweep_baseline_from_arm.pyscript11.0 KB
tests/test_sweep_decode_capture.pyexecutablescript31.8 KB
tests/test_sweep_dp.pyexecutablescript45.4 KB
tests/test_sweep_fresh_cache.pyscript5.5 KB
tests/test_sweep_profiler_guard.pyscript22.6 KB
tests/test_sweep_real_prompts.pyscript20.1 KB
tests/test_sweep_refactor.pyexecutablescript37.6 KB
tests/test_sweep_round_slot.pyexecutablescript7.4 KB
tests/test_validation_report_phase_triage.pyexecutablescript10.0 KB
tests/test_validation_report_v2.pyexecutablescript6.2 KB
tests/test_verify_validation_gates.pyexecutablescript52.9 KB
tests/test_workload_matrix.pyexecutablescript19.3 KB

eval/scripts/archive_run.py

View on GitHub
#!/usr/bin/env python3
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: CC-BY-NC-4.0
"""Archive a scored AMMO eval run into the versioned eval repository.

Creates the repository structure if it doesn't exist. Each run is stored
under versions/{version_id}/runs/{target_slug}/run_{N}/.

Version IDs are derived as: {git_short_hash}_{slugified_description}

Usage:
  python archive_run.py \
    --scorecard scorecard.json \
    --snapshot artifacts_snapshot.json \
    --description "Improve debate convergence" \
    --repository ~/.codex/ammo-eval
"""

from __future__ import annotations

import argparse
import json
import re
import shutil
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, Optional


def _slugify(text: str, max_len: int = 40) -> str:
    """Convert text to a URL-safe slug."""
    text = text.lower().strip()
    text = re.sub(r"[^a-z0-9\s-]", "", text)
    text = re.sub(r"[\s_]+", "-", text)
    text = re.sub(r"-+", "-", text).strip("-")
    return text[:max_len]


def _git_short_hash() -> Optional[str]:
    """Get the short git commit hash of the current HEAD."""
    try:
        result = subprocess.run(
            ["git", "rev-parse", "--short", "HEAD"],
            capture_output=True, text=True, timeout=10,
        )
        if result.returncode == 0:
            return result.stdout.strip()
    except (subprocess.TimeoutExpired, FileNotFoundError):
        pass
    return None


def _git_skill_diff() -> Optional[str]:
    """Get the git diff of the ammo skill directory vs HEAD."""
    try:
        result = subprocess.run(
            ["git", "diff", "HEAD", "--", ".codex/skills/ammo/"],
            capture_output=True, text=True, timeout=30,
        )
        if result.returncode == 0:
            return result.stdout if result.stdout.strip() else None
    except (subprocess.TimeoutExpired, FileNotFoundError):
        pass
    return None


def _derive_version_id(git_hash: Optional[str], description: str) -> str:
    """Derive version_id from git hash + description."""
    slug = _slugify(description)
    if git_hash:
        return f"{git_hash}_{slug}"
    # Fallback: use timestamp
    ts = datetime.now().strftime("%Y%m%d%H%M%S")
    return f"{ts}_{slug}"


def _next_run_number(target_dir: Path) -> int:
    """Find the next available run number for a target."""
    if not target_dir.exists():
        return 1
    existing = [
        d.name for d in target_dir.iterdir()
        if d.is_dir() and d.name.startswith("run_")
    ]
    numbers = []
    for name in existing:
        try:
            numbers.append(int(name.split("_")[1]))
        except (IndexError, ValueError):
            continue
    return max(numbers) + 1 if numbers else 1


def _derive_target_slug(snapshot: Dict[str, Any]) -> str:
    """Derive a filesystem-safe target slug from the snapshot."""
    target = snapshot.get("target") or {}
    model = target.get("model_id", "unknown")
    # Take last part of model path, lowercase
    model_short = model.split("/")[-1].lower()
    model_short = re.sub(r"[^a-z0-9]", "-", model_short)
    model_short = re.sub(r"-+", "-", model_short).strip("-")[:30]

    hw = (target.get("hardware") or "unknown").lower()
    dtype = (target.get("dtype") or "unknown").lower()
    tp = target.get("tp", 1)
    return f"{model_short}_{hw}_{dtype}_tp{tp}"


def archive_run(
    scorecard_path: Path,
    snapshot_path: Path,
    description: str,
    repository: Path,
    run_number: Optional[int] = None,
    transcript_grading_path: Optional[Path] = None,
    git_hash: Optional[str] = None,
    changes_snapshot_path: Optional[Path] = None,
    causal_dag_path: Optional[Path] = None,
    postmortem_narrative_path: Optional[Path] = None,
    causal_viz_path: Optional[Path] = None,
) -> Dict[str, str]:
    """Archive a scored run into the repository. Returns paths created."""
    scorecard = json.loads(scorecard_path.read_text(encoding="utf-8"))
    snapshot = json.loads(snapshot_path.read_text(encoding="utf-8"))

    if git_hash is None:
        git_hash = _git_short_hash()
    version_id = _derive_version_id(git_hash, description)
    target_slug = _derive_target_slug(snapshot)

    # Create directory structure
    version_dir = repository / "versions" / version_id
    target_dir = version_dir / "runs" / target_slug

    if run_number is None:
        run_number = _next_run_number(target_dir)
    run_dir = target_dir / f"run_{run_number}"
    run_dir.mkdir(parents=True, exist_ok=True)

    # Copy scorecard and snapshot
    shutil.copy2(scorecard_path, run_dir / "scorecard.json")
    shutil.copy2(snapshot_path, run_dir / "artifacts_snapshot.json")

    # Copy report.md if it exists alongside scorecard
    report_path = scorecard_path.parent / "report.md"
    if report_path.exists():
        shutil.copy2(report_path, run_dir / "report.md")

    # Copy transcript grading if provided
    if transcript_grading_path and transcript_grading_path.exists():
        shutil.copy2(transcript_grading_path, run_dir / "transcript_grading.json")

    # Store reference to original artifact directory
    artifact_dir = snapshot.get("artifact_dir", "unknown")
    (run_dir / "artifact_dir_ref.txt").write_text(artifact_dir + "\n", encoding="utf-8")

    # Copy changes snapshot (worktree patches, artifact copies, manifest)
    if changes_snapshot_path and changes_snapshot_path.is_dir():
        dest = run_dir / "changes_snapshot"
        shutil.copytree(changes_snapshot_path, dest, dirs_exist_ok=True)

    # Copy causal engine outputs if provided
    if causal_dag_path and causal_dag_path.exists():
        shutil.copy2(causal_dag_path, run_dir / "causal_dag.json")
    if postmortem_narrative_path and postmortem_narrative_path.exists():
        shutil.copy2(postmortem_narrative_path, run_dir / "postmortem.md")
    if causal_viz_path and causal_viz_path.exists():
        shutil.copy2(causal_viz_path, run_dir / "causal_viz.html")

    # Create/update meta.json for the version
    meta_path = version_dir / "meta.json"
    if not meta_path.exists():
        skill_diff = _git_skill_diff()
        meta = {
            "version_id": version_id,
            "git_commit": git_hash,
            "description": description,
            "created_at": datetime.now(timezone.utc).isoformat(),
            "skill_diff": skill_diff,
        }
        meta_path.parent.mkdir(parents=True, exist_ok=True)
        meta_path.write_text(json.dumps(meta, indent=2) + "\n", encoding="utf-8")

    return {
        "version_id": version_id,
        "target_slug": target_slug,
        "run_number": run_number,
        "run_dir": str(run_dir),
        "version_dir": str(version_dir),
    }


def main() -> int:
    parser = argparse.ArgumentParser(
        description=__doc__,
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    parser.add_argument("--scorecard", type=str, required=True)
    parser.add_argument("--snapshot", type=str, required=True)
    parser.add_argument("--description", type=str, required=True)
    parser.add_argument(
        "--repository", type=str,
        default=str(Path.home() / ".codex" / "ammo-eval"),
    )
    parser.add_argument("--run-number", type=int, default=None)
    parser.add_argument("--transcript-grading", type=str, default=None)
    parser.add_argument("--git-hash", type=str, default=None)
    parser.add_argument("--changes-snapshot", type=str, default=None,
                        help="Path to changes snapshot directory from snapshot_changes.py")
    parser.add_argument("--causal-dag", type=str, default=None,
                        help="Path to causal_dag_final.json from generate_postmortem.py")
    parser.add_argument("--postmortem-narrative", type=str, default=None,
                        help="Path to postmortem.md from generate_postmortem.py")
    parser.add_argument("--causal-viz", type=str, default=None,
                        help="Path to causal_viz.html from generate_postmortem.py")

    args = parser.parse_args()

    result = archive_run(
        scorecard_path=Path(args.scorecard).expanduser().resolve(),
        snapshot_path=Path(args.snapshot).expanduser().resolve(),
        description=args.description,
        repository=Path(args.repository).expanduser().resolve(),
        run_number=args.run_number,
        transcript_grading_path=(
            Path(args.transcript_grading).expanduser().resolve()
            if args.transcript_grading else None
        ),
        git_hash=args.git_hash,
        changes_snapshot_path=(
            Path(args.changes_snapshot).expanduser().resolve()
            if args.changes_snapshot else None
        ),
        causal_dag_path=(
            Path(args.causal_dag).expanduser().resolve()
            if args.causal_dag else None
        ),
        postmortem_narrative_path=(
            Path(args.postmortem_narrative).expanduser().resolve()
            if args.postmortem_narrative else None
        ),
        causal_viz_path=(
            Path(args.causal_viz).expanduser().resolve()
            if args.causal_viz else None
        ),
    )

    print(json.dumps(result, indent=2))
    return 0


if __name__ == "__main__":
    sys.exit(main())
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall