Skip to content

amazon-science/ammo / ai_cli_session/.codex/skills/ammo

Profile and optimize GPU kernels for vLLM inference on NVIDIA GPUs. Use when targeting a specific model, hardware, dtype, parallelism, and workload to improve latency.

FileKindSize
RUNTIME.mdmarkdown4.9 KB
SKILL.mdskill md18.5 KB
agents/ammo-auditor.mdmarkdown9.1 KB
agents/ammo-champion.mdmarkdown9.6 KB
agents/ammo-delegate.mdmarkdown3.2 KB
agents/ammo-implementer.mdmarkdown13.8 KB
agents/ammo-investigator.mdmarkdown4.0 KB
agents/ammo-report.mdmarkdown2.9 KB
agents/ammo-researcher.mdmarkdown10.3 KB
agents/ammo-resolver.mdmarkdown4.4 KB
agents/ammo-transcript-monitor.mdmarkdown7.8 KB
agents/openai.yamlexecutablescript313 B
data/ATTRIBUTION.mdmarkdown1.4 KB
data/gsm8k_full.jsonexecutablescript735.5 KB
data/gsm8k_subset.jsonexecutablescript116.8 KB
eval/EVAL-SKILL.mdexecutablescript21.8 KB
eval/agents/transcript_grader.mdexecutablescript18.0 KB
eval/causal/__init__.pyexecutablescript137 B
eval/causal/agents/causal_analyzer.mdexecutablescript10.5 KB
eval/causal/build_dag.pyexecutablescript21.6 KB
eval/causal/diff_versions.pyexecutablescript38.5 KB
eval/causal/extract_events.pyexecutablescript36.3 KB
eval/causal/generate_postmortem.pyexecutablescript17.0 KB
eval/causal/score_nodes.pyexecutablescript21.5 KB
eval/causal/templates/narrative_template.mdexecutablescript472 B
eval/causal/templates/viz_template.htmlexecutablescript9.5 KB
eval/causal/tests/__init__.pyexecutablescript151 B
eval/causal/tests/conftest.pyexecutablescript448 B
eval/causal/tests/fixtures/sample_session.jsonlexecutablescript10.4 KB
eval/causal/tests/fixtures/sample_session_data.jsonexecutablescript3.4 KB
eval/causal/tests/fixtures/sample_snapshot.jsonexecutablescript2.5 KB
eval/causal/tests/test_build_dag.pyexecutablescript2.7 KB
eval/causal/tests/test_diff_versions.pyexecutablescript13.5 KB
eval/causal/tests/test_extract_events.pyexecutablescript3.7 KB
eval/causal/tests/test_generate_postmortem.pyexecutablescript3.1 KB
eval/causal/tests/test_score_nodes.pyexecutablescript3.3 KB
eval/scripts/aggregate_versions.pyexecutablescript15.2 KB
eval/scripts/archive_run.pyexecutablescript9.1 KB
eval/scripts/codex_session.pyexecutablescript10.6 KB
eval/scripts/generate_dashboard.pyexecutablescript5.3 KB
eval/scripts/parse_artifacts.pyexecutablescript48.1 KB
eval/scripts/parse_session_logs.pyexecutablescript51.3 KB
eval/scripts/rescore_archived.pyexecutablescript5.5 KB
eval/scripts/score_campaign.pyexecutablescript43.8 KB
eval/scripts/snapshot_changes.pyexecutablescript11.1 KB
eval/scripts/verify_citations.pyexecutablescript8.8 KB
eval/tests/test_dashboard_bugs.pyexecutablescript8.8 KB
eval/tests/test_parse_session_logs.pyexecutablescript3.9 KB
eval/tests/test_parse_session_logs_round_inference.pyexecutablescript11.0 KB
eval/tests/test_score_campaign_diluted.pyexecutablescript9.4 KB
eval/viewer/dashboard_template.htmlexecutablescript15.1 KB
instrumentation/sitecustomize.pyexecutablescript3.6 KB
orchestration/audit-protocol.mdmarkdown9.3 KB
orchestration/debate-protocol.mdmarkdown11.2 KB
orchestration/integration-logic.mdmarkdown10.3 KB
orchestration/parallel-tracks.mdmarkdown8.9 KB
references/artifact-layout.mdexecutablescript22.4 KB
references/audit-invariants.mdmarkdown10.6 KB
references/champion-common-patterns.mdmarkdown3.4 KB
references/crossover-probing.mdexecutablescript4.7 KB
references/cudagraph-safety.mdmarkdown2.7 KB
references/debate-rules.mdmarkdown4.7 KB
references/debate-scoring-rubric.mdexecutablescript16.7 KB
references/e2e-delta-math.mdexecutablescript17.9 KB
references/e2e-latency-guide.mdexecutablescript9.6 KB
references/fusion-feasibility-heuristics.mdmarkdown5.6 KB
references/gpu-pool.mdexecutablescript5.4 KB
references/impl-track-rules.mdexecutablescript13.8 KB
references/nsys-profiling-guide.mdmarkdown9.8 KB
references/optimization-categories.mdexecutablescript11.0 KB
references/shutdown-protocol.mdexecutablescript3.1 KB
references/technology-selection.mdmarkdown5.0 KB
references/torch-compile-contract.mdmarkdown13.8 KB
references/validation-defaults.mdexecutablescript34.2 KB
references/workload-invocation.mdexecutablescript4.9 KB
references/writing-style.mdexecutablescript6.9 KB
scripts/ammo_state.pyexecutablescript142.0 KB
scripts/artifact_layout.jsonconfig16.3 KB
scripts/check_projection_accuracy.pyexecutablescript11.7 KB
scripts/create_evidence_template.pyexecutablescript9.7 KB
scripts/create_worktree_with_build.shexecutablescript10.2 KB
scripts/generate_validation_report.pyexecutablescript35.7 KB
scripts/gpu_reservation.pyexecutablescript27.1 KB
scripts/hook_cmd_classify.pyscript13.3 KB
scripts/mine_trace.pyscript29.4 KB
scripts/monitor_queue_ack.pyexecutablescript6.1 KB
scripts/ncu_sanity.shexecutablescript5.2 KB
scripts/ncu_sanity_driver.pyexecutablescript8.5 KB
scripts/new_target.pyexecutablescript22.0 KB
scripts/preflight_vllm_api.pyexecutablescript12.3 KB
scripts/reconcile_track_state.pyexecutablescript15.8 KB
scripts/remove_worktree_cleanup.shexecutablescript1.1 KB
scripts/resolve_codex_transcript.pyexecutablescript3.6 KB
scripts/run_vllm_bench_latency_sweep.pyexecutablescript249.6 KB
scripts/transcript_filter.pyexecutablescript38.4 KB
scripts/transitions.jsonexecutablescript1.5 KB
scripts/verify_validation_gates.pyexecutablescript88.8 KB
tests/agents/test-champion.mdexecutablescript20.4 KB
tests/agents/test-impl-champion.mdexecutablescript14.3 KB
tests/agents/test-implementer.mdexecutablescript15.8 KB
tests/agents/test-orchestrator.mdexecutablescript33.8 KB
tests/agents/test-researcher.mdexecutablescript21.0 KB
tests/agents/test-transcript-monitor.mdexecutablescript13.4 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.mdmarkdown1.1 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.summary.jsonconfig582 B
tests/fixtures/diluted_pass/legacy_no_phase.jsonconfig400 B
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.mdmarkdown2.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.summary.jsonconfig1.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.jsonconfig626 B
tests/fixtures/mining/real_campaign_bottleneck_analysis.mdmarkdown20.7 KB
tests/run_all.shexecutablescript1.6 KB
tests/scripts/test_transcript_filter.pyexecutablescript41.1 KB
tests/test_agent_runtime_contracts.pyexecutablescript1.8 KB
tests/test_ammo_state.pyexecutablescript71.7 KB
tests/test_artifact_layout_manifest.pyscript11.6 KB
tests/test_audit_started.pyscript18.2 KB
tests/test_correctness_comparator.pyscript12.6 KB
tests/test_diluted_pass.pyscript24.4 KB
tests/test_drift_compat.pyscript5.1 KB
tests/test_evidence_freshness.pyexecutablescript5.6 KB
tests/test_fail_closed.pyexecutablescript14.7 KB
tests/test_gpu_reservation.pyexecutablescript24.8 KB
tests/test_gpu_reservation_session_id.pyexecutablescript11.2 KB
tests/test_guidance_semantic_contract.pyscript24.9 KB
tests/test_hook_cmd_classify_parity.pyscript11.7 KB
tests/test_hook_semantics.pyexecutablescript95.3 KB
tests/test_mine_trace.pyscript24.2 KB
tests/test_mining_invalidated_schema.pyexecutablescript9.4 KB
tests/test_new_target_dp_ep.pyexecutablescript7.4 KB
tests/test_new_target_v2_scaffold.pyexecutablescript13.1 KB
tests/test_parse_artifacts.pyexecutablescript3.0 KB
tests/test_policy_convergence.pyscript797 B
tests/test_preflight_vllm_api.pyscript13.3 KB
tests/test_projection_accuracy_v2.pyexecutablescript4.1 KB
tests/test_reconcile_track_state.pyexecutablescript18.2 KB
tests/test_resolve_codex_transcript.pyexecutablescript3.1 KB
tests/test_state_schema_semantics.pyexecutablescript7.6 KB
tests/test_status_set_sync.pyexecutablescript9.1 KB
tests/test_sweep_baseline_from_arm.pyscript11.0 KB
tests/test_sweep_decode_capture.pyexecutablescript31.8 KB
tests/test_sweep_dp.pyexecutablescript45.4 KB
tests/test_sweep_fresh_cache.pyscript5.5 KB
tests/test_sweep_profiler_guard.pyscript22.6 KB
tests/test_sweep_real_prompts.pyscript20.1 KB
tests/test_sweep_refactor.pyexecutablescript37.6 KB
tests/test_sweep_round_slot.pyexecutablescript7.4 KB
tests/test_validation_report_phase_triage.pyexecutablescript10.0 KB
tests/test_validation_report_v2.pyexecutablescript6.2 KB
tests/test_verify_validation_gates.pyexecutablescript52.9 KB
tests/test_workload_matrix.pyexecutablescript19.3 KB

scripts/new_target.py

View on GitHub
#!/usr/bin/env python3
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: CC-BY-NC-4.0
"""ammo v2: scaffold a new target artifact directory.

Creates:
  - constraints.md
  - state.json (simplified v2 schema)
  - target.json (input for run_vllm_bench_latency_sweep.py)

Safety: refuses to overwrite existing files unless --force is provided.

Example:
  .venv/bin/python .codex/skills/ammo/scripts/new_target.py \\
    --artifact-dir kernel_opt_artifacts/auto_qwen3_l40s_fp8_tp1 \\
    --model-id Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 \\
    --hardware L40S --dtype fp8 --tp 1 --ep 1
"""

from __future__ import annotations

import argparse
import json
import os
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List


PLACEHOLDER = "<FILL_ME>"


def _detect_gpu_model() -> str | None:
    """Return the first GPU's name from nvidia-smi, or None when unavailable.

    nvidia-smi ignores CUDA_VISIBLE_DEVICES, so on a heterogeneous host the
    unscoped query can name a GPU this session does not own. Scope the query
    with -i to the session's visible devices; a bad index fails closed to the
    placeholder instead of a confidently-wrong model name.
    """
    cmd = ["nvidia-smi", "--query-gpu=name", "--format=csv,noheader"]
    cvd = os.environ.get("CUDA_VISIBLE_DEVICES", "").strip()
    if cvd and cvd != "-1":
        cmd += ["-i", cvd]
    try:
        result = subprocess.run(
            cmd,
            capture_output=True,
            text=True,
            timeout=10,
        )
    except (OSError, subprocess.SubprocessError):
        return None
    if result.returncode != 0:
        return None
    lines = [line.strip() for line in result.stdout.splitlines() if line.strip()]
    return lines[0] if lines else None


@dataclass(frozen=True)
class TargetFields:
    model_id: str
    hardware: str
    dtype: str
    tp: int
    ep: int
    max_model_len: int
    input_len: int
    output_len: int
    batch_sizes: List[int]
    num_iters: int
    noise_tolerance_pct: float
    catastrophic_regression_pct: float
    min_e2e_improvement_pct: float = 0.5
    data_parallel_size: int = 1
    enable_expert_parallel: bool = False
    max_num_seqs: int | None = None
    workload_matrix: List[Dict[str, int]] | None = None
    max_model_len_auto_requested: bool = False


def _cached_model_max_len(model_id: str | None) -> int | None:
    """Read max_position_embeddings from the local HF cache, if present."""
    if not model_id or "/" not in model_id:
        return None
    hub = Path(
        os.environ.get("HF_HOME", Path.home() / ".cache" / "huggingface")
    ) / "hub" / ("models--" + model_id.replace("/", "--"))
    for config in sorted(hub.glob("snapshots/*/config.json")):
        try:
            data = json.loads(config.read_text(encoding="utf-8"))
        except (OSError, json.JSONDecodeError):
            continue
        limit = data.get("max_position_embeddings")
        if isinstance(limit, int) and limit > 0:
            return limit
    return None


def _resolve_max_model_len(value: str | int, model_id: str | None = None) -> int:
    """Resolve an explicit positive integer; ``auto`` reads the cached model config."""
    raw = str(value).strip()
    if raw.lower() == "auto":
        resolved = _cached_model_max_len(model_id)
        if resolved is None:
            raise SystemExit(
                "--max-model-len auto could not be resolved: no cached config.json "
                f"with max_position_embeddings for {model_id!r}; "
                "supply the concrete model limit as an integer."
            )
        return resolved
    try:
        resolved = int(raw)
    except ValueError as exc:
        raise SystemExit("--max-model-len must be a positive integer or 'auto'") from exc
    if resolved <= 0:
        raise SystemExit("--max-model-len must be positive")
    return resolved


def _parse_workload_matrix(spec: str | None, batch_sizes: List[int]) -> List[Dict[str, int]] | None:
    """Expand ``ISL:OSL`` pairs across the supplied batch-size buckets."""
    if not spec:
        return None
    pairs: List[tuple[int, int]] = []
    for token in spec.split(","):
        try:
            isl_raw, osl_raw = token.strip().split(":", 1)
            isl, osl = int(isl_raw), int(osl_raw)
        except (TypeError, ValueError) as exc:
            raise SystemExit("--isl-osl must be comma-separated positive ISL:OSL pairs") from exc
        if isl <= 0 or osl <= 0:
            raise SystemExit("--isl-osl values must be positive")
        pairs.append((isl, osl))
    return [
        {"input_len": isl, "output_len": osl, "batch_size": bs}
        for isl, osl in pairs
        for bs in batch_sizes
    ]


def _write_text(path: Path, text: str, *, force: bool) -> None:
    if path.exists() and not force:
        raise FileExistsError(f"Refusing to overwrite existing file: {path} (use --force)")
    path.write_text(text, encoding="utf-8")


def _write_json(path: Path, obj: Dict[str, Any], *, force: bool) -> None:
    if path.exists() and not force:
        raise FileExistsError(f"Refusing to overwrite existing file: {path} (use --force)")
    path.write_text(json.dumps(obj, indent=2, sort_keys=True) + "\n", encoding="utf-8")


def _default_target_fields(args: argparse.Namespace) -> TargetFields:
    model_id = args.model_id or PLACEHOLDER
    hardware = args.hardware or PLACEHOLDER
    dtype = args.dtype or PLACEHOLDER

    return TargetFields(
        model_id=model_id,
        hardware=hardware,
        dtype=dtype,
        tp=args.tp,
        ep=args.ep,
        max_model_len=_resolve_max_model_len(args.max_model_len, args.model_id),
        input_len=args.input_len,
        output_len=args.output_len,
        batch_sizes=args.batch_sizes,
        num_iters=args.num_iters,
        noise_tolerance_pct=args.noise_tolerance_pct,
        catastrophic_regression_pct=args.catastrophic_regression_pct,
        min_e2e_improvement_pct=getattr(args, "min_e2e_improvement", None)
        if getattr(args, "min_e2e_improvement", None) is not None
        else _schema_default("min_e2e_improvement_pct", 0.5),
        data_parallel_size=getattr(args, "data_parallel_size", 1),
        enable_expert_parallel=getattr(args, "enable_expert_parallel", False),
        max_num_seqs=getattr(args, "max_num_seqs", None),
        workload_matrix=_parse_workload_matrix(
            getattr(args, "isl_osl", None), args.batch_sizes
        ),
        max_model_len_auto_requested=(
            str(args.max_model_len).strip().lower() == "auto"
        ),
    )


def _constraints_md(fields: TargetFields) -> str:
    return f"""# Constraints (Stage 1)

## Target envelope

- Model: {fields.model_id}
- Hardware: {fields.hardware}
- Dtype / quant format: {fields.dtype}
- TP / EP: tp={fields.tp}, ep={fields.ep}
- Data Parallel / Expert Parallel: dp={fields.data_parallel_size}, enable_expert_parallel={fields.enable_expert_parallel}
- Max model len: {fields.max_model_len}
- Decode buckets (batch sizes): {fields.batch_sizes}
- E2E workload: input_len={fields.input_len}, output_len={fields.output_len}

## TODOs (required before Stage 2)

- [ ] Read vLLM source for target component, document forward path and correctness invariants
- [ ] Capture baseline truth snapshot under production parity (CUDA graphs / torch.compile)
- [ ] Record baseline kernel timings from nsys profiling

"""

_SCHEMA_PATH = Path(__file__).resolve().parents[3] / "schemas" / "state.schema.json"


def _schema_default(path: str, fallback: float) -> float:
    """Read a default value from state.schema.json at a dotted `campaign.config.*` path.

    Keeping the defaults in the schema lets us change them once and have writers
    pick them up automatically — no per-call-site drift.
    """
    try:
        schema = json.loads(_SCHEMA_PATH.read_text())
        props = schema["properties"]["campaign"]["properties"]["config"]["properties"]
        return float(props[path]["default"])
    except (FileNotFoundError, KeyError, json.JSONDecodeError, TypeError, ValueError):
        return fallback


def _state_json(fields: TargetFields, artifact_dir: Path, min_e2e_improvement_pct: float | None = None,
                noise_tolerance_pct: float | None = None,
                catastrophic_regression_pct: float | None = None,
                gpu_model: str | None = None) -> Dict[str, Any]:
    # TargetFields is the programmatic contract. CLI defaults are populated
    # from the schema before this object is built.
    if min_e2e_improvement_pct is None:
        min_e2e_improvement_pct = fields.min_e2e_improvement_pct
    if noise_tolerance_pct is None:
        noise_tolerance_pct = fields.noise_tolerance_pct
    if catastrophic_regression_pct is None:
        catastrophic_regression_pct = fields.catastrophic_regression_pct

    # Server/API campaign identity and Codex root-thread identity are distinct.
    session_id = os.environ.get("AMMO_SESSION_ID")
    codex_thread_id = os.environ.get("CODEX_SESSION_ID")
    if not codex_thread_id:
        identity_path = (
            Path(os.environ.get("AMMO_GPU_RES_DIR") or "/tmp/ammo_gpu_res").expanduser()
            / "codex_hook_session.json"
        )
        try:
            identity_doc = json.loads(identity_path.read_text(encoding="utf-8"))
            if identity_doc.get("server_session_id") == session_id:
                codex_thread_id = identity_doc.get("codex_thread_id")
        except (OSError, json.JSONDecodeError, AttributeError):
            codex_thread_id = None
    return {
        "target": {
            "model_id": fields.model_id,
            "hardware": fields.hardware,
            "dtype": fields.dtype,
            "tp": fields.tp,
            "dp": fields.data_parallel_size,
            "ep": fields.ep,
            "component": "auto",
        },
        "session_id": session_id,
        "codex_thread_id": codex_thread_id,
        "gpu_resources": {
            "gpu_count": 1,
            "gpu_model": gpu_model if gpu_model else PLACEHOLDER,
            "memory_total_gib": 0,
            "cuda_visible_devices": "0",
        },
        "campaign": {
            "schema_version": "4.2",
            "status": "active",
            "current_round": 1,
            "current_stage": "1_baseline",
            "config": {
                "min_e2e_improvement_pct": min_e2e_improvement_pct,
                "noise_tolerance_pct": noise_tolerance_pct,
                "catastrophic_regression_pct": catastrophic_regression_pct,
            },
            "shipped_optimizations": [],
            "agent_costs": [],
            "rounds": [
                {
                    "round_id": 1,
                    "status": "IN_PROGRESS",
                    "team_name": None,
                    "profiling_baseline_path": None,
                    "baseline": {
                        "started_at": None,
                        "completed_at": None,
                        "e2e_latency": None,
                        "per_bs_verdict": None,
                    },
                    "bottleneck_mining": {
                        "started_at": None,
                        "completed_at": None,
                        "top_bottleneck_share_pct": None,
                    },
                    "debate": {
                        "started_at": None,
                        "completed_at": None,
                        "candidates": [],
                        "rounds_completed": 0,
                        "max_rounds": 4,
                        "selected_winners": [],
                    },
                    "parallel_tracks": {
                        "started_at": None,
                        "completed_at": None,
                        "tracks": {},
                    },
                    "integration": {
                        "started_at": None,
                        "completed_at": None,
                        "status": "pending",
                        "passing_candidates": [],
                        "failed_candidates": [],
                        "selected_candidates": [],
                        "conflict_analysis": None,
                        "combined_patch_branch": None,
                        "combined_e2e_result": None,
                        "e2e_latency_combined": None,
                        "per_bs_verdict": None,
                        "commit_sha": None,
                        "final_decision": None,
                        "resolver_invoked": None,
                        "resolver_outcome": None,
                        "conflicting_tracks": None,
                    },
                    "campaign_eval": {
                        "started_at": None,
                        "completed_at": None,
                    },
                    "audit": {},
                    "shipped": [],
                    "dropped": [],
                    "cumulative_speedup_after": None,
                    "combined_e2e_speedup_x": None,
                    "combined_e2e_delta_pp": None,
                    "note": None,
                    "round_summary": None,
                }
            ],
        },
    }


def _compose_extra_args(fields: TargetFields) -> List[str]:
    """Compose bench.extra_args from DP / EP fields.

    DP > 1 triggers auto-injection of --distributed-executor-backend
    external_launcher, which vLLM requires for data-parallel execution via
    torchrun. The sweep script validates that no conflicting backend value
    was appended post-hoc (e.g., via frontend additionalFlags).
    """
    extra: List[str] = []
    if fields.data_parallel_size > 1:
        extra.extend([
            "--data-parallel-size", str(fields.data_parallel_size),
            "--distributed-executor-backend", "external_launcher",
        ])
    if fields.enable_expert_parallel:
        extra.append("--enable-expert-parallel")
    if fields.max_num_seqs is not None:
        extra.extend(["--max-num-seqs", str(fields.max_num_seqs)])
    return extra


def _target_json(fields: TargetFields, artifact_dir: Path) -> Dict[str, Any]:
    # This schema is consumed by run_vllm_bench_latency_sweep.py.
    return {
        "artifact_dir": str(artifact_dir),
        "target": {
            "model_id": fields.model_id,
            "dtype": fields.dtype,
            "tp": fields.tp,
            "ep": fields.ep,
            "max_model_len": fields.max_model_len,
        },
        "workload": (
            {"workload_matrix": fields.workload_matrix, "num_iters": fields.num_iters}
            if fields.workload_matrix is not None
            else {
                "input_len": fields.input_len,
                "output_len": fields.output_len,
                "batch_sizes": fields.batch_sizes,
                "num_iters": fields.num_iters,
            }
        ),
        "bench": {
            "runner": "vllm_bench_latency",
            "vllm_cmd": "vllm",
            "extra_args": _compose_extra_args(fields),
            "baseline_extra_args": [],
            "opt_extra_args": [],
            "baseline_env": {},
            "opt_env": {
                "<ENABLE_FLAG>": "1"
            },
            "baseline_label": "baseline",
            "opt_label": "opt",
            "fastpath_evidence": {
                "baseline": {
                    "require_patterns": [],
                    "forbid_patterns": [],
                },
                "opt": {
                    "require_patterns": [],
                    "forbid_patterns": [],
                },
                "note": "Fill require_patterns to assert optimized fast-path executed (recommended).",
            },
        },
        "gating": {
            "noise_tolerance_pct": fields.noise_tolerance_pct,
            "catastrophic_regression_pct": fields.catastrophic_regression_pct,
            "min_e2e_improvement_pct": fields.min_e2e_improvement_pct,
        },
        "notes": {
            "production_parity": "Ensure CUDA graphs / torch.compile settings match production. See references/e2e-latency-guide.md.",
            **(
                {
                    "max_model_len_request": (
                        "auto requested; scaffolded with the default 4096. "
                        "Actual model support depends on the model config."
                    )
                }
                if fields.max_model_len_auto_requested
                else {}
            ),
        },
    }


def main() -> None:
    p = argparse.ArgumentParser(description=__doc__)
    p.add_argument("--artifact-dir", type=str, required=True, help="Directory to create/populate")
    p.add_argument("--force", action="store_true", help="Overwrite existing files")

    # Optional target metadata (safe defaults + placeholders)
    p.add_argument("--model-id", type=str, default=None)
    p.add_argument("--hardware", type=str, default=None)
    p.add_argument("--dtype", type=str, default=None)
    p.add_argument("--gpu-model", type=str, default=None,
                   help="GPU model name for state.json gpu_resources.gpu_model. "
                        "Default: detected from nvidia-smi (first GPU); "
                        "falls back to the placeholder when detection fails.")

    p.add_argument("--tp", type=int, default=1)
    p.add_argument("--ep", type=int, default=1,
                   help="Legacy expert-parallel sizing field — persisted to target.ep "
                        "in state.json. Does NOT enable vLLM expert parallelism. "
                        "Use --enable-expert-parallel to actually enable EP.")
    p.add_argument("--data-parallel-size", type=int, default=1,
                   help="vLLM data-parallel size. When > 1, new_target.py injects "
                        "--distributed-executor-backend external_launcher into "
                        "bench.extra_args (required for torchrun DP).")
    p.add_argument("--enable-expert-parallel", action="store_true",
                   help="Enable vLLM expert parallelism (passes "
                        "--enable-expert-parallel through to bench.extra_args).")
    p.add_argument("--min-e2e-improvement", type=float,
                   default=_schema_default("min_e2e_improvement_pct", 0.5),
                   help="Campaign floor: stop when no candidate can yield >= this %% E2E "
                        "improvement, and per-BS PASS requires clearing "
                        "max(this, noise_tolerance_pct) (schema default; keep >= noise tolerance)")

    p.add_argument("--max-model-len", default="4096",
                   help="Positive integer, or 'auto' to resolve "
                        "max_position_embeddings from the cached HF config "
                        "(fails closed if uncached)")
    p.add_argument("--input-len", type=int, default=64)
    p.add_argument("--output-len", type=int, default=512)
    p.add_argument("--batch-sizes", type=int, nargs="+", default=[1, 8, 32])
    p.add_argument("--isl-osl",
                   help="Comma-separated ISL:OSL pairs crossed with --batch-sizes")
    p.add_argument("--max-num-seqs", type=int,
                   help="Frozen serving concurrency (default: max batch size)")
    p.add_argument("--num-iters", type=int, default=10)

    # Gating options (BS-dependent optimization support).
    # Defaults resolved from state.schema.json so the schema is the single source.
    p.add_argument("--noise-tolerance-pct", type=float,
                   default=_schema_default("noise_tolerance_pct", 0.5),
                   help="Per-BS speedup within this %% of 1.0 is classified NOISE (schema default)")
    p.add_argument("--catastrophic-regression-pct", type=float,
                   default=_schema_default("catastrophic_regression_pct", 5.0),
                   help="Per-BS regression beyond this %% is classified CATASTROPHIC (schema default)")

    args = p.parse_args()

    artifact_dir = Path(args.artifact_dir).expanduser().resolve()
    artifact_dir.mkdir(parents=True, exist_ok=True)

    fields = _default_target_fields(args)

    # Create v2 round-scoped scaffold for round 1.
    # Reference: ai_cli_session/.codex/skills/ammo/references/artifact-layout.md
    round_dir = artifact_dir / "rounds" / "1"
    for subdir in (
        "profiling/nsys",
        "profiling/ncu",
        "sweeps/baseline/json",
        "sweeps/baseline/logs",
        "sweeps/baseline/status",
        "sweeps/opt",
        "sweeps/opt_correctness",
        "sweeps/opt_profiling",
        "sweeps/integration",
        "sweeps/integration_profiling",
        "sweeps/golden_capture",
        "mining",
        "debate/proposals",
        "debate/micro_experiments",
        "debate/monitor_audits",
        "tracks",
        "audits",
        "_archive",
    ):
        (round_dir / subdir).mkdir(parents=True, exist_ok=True)
    # Cross-round (campaign-level) blockers dir — NOT round-scoped.
    (artifact_dir / "blockers").mkdir(exist_ok=True)

    gpu_model = (args.gpu_model or "").strip() or _detect_gpu_model()
    if not gpu_model:
        print(
            "WARNING: gpu_resources.gpu_model is still the placeholder "
            f"{PLACEHOLDER!r} — nvidia-smi detection failed and --gpu-model "
            "was not given. Fill it in state.json before validation.",
            file=sys.stderr,
        )

    state = _state_json(
        fields, artifact_dir, args.min_e2e_improvement,
        noise_tolerance_pct=args.noise_tolerance_pct,
        catastrophic_regression_pct=args.catastrophic_regression_pct,
        gpu_model=gpu_model,
    )
    try:
        from jsonschema import Draft202012Validator
    except ImportError as exc:
        p.error(f"cannot validate generated state against schema: {exc}")
    schema = json.loads(_SCHEMA_PATH.read_text(encoding="utf-8"))
    errors = sorted(
        Draft202012Validator(schema).iter_errors(state),
        key=lambda error: list(error.path),
    )
    if errors:
        details = "; ".join(error.message for error in errors[:5])
        p.error(f"generated state violates schema: {details}")
    _write_json(artifact_dir / "state.json", state, force=args.force)
    _write_json(artifact_dir / "target.json", _target_json(fields, artifact_dir), force=args.force)

    print(f"Initialized artifact directory: {artifact_dir}")
    print("Created: state.json, target.json, rounds/1/ scaffold")
    print("Next: write rounds/1/constraints.md (Phase 1)")


if __name__ == "__main__":
    main()
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall