Skip to content

amazon-science/ammo / ai_cli_session/.codex/skills/ammo

Profile and optimize GPU kernels for vLLM inference on NVIDIA GPUs. Use when targeting a specific model, hardware, dtype, parallelism, and workload to improve latency.

FileKindSize
RUNTIME.mdmarkdown4.9 KB
SKILL.mdskill md18.5 KB
agents/ammo-auditor.mdmarkdown9.1 KB
agents/ammo-champion.mdmarkdown9.6 KB
agents/ammo-delegate.mdmarkdown3.2 KB
agents/ammo-implementer.mdmarkdown13.8 KB
agents/ammo-investigator.mdmarkdown4.0 KB
agents/ammo-report.mdmarkdown2.9 KB
agents/ammo-researcher.mdmarkdown10.3 KB
agents/ammo-resolver.mdmarkdown4.4 KB
agents/ammo-transcript-monitor.mdmarkdown7.8 KB
agents/openai.yamlexecutablescript313 B
data/ATTRIBUTION.mdmarkdown1.4 KB
data/gsm8k_full.jsonexecutablescript735.5 KB
data/gsm8k_subset.jsonexecutablescript116.8 KB
eval/EVAL-SKILL.mdexecutablescript21.8 KB
eval/agents/transcript_grader.mdexecutablescript18.0 KB
eval/causal/__init__.pyexecutablescript137 B
eval/causal/agents/causal_analyzer.mdexecutablescript10.5 KB
eval/causal/build_dag.pyexecutablescript21.6 KB
eval/causal/diff_versions.pyexecutablescript38.5 KB
eval/causal/extract_events.pyexecutablescript36.3 KB
eval/causal/generate_postmortem.pyexecutablescript17.0 KB
eval/causal/score_nodes.pyexecutablescript21.5 KB
eval/causal/templates/narrative_template.mdexecutablescript472 B
eval/causal/templates/viz_template.htmlexecutablescript9.5 KB
eval/causal/tests/__init__.pyexecutablescript151 B
eval/causal/tests/conftest.pyexecutablescript448 B
eval/causal/tests/fixtures/sample_session.jsonlexecutablescript10.4 KB
eval/causal/tests/fixtures/sample_session_data.jsonexecutablescript3.4 KB
eval/causal/tests/fixtures/sample_snapshot.jsonexecutablescript2.5 KB
eval/causal/tests/test_build_dag.pyexecutablescript2.7 KB
eval/causal/tests/test_diff_versions.pyexecutablescript13.5 KB
eval/causal/tests/test_extract_events.pyexecutablescript3.7 KB
eval/causal/tests/test_generate_postmortem.pyexecutablescript3.1 KB
eval/causal/tests/test_score_nodes.pyexecutablescript3.3 KB
eval/scripts/aggregate_versions.pyexecutablescript15.2 KB
eval/scripts/archive_run.pyexecutablescript9.1 KB
eval/scripts/codex_session.pyexecutablescript10.6 KB
eval/scripts/generate_dashboard.pyexecutablescript5.3 KB
eval/scripts/parse_artifacts.pyexecutablescript48.1 KB
eval/scripts/parse_session_logs.pyexecutablescript51.3 KB
eval/scripts/rescore_archived.pyexecutablescript5.5 KB
eval/scripts/score_campaign.pyexecutablescript43.8 KB
eval/scripts/snapshot_changes.pyexecutablescript11.1 KB
eval/scripts/verify_citations.pyexecutablescript8.8 KB
eval/tests/test_dashboard_bugs.pyexecutablescript8.8 KB
eval/tests/test_parse_session_logs.pyexecutablescript3.9 KB
eval/tests/test_parse_session_logs_round_inference.pyexecutablescript11.0 KB
eval/tests/test_score_campaign_diluted.pyexecutablescript9.4 KB
eval/viewer/dashboard_template.htmlexecutablescript15.1 KB
instrumentation/sitecustomize.pyexecutablescript3.6 KB
orchestration/audit-protocol.mdmarkdown9.3 KB
orchestration/debate-protocol.mdmarkdown11.2 KB
orchestration/integration-logic.mdmarkdown10.3 KB
orchestration/parallel-tracks.mdmarkdown8.9 KB
references/artifact-layout.mdexecutablescript22.4 KB
references/audit-invariants.mdmarkdown10.6 KB
references/champion-common-patterns.mdmarkdown3.4 KB
references/crossover-probing.mdexecutablescript4.7 KB
references/cudagraph-safety.mdmarkdown2.7 KB
references/debate-rules.mdmarkdown4.7 KB
references/debate-scoring-rubric.mdexecutablescript16.7 KB
references/e2e-delta-math.mdexecutablescript17.9 KB
references/e2e-latency-guide.mdexecutablescript9.6 KB
references/fusion-feasibility-heuristics.mdmarkdown5.6 KB
references/gpu-pool.mdexecutablescript5.4 KB
references/impl-track-rules.mdexecutablescript13.8 KB
references/nsys-profiling-guide.mdmarkdown9.8 KB
references/optimization-categories.mdexecutablescript11.0 KB
references/shutdown-protocol.mdexecutablescript3.1 KB
references/technology-selection.mdmarkdown5.0 KB
references/torch-compile-contract.mdmarkdown13.8 KB
references/validation-defaults.mdexecutablescript34.2 KB
references/workload-invocation.mdexecutablescript4.9 KB
references/writing-style.mdexecutablescript6.9 KB
scripts/ammo_state.pyexecutablescript142.0 KB
scripts/artifact_layout.jsonconfig16.3 KB
scripts/check_projection_accuracy.pyexecutablescript11.7 KB
scripts/create_evidence_template.pyexecutablescript9.7 KB
scripts/create_worktree_with_build.shexecutablescript10.2 KB
scripts/generate_validation_report.pyexecutablescript35.7 KB
scripts/gpu_reservation.pyexecutablescript27.1 KB
scripts/hook_cmd_classify.pyscript13.3 KB
scripts/mine_trace.pyscript29.4 KB
scripts/monitor_queue_ack.pyexecutablescript6.1 KB
scripts/ncu_sanity.shexecutablescript5.2 KB
scripts/ncu_sanity_driver.pyexecutablescript8.5 KB
scripts/new_target.pyexecutablescript22.0 KB
scripts/preflight_vllm_api.pyexecutablescript12.3 KB
scripts/reconcile_track_state.pyexecutablescript15.8 KB
scripts/remove_worktree_cleanup.shexecutablescript1.1 KB
scripts/resolve_codex_transcript.pyexecutablescript3.6 KB
scripts/run_vllm_bench_latency_sweep.pyexecutablescript249.6 KB
scripts/transcript_filter.pyexecutablescript38.4 KB
scripts/transitions.jsonexecutablescript1.5 KB
scripts/verify_validation_gates.pyexecutablescript88.8 KB
tests/agents/test-champion.mdexecutablescript20.4 KB
tests/agents/test-impl-champion.mdexecutablescript14.3 KB
tests/agents/test-implementer.mdexecutablescript15.8 KB
tests/agents/test-orchestrator.mdexecutablescript33.8 KB
tests/agents/test-researcher.mdexecutablescript21.0 KB
tests/agents/test-transcript-monitor.mdexecutablescript13.4 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.mdmarkdown1.1 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.summary.jsonconfig582 B
tests/fixtures/diluted_pass/legacy_no_phase.jsonconfig400 B
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.mdmarkdown2.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.summary.jsonconfig1.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.jsonconfig626 B
tests/fixtures/mining/real_campaign_bottleneck_analysis.mdmarkdown20.7 KB
tests/run_all.shexecutablescript1.6 KB
tests/scripts/test_transcript_filter.pyexecutablescript41.1 KB
tests/test_agent_runtime_contracts.pyexecutablescript1.8 KB
tests/test_ammo_state.pyexecutablescript71.7 KB
tests/test_artifact_layout_manifest.pyscript11.6 KB
tests/test_audit_started.pyscript18.2 KB
tests/test_correctness_comparator.pyscript12.6 KB
tests/test_diluted_pass.pyscript24.4 KB
tests/test_drift_compat.pyscript5.1 KB
tests/test_evidence_freshness.pyexecutablescript5.6 KB
tests/test_fail_closed.pyexecutablescript14.7 KB
tests/test_gpu_reservation.pyexecutablescript24.8 KB
tests/test_gpu_reservation_session_id.pyexecutablescript11.2 KB
tests/test_guidance_semantic_contract.pyscript24.9 KB
tests/test_hook_cmd_classify_parity.pyscript11.7 KB
tests/test_hook_semantics.pyexecutablescript95.3 KB
tests/test_mine_trace.pyscript24.2 KB
tests/test_mining_invalidated_schema.pyexecutablescript9.4 KB
tests/test_new_target_dp_ep.pyexecutablescript7.4 KB
tests/test_new_target_v2_scaffold.pyexecutablescript13.1 KB
tests/test_parse_artifacts.pyexecutablescript3.0 KB
tests/test_policy_convergence.pyscript797 B
tests/test_preflight_vllm_api.pyscript13.3 KB
tests/test_projection_accuracy_v2.pyexecutablescript4.1 KB
tests/test_reconcile_track_state.pyexecutablescript18.2 KB
tests/test_resolve_codex_transcript.pyexecutablescript3.1 KB
tests/test_state_schema_semantics.pyexecutablescript7.6 KB
tests/test_status_set_sync.pyexecutablescript9.1 KB
tests/test_sweep_baseline_from_arm.pyscript11.0 KB
tests/test_sweep_decode_capture.pyexecutablescript31.8 KB
tests/test_sweep_dp.pyexecutablescript45.4 KB
tests/test_sweep_fresh_cache.pyscript5.5 KB
tests/test_sweep_profiler_guard.pyscript22.6 KB
tests/test_sweep_real_prompts.pyscript20.1 KB
tests/test_sweep_refactor.pyexecutablescript37.6 KB
tests/test_sweep_round_slot.pyexecutablescript7.4 KB
tests/test_validation_report_phase_triage.pyexecutablescript10.0 KB
tests/test_validation_report_v2.pyexecutablescript6.2 KB
tests/test_verify_validation_gates.pyexecutablescript52.9 KB
tests/test_workload_matrix.pyexecutablescript19.3 KB

scripts/mine_trace.py

View on GitHub
#!/usr/bin/env python3
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: CC-BY-NC-4.0
"""Mine the Stage-2 dilution and component tables from nsys sqlite + sweep JSON.

This script is the sole owner of every mined NUMBER. The agent owns judgment
only, and supplies it through `mine_config.json`: segmentation policy, family
labels and symbol patterns, trace-to-BS mapping, and physical ceilings.

Outputs (round-scoped, never hand-edited):
  rounds/{N}/mining/mined.json  — schema `mine_trace/1`, machine-readable
  rounds/{N}/mining/tables.md   — the two mandated markdown tables

Fails loud (exit 1, no output written) on: a missing kernel table or required
column, `decode_busy` outside [0.20,1.0], merged busy > span, a non-exhaustive
family partition beyond `allow_residual`, an unresolvable decode step count.

Warns (in mined.json `warnings[]` and on stderr) on: a stored share that
disagrees with the re-derived share, a marginal delimiter separation ratio, a
per-rank busy spread above 0.02, an absent `tpot_s`, mixed nsys versions.

Config (agent judgment only):

    {
      "round": 1,
      "artifact_dir": "<campaign artifact dir>",
      "arm": "baseline",
      "e2e_results": "rounds/1/sweeps/baseline/e2e_latency_results.json",
      "segmentation": {"policy": "max_gap"},
      "traces": [{"bs": 32, "sqlite": "rounds/1/profiling/nsys/x.sqlite",
                  "label": "step 512"}],
      "families": [{"label": "dense_proj_GEMM",
                    "symbol_patterns": ["ampere_bf16_s16816gemm"],
                    "physical_ceiling": 1.619, "prefill_active": true}],
      "allow_residual": 0.0
    }

Usage:
    python3 mine_trace.py --config rounds/1/mining/mine_config.json
    python3 mine_trace.py --config <cfg> --out-json <path> --out-md <path>
"""

from __future__ import annotations

import argparse
import json
import re
import sqlite3
import sys
from pathlib import Path

SCHEMA = "mine_trace/1"
KERNEL_TABLES = ("CUPTI_ACTIVITY_KIND_KERNEL", "CUPTI_ACTIVITY_KIND_CONCURRENT_KERNEL")
REQUIRED_COLUMNS = (
    "start",
    "end",
    "deviceId",
    "streamId",
    "globalPid",
    "demangledName",
    "graphId",
)
GENERATED_HEADER = "<!-- GENERATED by mine_trace.py — DO NOT EDIT NUMBERS BY HAND -->"
DECODE_BUSY_MIN = 0.20
DECODE_BUSY_MAX = 1.0
DEFAULT_MIN_SEPARATION_RATIO = 2.0
DEFAULT_STEP_COUNT_TOLERANCE_PCT = 2.0
MAX_RANK_BUSY_SPREAD = 0.02
CLOSURE_TOLERANCE = 0.02
RESIDUAL_LABEL = "__RESIDUAL__"


class Fatal(Exception):
    """A condition that makes the mined numbers unsafe to publish."""


# ---------------------------------------------------------------------------
# sqlite reading
# ---------------------------------------------------------------------------

def load_kernels(db_path):
    """Return (rows, meta, kernel_table). Rows are ordered by start time.

    Each row: (start, end, symbol, deviceId, streamId, globalPid, graphId).
    """
    path = Path(db_path)
    if not path.is_file():
        raise Fatal(f"{db_path}: sqlite export not found")
    con = sqlite3.connect(f"file:{path}?mode=ro", uri=True)
    try:
        tables = {r[0] for r in con.execute("SELECT name FROM sqlite_master WHERE type='table'")}
        table = next((t for t in KERNEL_TABLES if t in tables), None)
        if table is None:
            raise Fatal(
                f"{db_path}: no CUPTI kernel table "
                f"(want one of {', '.join(KERNEL_TABLES)}; found {len(tables)} tables)"
            )
        if "StringIds" not in tables:
            raise Fatal(f"{db_path}: StringIds table missing; cannot resolve kernel symbols")
        columns = {r[1] for r in con.execute(f"PRAGMA table_info({table})")}
        missing = [c for c in REQUIRED_COLUMNS if c not in columns]
        if missing:
            raise Fatal(f"{db_path}: {table} lacks required column(s) {', '.join(missing)}")
        meta = {}
        if "META_DATA_EXPORT" in tables:
            meta = dict(con.execute("SELECT name, value FROM META_DATA_EXPORT"))
        rows = con.execute(
            f"SELECT k.start, k.end, s.value, k.deviceId, k.streamId, k.globalPid, k.graphId "
            f"FROM {table} k JOIN StringIds s ON s.id = k.demangledName ORDER BY k.start, k.end"
        ).fetchall()
    finally:
        con.close()
    if len(rows) < 2:
        raise Fatal(f"{db_path}: {len(rows)} kernel rows; too few to segment a step")
    return rows, meta, table


# ---------------------------------------------------------------------------
# interval math
# ---------------------------------------------------------------------------

def merged_busy(intervals):
    """Union of [start, end) intervals in ns. NEVER sums concurrent kernels."""
    if not intervals:
        return 0
    ordered = sorted(intervals)
    total = 0
    cur_start, cur_end = ordered[0]
    for start, end in ordered[1:]:
        if start > cur_end:
            total += cur_end - cur_start
            cur_start, cur_end = start, end
        else:
            cur_end = max(cur_end, end)
    return total + (cur_end - cur_start)


def duration_sum(intervals):
    return sum(end - start for start, end in intervals)


def check_window_integrity(source, intervals, busy_ns, span_ns):
    """Reject a window whose interval math cannot describe real GPU work."""
    negative = [iv for iv in intervals if iv[1] < iv[0]]
    if negative:
        raise Fatal(
            f"{source}: {len(negative)} kernel row(s) have end < start "
            f"(first {negative[0]}); the export is corrupt"
        )
    if busy_ns > span_ns:
        raise Fatal(f"{source}: merged busy {busy_ns} ns exceeds window span {span_ns} ns")


# ---------------------------------------------------------------------------
# segmentation (policy is agent judgment; the report is mechanical)
# ---------------------------------------------------------------------------

def segment(rows, segmentation):
    """Return (window_start_index, delimiter_report). Deterministic per policy."""
    policy = (segmentation or {}).get("policy")
    if policy == "max_gap":
        gaps = sorted(
            ((rows[i][0] - rows[i - 1][1], i) for i in range(1, len(rows))),
            key=lambda g: (-g[0], g[1]),
        )
        gap_ns, index = gaps[0]
        next_gap_ns = gaps[1][0] if len(gaps) > 1 else None
        ratio = (gap_ns / next_gap_ns) if next_gap_ns else None
        return index, {
            "policy": "max_gap",
            "gap_ns": gap_ns,
            "at_index": index,
            "next_gap_ns": next_gap_ns,
            "separation_ratio": ratio,
            "first_kernel_after": rows[index][2],
        }
    if policy == "delimiter_symbol":
        symbol = (segmentation or {}).get("symbol")
        if not symbol:
            raise Fatal("segmentation.policy=delimiter_symbol requires segmentation.symbol")
        hits = [i for i, r in enumerate(rows) if symbol in r[2]]
        if not hits:
            raise Fatal(f"segmentation delimiter symbol {symbol!r} absent from trace")
        index = hits[-1]
        return index, {
            "policy": "delimiter_symbol",
            "symbol": symbol,
            "occurrences": len(hits),
            "at_index": index,
            "separation_ratio": None,
            "first_kernel_after": rows[index][2],
        }
    raise Fatal(f"unknown segmentation policy {policy!r} (want max_gap or delimiter_symbol)")


# ---------------------------------------------------------------------------
# family partition (labels and patterns are agent judgment; assignment is not)
# ---------------------------------------------------------------------------

def compile_families(families):
    if not families:
        raise Fatal("config.families is empty; a partition needs at least one family")
    compiled = []
    for entry in families:
        label = entry.get("label")
        patterns = entry.get("symbol_patterns") or []
        if not label:
            raise Fatal("every config.families entry needs a label")
        if not patterns:
            raise Fatal(f"family {label!r} has no symbol_patterns")
        try:
            regexes = [re.compile(p) for p in patterns]
        except re.error as exc:
            raise Fatal(f"family {label!r} has an invalid regex: {exc}")
        compiled.append({
            "label": label,
            "regexes": regexes,
            "physical_ceiling": entry.get("physical_ceiling"),
            "prefill_active": entry.get("prefill_active"),
        })
    return compiled


def assign_symbols(rows, families):
    """Exhaustive symbol -> family map. First matching family wins."""
    assignment = {}
    residual = []
    for row in rows:
        symbol = row[2]
        if symbol in assignment:
            continue
        label = None
        for family in families:
            if any(rx.search(symbol) for rx in family["regexes"]):
                label = family["label"]
                break
        assignment[symbol] = label
        if label is None:
            residual.append(symbol)
    return assignment, residual


# ---------------------------------------------------------------------------
# per-trace mining
# ---------------------------------------------------------------------------

def mine_one_trace(trace, families, segmentation, warnings):
    rows, meta, table = load_kernels(trace["sqlite"])
    index, delimiter = segment(rows, segmentation)
    window = rows[index:]
    if not window:
        raise Fatal(f"{trace['sqlite']}: segmentation left an empty window")

    intervals = [(r[0], r[1]) for r in window]
    busy_ns = merged_busy(intervals)
    sum_ns = duration_sum(intervals)
    span_ns = max(e for _, e in intervals) - min(s for s, _ in intervals)
    check_window_integrity(trace["sqlite"], intervals, busy_ns, span_ns)

    per_rank = {}
    ranks = {}
    for row in window:
        ranks.setdefault((row[3], row[5]), []).append((row[0], row[1]))
    for (device_id, global_pid), rank_intervals in sorted(ranks.items()):
        rank_busy = merged_busy(rank_intervals)
        rank_span = (
            max(e for _, e in rank_intervals) - min(s for s, _ in rank_intervals)
        )
        per_rank[f"dev{device_id}_pid{global_pid}"] = {
            "merged_busy_ns": rank_busy,
            "sum_ns": duration_sum(rank_intervals),
            "span_ns": rank_span,
            "n_kernels": len(rank_intervals),
            "idle_ns": rank_span - rank_busy,
            "decode_busy_in_window": (rank_busy / rank_span) if rank_span else None,
        }
    rank_busy_values = [
        v["decode_busy_in_window"] for v in per_rank.values()
        if v["decode_busy_in_window"] is not None
    ]
    spread = (max(rank_busy_values) - min(rank_busy_values)) if len(rank_busy_values) > 1 else 0.0
    if spread > MAX_RANK_BUSY_SPREAD:
        warnings.append(
            f"{trace['sqlite']}: per-rank busy spread {spread:.4f} exceeds "
            f"{MAX_RANK_BUSY_SPREAD}; check the region boundary for a leaked warmup gap"
        )

    assignment, residual = assign_symbols(window, families)
    grouped = {}
    for row in window:
        grouped.setdefault(assignment[row[2]] or RESIDUAL_LABEL, []).append(row)
    family_rows = {}
    for label, rows_in in grouped.items():
        fam_intervals = [(r[0], r[1]) for r in rows_in]
        family_rows[label] = {
            "merged_busy_ns": merged_busy(fam_intervals),
            "sum_ns": duration_sum(fam_intervals),
            "n_kernels": len(rows_in),
            "in_graph_ns": sum(r[1] - r[0] for r in rows_in if r[6] is not None),
            "symbols": sorted({r[2] for r in rows_in}),
        }
    residual_sum_ns = family_rows.get(RESIDUAL_LABEL, {}).get("sum_ns", 0)
    assigned_sum_ns = sum_ns - residual_sum_ns

    separation_ratio = delimiter.get("separation_ratio")
    min_ratio = segmentation.get("min_separation_ratio", DEFAULT_MIN_SEPARATION_RATIO)
    if separation_ratio is not None and separation_ratio < min_ratio:
        warnings.append(
            f"{trace['sqlite']}: delimiter separation_ratio {separation_ratio:.2f} is below "
            f"{min_ratio}; the steady-state window is marginal — justify it in the artifact"
        )

    return {
        "sqlite": str(trace["sqlite"]),
        "label": trace.get("label"),
        "kernel_table": table,
        "nsys_product_version": meta.get("EXPORT_PRODUCT_VERSION"),
        "nsys_schema_version": meta.get("EXPORT_SCHEMA_VERSION"),
        "n_kernels_trace": len(rows),
        "n_kernels_window": len(window),
        "delimiter": delimiter,
        "merged_busy_ns": busy_ns,
        "sum_ns": sum_ns,
        "overlap_ns": sum_ns - busy_ns,
        "span_ns": span_ns,
        "measured_idle_ns": span_ns - busy_ns,
        "busy_over_span": busy_ns / span_ns if span_ns else None,
        "sum_over_span": sum_ns / span_ns if span_ns else None,
        "in_graph_fraction": (
            sum(r[1] - r[0] for r in window if r[6] is not None) / sum_ns if sum_ns else None
        ),
        "per_rank": per_rank,
        "per_rank_busy_spread": spread,
        "families": family_rows,
        "residual_symbols": residual,
        "residual_pct": 100.0 * residual_sum_ns / sum_ns if sum_ns else 0.0,
        "partition_coverage": assigned_sum_ns / sum_ns if sum_ns else 0.0,
    }


# ---------------------------------------------------------------------------
# step count ladder
# ---------------------------------------------------------------------------

def resolve_step_count(bs, metrics, result, sweep, config, warnings):
    """Return (n_decode_steps, source, candidates). FATAL when nothing resolves."""
    decode_avg_s = metrics["decode_avg_s"]
    num_iters = (
        config.get("num_iters")
        or (sweep.get("workload") or {}).get("num_iters")
        or result.get("num_iters")
    )
    output_len = result.get("output_len") or (sweep.get("workload") or {}).get("output_len")
    spec = metrics.get("spec_decode") or {}
    tpot_s = metrics.get("tpot_s")

    candidates = {}
    if spec.get("num_drafts") and num_iters:
        candidates["spec_decode.num_drafts"] = spec["num_drafts"] / (num_iters * bs)
    if isinstance(tpot_s, (int, float)) and tpot_s > 0:
        candidates["decode_avg_s/tpot_s"] = decode_avg_s / tpot_s
    if output_len and output_len > 1:
        candidates["output_len-1"] = float(output_len - 1)

    for source in ("spec_decode.num_drafts", "decode_avg_s/tpot_s", "output_len-1"):
        if source not in candidates:
            continue
        if source == "output_len-1":
            warnings.append(
                f"bs={bs}: tpot_s absent and no spec_decode counters; n_decode_steps from "
                f"output_len-1 assumes one output token per step"
            )
        if source == "decode_avg_s/tpot_s" and "output_len-1" in candidates:
            tolerance = config.get("step_count_tolerance_pct", DEFAULT_STEP_COUNT_TOLERANCE_PCT)
            a, b = candidates[source], candidates["output_len-1"]
            if b and abs(a - b) / b * 100.0 > tolerance:
                raise Fatal(
                    f"bs={bs}: step-count rungs disagree — decode_avg_s/tpot_s={a:.3f} vs "
                    f"output_len-1={b:.3f} (> {tolerance}%); resolve the production step count"
                )
        return candidates[source], source, candidates
    raise Fatal(
        f"bs={bs}: cannot derive the decode step count (no spec_decode counters, no tpot_s, "
        f"no output_len)"
    )


# ---------------------------------------------------------------------------
# per-BS mining
# ---------------------------------------------------------------------------

def _mean(values):
    return sum(values) / len(values) if values else 0.0


def mine_batch_size(bs, traces, metrics, result, sweep, config, families, warnings):
    avg_s = metrics.get("avg_s")
    decode_avg_s = metrics.get("decode_avg_s")
    prefill_avg_s = metrics.get("prefill_avg_s")
    if not avg_s or not decode_avg_s:
        raise Fatal(f"bs={bs}: arm has no avg_s/decode_avg_s; it cannot supply phase timing")

    # Denominator rule, mechanized: both shares divide the authoritative wall.
    decode_share_of_e2e = decode_avg_s / avg_s
    prefill_share = (avg_s - decode_avg_s) / avg_s
    phase_sum_share = (
        decode_avg_s / (prefill_avg_s + decode_avg_s) if prefill_avg_s is not None else None
    )
    stored = metrics.get("decode_share_of_e2e")
    if stored is not None and abs(stored - decode_share_of_e2e) > 1e-9:
        detail = f"; phase-sum value is {phase_sum_share:.9f}" if phase_sum_share else ""
        warnings.append(
            f"bs={bs}: stored decode_share_of_e2e={stored:.9f} disagrees with the re-derived "
            f"wall share {decode_share_of_e2e:.9f}{detail}; the re-derived value is used"
        )

    depths = [mine_one_trace(t, families, config.get("segmentation") or {}, warnings) for t in traces]
    mean_busy_ns = _mean([d["merged_busy_ns"] for d in depths])
    mean_sum_ns = _mean([d["sum_ns"] for d in depths])

    residual_pct = _mean([d["residual_pct"] for d in depths])
    allow_residual = float(config.get("allow_residual") or 0.0)
    if residual_pct > allow_residual:
        unmatched = sorted({s for d in depths for s in d["residual_symbols"]})
        raise Fatal(
            f"bs={bs}: family partition is not exhaustive — {residual_pct:.3f}% of window kernel "
            f"time is unassigned (allow_residual={allow_residual}); "
            f"{len(unmatched)} unmatched symbol(s), first: {unmatched[:3]}"
        )
    if residual_pct > 0:
        warnings.append(
            f"bs={bs}: family partition leaves {residual_pct:.3f}% of window kernel time in "
            f"{RESIDUAL_LABEL}; disclose it in the artifact"
        )

    n_steps, step_source, step_candidates = resolve_step_count(
        bs, metrics, result, sweep, config, warnings
    )
    decode_kernel_s = (mean_busy_ns / 1e9) * n_steps
    decode_busy = decode_kernel_s / decode_avg_s
    if not DECODE_BUSY_MIN <= decode_busy <= DECODE_BUSY_MAX:
        raise Fatal(
            f"bs={bs}: decode_busy={decode_busy:.4f} outside [{DECODE_BUSY_MIN},{DECODE_BUSY_MAX}] "
            f"(decode_kernel_s={decode_kernel_s:.4f}, decode_wall_s={decode_avg_s:.4f}, "
            f"n_decode_steps={n_steps:.3f} via {step_source}) — a measurement or scaling defect"
        )
    multiplier = decode_busy * decode_share_of_e2e
    if multiplier > decode_share_of_e2e + 1e-12:
        raise Fatal(
            f"bs={bs}: ceiling violated — decode_busy x decode_share_of_e2e={multiplier:.6f} "
            f"exceeds decode_avg_s/avg_s={decode_share_of_e2e:.6f}"
        )
    inter_kernel_share = (1.0 - decode_busy) * decode_share_of_e2e
    other_share = 1.0 - prefill_share - multiplier - inter_kernel_share
    if abs(other_share) >= CLOSURE_TOLERANCE:
        raise Fatal(
            f"bs={bs}: four-slice closure residual {other_share:.6f} exceeds "
            f"{CLOSURE_TOLERANCE}; the shares are wrongly denominated"
        )

    labels = sorted({label for d in depths for label in d["families"]})

    def _fam_mean(label, key):
        return _mean([d["families"].get(label, {}).get(key, 0) for d in depths])

    family_total = sum(_fam_mean(label, "merged_busy_ns") for label in labels)
    if family_total <= 0:
        raise Fatal(f"bs={bs}: family partition has zero busy time")
    ceilings = {f["label"]: f.get("physical_ceiling") for f in families}
    prefill_flags = {f["label"]: f.get("prefill_active") for f in families}
    family_out = []
    for label in labels:
        busy_ns = _fam_mean(label, "merged_busy_ns")
        fam_sum_ns = _fam_mean(label, "sum_ns")
        f_decode = busy_ns / family_total
        f_e2e = f_decode * multiplier
        ceiling = ceilings.get(label)
        family_out.append({
            "label": label,
            "f_decode": f_decode,
            "f_e2e": f_e2e,
            "busy_ns": busy_ns,
            "sum_ns": fam_sum_ns,
            "n_kernels_per_step": _fam_mean(label, "n_kernels"),
            "in_graph_fraction": (
                _fam_mean(label, "in_graph_ns") / fam_sum_ns if fam_sum_ns else None
            ),
            "physical_ceiling": ceiling,
            "addressable_e2e": (f_e2e * (1.0 - 1.0 / ceiling)) if ceiling else None,
            "prefill_active": prefill_flags.get(label),
            "per_depth_busy_ns": [
                d["families"].get(label, {}).get("merged_busy_ns", 0) for d in depths
            ],
            "symbols": sorted(
                {s for d in depths for s in d["families"].get(label, {}).get("symbols", [])}
            ),
        })
    family_out.sort(key=lambda f: -f["f_e2e"])

    return {
        "bs": bs,
        "trace": depths[-1]["sqlite"],
        "traces": [d["sqlite"] for d in depths],
        "avg_s": avg_s,
        "decode_avg_s": decode_avg_s,
        "prefill_avg_s": prefill_avg_s,
        "tpot_s": metrics.get("tpot_s"),
        "n_decode_steps": n_steps,
        "step_count_source": step_source,
        "step_count_candidates": step_candidates,
        "mean_merged_busy_per_step_s": mean_busy_ns / 1e9,
        "mean_sum_per_step_s": mean_sum_ns / 1e9,
        "decode_kernel_s": decode_kernel_s,
        "decode_busy": decode_busy,
        "decode_share_of_e2e": decode_share_of_e2e,
        "decode_share_of_phase_sum": phase_sum_share,
        "stored_decode_share_of_e2e": stored,
        "decode_kernel_share": multiplier,
        "inter_kernel_share": inter_kernel_share,
        "prefill_share": prefill_share,
        "prefill_avg_share": (prefill_avg_s / avg_s) if prefill_avg_s is not None else None,
        "ramp_residual_share": (
            (avg_s - decode_avg_s - prefill_avg_s) / avg_s if prefill_avg_s is not None else None
        ),
        "other_share": other_share,
        "f_e2e_multiplier": multiplier,
        "in_graph_fraction": _mean([d["in_graph_fraction"] or 0.0 for d in depths]),
        "partition_coverage": _mean([d["partition_coverage"] for d in depths]),
        "residual_pct": residual_pct,
        "residual_symbols": sorted({s for d in depths for s in d["residual_symbols"]}),
        "families": family_out,
        "depths": depths,
    }


# ---------------------------------------------------------------------------
# markdown emission
# ---------------------------------------------------------------------------

def _cell(value, digits):
    return "n/a" if value is None else f"{value:.{digits}f}"


def render_tables(per_bs, config):
    lines = [
        GENERATED_HEADER,
        "",
        "## Workload Dilution (per BS)",
        "",
        "| BS | total_e2e_s | prefill_s | decode_wall_s | decode_kernel_s | decode_busy "
        "| decode_share_of_e2e | inter_kernel_share | prefill_share |",
        "|---|---|---|---|---|---|---|---|---|",
    ]
    for row in per_bs:
        lines.append(
            f"| {row['bs']} | {row['avg_s']:.4f} | {_cell(row['prefill_avg_s'], 4)} "
            f"| {row['decode_avg_s']:.4f} | {row['decode_kernel_s']:.4f} "
            f"| {row['decode_busy']:.4f} | {row['decode_share_of_e2e']:.4f} "
            f"| {row['inter_kernel_share']:.5f} | {row['prefill_share']:.4f} |"
        )
    # `decode_frac` is machine-read by ammo_state.parse_mining_md for the mining
    # enrichment; emit it here so no agent transcribes it. Largest BS = the
    # production-representative bucket.
    largest = per_bs[-1]
    lines += [
        "",
        f"`decode_frac = {largest['decode_share_of_e2e']:.4f}` (BS={largest['bs']}, "
        f"= decode_avg_s / avg_s).",
    ]
    lines += [
        "",
        "## Top Components (by f_e2e)",
        "",
        "<!-- physical_ceiling comes from mine_config.json families[].physical_ceiling; "
        "a family without a grounded roofline stays (disclosed). -->",
        "",
        "| Component | BS | decode-graph % | f_e2e | physical_ceiling "
        "| f_e2e x (1-1/ceiling) | prefill-active? |",
        "|---|---|---|---|---|---|---|",
    ]
    prefill_ceiling = config.get("prefill_physical_ceiling")
    for row in per_bs:
        for fam in row["families"]:
            ceiling = fam["physical_ceiling"]
            addressable = fam["addressable_e2e"]
            flag = fam["prefill_active"]
            lines.append(
                f"| {fam['label']} | {row['bs']} | {fam['f_decode'] * 100:.2f} "
                f"| {fam['f_e2e']:.4f} "
                f"| {f'{ceiling:.3f}x' if ceiling else '(disclosed)'} "
                f"| {f'{addressable:.4f}' if addressable is not None else '(disclosed)'} "
                f"| {'Yes' if flag else ('No' if flag is not None else '(agent)')} |"
            )
        lines.append(
            f"| **inter_kernel_slack** | {row['bs']} | — | {row['inter_kernel_share']:.5f} "
            f"| 1.000x | {row['inter_kernel_share']:.5f} | — |"
        )
        addressable_prefill = (
            f"{row['prefill_share'] * (1.0 - 1.0 / prefill_ceiling):.4f}"
            if prefill_ceiling
            else "(disclosed)"
        )
        lines.append(
            f"| **prefill (all)** | {row['bs']} | — | {row['prefill_share']:.4f} "
            f"| {f'{prefill_ceiling:.3f}x' if prefill_ceiling else '(disclosed)'} "
            f"| {addressable_prefill} | (is prefill) |"
        )
    return "\n".join(lines) + "\n"


# ---------------------------------------------------------------------------
# driver
# ---------------------------------------------------------------------------

def _resolve(base, value):
    path = Path(value)
    return path if path.is_absolute() else (Path(base) / path)


def build(config, warnings):
    artifact_dir = config.get("artifact_dir") or "."
    families = compile_families(config.get("families"))
    sweep_path = _resolve(
        artifact_dir, config.get("e2e_results") or config.get("e2e_latency_results") or ""
    )
    if not sweep_path.is_file():
        raise Fatal(f"e2e_results not found: {sweep_path}")
    sweep = json.loads(sweep_path.read_text(encoding="utf-8"))
    arm = config.get("arm") or "baseline"

    by_bs = {}
    for trace in config.get("traces") or []:
        if "bs" not in trace or "sqlite" not in trace:
            raise Fatal("every config.traces entry needs bs and sqlite")
        resolved = dict(trace)
        resolved["sqlite"] = str(_resolve(artifact_dir, trace["sqlite"]))
        by_bs.setdefault(int(trace["bs"]), []).append(resolved)
    if not by_bs:
        raise Fatal("config.traces is empty; nothing to mine")

    results = {}
    for result in sweep.get("results") or []:
        if "batch_size" in result:
            results[int(result["batch_size"])] = result

    per_bs = []
    for bs in sorted(by_bs):
        result = results.get(bs)
        if result is None:
            raise Fatal(f"bs={bs} has traces but no matching result in {sweep_path}")
        metrics = dict(result.get(arm) or {})
        for key, value in (result.get(arm) or {}).get("metrics", {}).items():
            metrics.setdefault(key, value)
        per_bs.append(
            mine_batch_size(bs, by_bs[bs], metrics, result, sweep, config, families, warnings)
        )

    trace_provenance = [
        {
            "bs": row["bs"],
            "sqlite": depth["sqlite"],
            "label": depth["label"],
            "kernel_table": depth["kernel_table"],
            "nsys_product_version": depth["nsys_product_version"],
            "nsys_schema_version": depth["nsys_schema_version"],
            "n_kernels_trace": depth["n_kernels_trace"],
        }
        for row in per_bs
        for depth in row["depths"]
    ]

    def _unique(key):
        values = sorted({t[key] for t in trace_provenance if t[key] is not None})
        if not values:
            return None
        if len(values) > 1:
            warnings.append(f"traces mix {key} values: {', '.join(values)}")
            return ", ".join(values)
        return values[0]

    kernel_tables = sorted({t["kernel_table"] for t in trace_provenance})
    return {
        "schema": SCHEMA,
        "provenance": {
            "nsys_product_version": _unique("nsys_product_version"),
            "export_schema_version": _unique("nsys_schema_version"),
            "kernel_table": ", ".join(kernel_tables),
            "traces": trace_provenance,
        },
        "config_echo": config,
        "e2e_results": str(sweep_path),
        "arm": arm,
        "per_bs": per_bs,
        "warnings": warnings,
    }


def main(argv=None):
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--config", required=True, help="mine_config.json (agent judgment)")
    parser.add_argument("--out-json", help="default: {artifact_dir}/rounds/{N}/mining/mined.json")
    parser.add_argument("--out-md", help="default: {artifact_dir}/rounds/{N}/mining/tables.md")
    args = parser.parse_args(argv)

    config = json.loads(Path(args.config).read_text(encoding="utf-8"))
    warnings = []
    try:
        mined = build(config, warnings)
    except Fatal as exc:
        for warning in warnings:
            print(f"WARN: {warning}", file=sys.stderr)
        print(f"FATAL: {exc}", file=sys.stderr)
        return 1

    mining_dir = Path(config.get("artifact_dir") or ".") / "rounds" / str(
        config.get("round", 1)
    ) / "mining"
    out_json = Path(args.out_json) if args.out_json else mining_dir / "mined.json"
    out_md = Path(args.out_md) if args.out_md else mining_dir / "tables.md"
    out_json.parent.mkdir(parents=True, exist_ok=True)
    out_md.parent.mkdir(parents=True, exist_ok=True)
    out_json.write_text(json.dumps(mined, indent=1, sort_keys=False) + "\n", encoding="utf-8")
    out_md.write_text(render_tables(mined["per_bs"], config), encoding="utf-8")

    for warning in warnings:
        print(f"WARN: {warning}", file=sys.stderr)
    print(f"wrote {out_json}")
    print(f"wrote {out_md}")
    return 0


if __name__ == "__main__":
    sys.exit(main())
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall