Skip to content

amazon-science/ammo / ai_cli_session/.codex/skills/ammo

Profile and optimize GPU kernels for vLLM inference on NVIDIA GPUs. Use when targeting a specific model, hardware, dtype, parallelism, and workload to improve latency.

FileKindSize
RUNTIME.mdmarkdown4.9 KB
SKILL.mdskill md18.5 KB
agents/ammo-auditor.mdmarkdown9.1 KB
agents/ammo-champion.mdmarkdown9.6 KB
agents/ammo-delegate.mdmarkdown3.2 KB
agents/ammo-implementer.mdmarkdown13.8 KB
agents/ammo-investigator.mdmarkdown4.0 KB
agents/ammo-report.mdmarkdown2.9 KB
agents/ammo-researcher.mdmarkdown10.3 KB
agents/ammo-resolver.mdmarkdown4.4 KB
agents/ammo-transcript-monitor.mdmarkdown7.8 KB
agents/openai.yamlexecutablescript313 B
data/ATTRIBUTION.mdmarkdown1.4 KB
data/gsm8k_full.jsonexecutablescript735.5 KB
data/gsm8k_subset.jsonexecutablescript116.8 KB
eval/EVAL-SKILL.mdexecutablescript21.8 KB
eval/agents/transcript_grader.mdexecutablescript18.0 KB
eval/causal/__init__.pyexecutablescript137 B
eval/causal/agents/causal_analyzer.mdexecutablescript10.5 KB
eval/causal/build_dag.pyexecutablescript21.6 KB
eval/causal/diff_versions.pyexecutablescript38.5 KB
eval/causal/extract_events.pyexecutablescript36.3 KB
eval/causal/generate_postmortem.pyexecutablescript17.0 KB
eval/causal/score_nodes.pyexecutablescript21.5 KB
eval/causal/templates/narrative_template.mdexecutablescript472 B
eval/causal/templates/viz_template.htmlexecutablescript9.5 KB
eval/causal/tests/__init__.pyexecutablescript151 B
eval/causal/tests/conftest.pyexecutablescript448 B
eval/causal/tests/fixtures/sample_session.jsonlexecutablescript10.4 KB
eval/causal/tests/fixtures/sample_session_data.jsonexecutablescript3.4 KB
eval/causal/tests/fixtures/sample_snapshot.jsonexecutablescript2.5 KB
eval/causal/tests/test_build_dag.pyexecutablescript2.7 KB
eval/causal/tests/test_diff_versions.pyexecutablescript13.5 KB
eval/causal/tests/test_extract_events.pyexecutablescript3.7 KB
eval/causal/tests/test_generate_postmortem.pyexecutablescript3.1 KB
eval/causal/tests/test_score_nodes.pyexecutablescript3.3 KB
eval/scripts/aggregate_versions.pyexecutablescript15.2 KB
eval/scripts/archive_run.pyexecutablescript9.1 KB
eval/scripts/codex_session.pyexecutablescript10.6 KB
eval/scripts/generate_dashboard.pyexecutablescript5.3 KB
eval/scripts/parse_artifacts.pyexecutablescript48.1 KB
eval/scripts/parse_session_logs.pyexecutablescript51.3 KB
eval/scripts/rescore_archived.pyexecutablescript5.5 KB
eval/scripts/score_campaign.pyexecutablescript43.8 KB
eval/scripts/snapshot_changes.pyexecutablescript11.1 KB
eval/scripts/verify_citations.pyexecutablescript8.8 KB
eval/tests/test_dashboard_bugs.pyexecutablescript8.8 KB
eval/tests/test_parse_session_logs.pyexecutablescript3.9 KB
eval/tests/test_parse_session_logs_round_inference.pyexecutablescript11.0 KB
eval/tests/test_score_campaign_diluted.pyexecutablescript9.4 KB
eval/viewer/dashboard_template.htmlexecutablescript15.1 KB
instrumentation/sitecustomize.pyexecutablescript3.6 KB
orchestration/audit-protocol.mdmarkdown9.3 KB
orchestration/debate-protocol.mdmarkdown11.2 KB
orchestration/integration-logic.mdmarkdown10.3 KB
orchestration/parallel-tracks.mdmarkdown8.9 KB
references/artifact-layout.mdexecutablescript22.4 KB
references/audit-invariants.mdmarkdown10.6 KB
references/champion-common-patterns.mdmarkdown3.4 KB
references/crossover-probing.mdexecutablescript4.7 KB
references/cudagraph-safety.mdmarkdown2.7 KB
references/debate-rules.mdmarkdown4.7 KB
references/debate-scoring-rubric.mdexecutablescript16.7 KB
references/e2e-delta-math.mdexecutablescript17.9 KB
references/e2e-latency-guide.mdexecutablescript9.6 KB
references/fusion-feasibility-heuristics.mdmarkdown5.6 KB
references/gpu-pool.mdexecutablescript5.4 KB
references/impl-track-rules.mdexecutablescript13.8 KB
references/nsys-profiling-guide.mdmarkdown9.8 KB
references/optimization-categories.mdexecutablescript11.0 KB
references/shutdown-protocol.mdexecutablescript3.1 KB
references/technology-selection.mdmarkdown5.0 KB
references/torch-compile-contract.mdmarkdown13.8 KB
references/validation-defaults.mdexecutablescript34.2 KB
references/workload-invocation.mdexecutablescript4.9 KB
references/writing-style.mdexecutablescript6.9 KB
scripts/ammo_state.pyexecutablescript142.0 KB
scripts/artifact_layout.jsonconfig16.3 KB
scripts/check_projection_accuracy.pyexecutablescript11.7 KB
scripts/create_evidence_template.pyexecutablescript9.7 KB
scripts/create_worktree_with_build.shexecutablescript10.2 KB
scripts/generate_validation_report.pyexecutablescript35.7 KB
scripts/gpu_reservation.pyexecutablescript27.1 KB
scripts/hook_cmd_classify.pyscript13.3 KB
scripts/mine_trace.pyscript29.4 KB
scripts/monitor_queue_ack.pyexecutablescript6.1 KB
scripts/ncu_sanity.shexecutablescript5.2 KB
scripts/ncu_sanity_driver.pyexecutablescript8.5 KB
scripts/new_target.pyexecutablescript22.0 KB
scripts/preflight_vllm_api.pyexecutablescript12.3 KB
scripts/reconcile_track_state.pyexecutablescript15.8 KB
scripts/remove_worktree_cleanup.shexecutablescript1.1 KB
scripts/resolve_codex_transcript.pyexecutablescript3.6 KB
scripts/run_vllm_bench_latency_sweep.pyexecutablescript249.6 KB
scripts/transcript_filter.pyexecutablescript38.4 KB
scripts/transitions.jsonexecutablescript1.5 KB
scripts/verify_validation_gates.pyexecutablescript88.8 KB
tests/agents/test-champion.mdexecutablescript20.4 KB
tests/agents/test-impl-champion.mdexecutablescript14.3 KB
tests/agents/test-implementer.mdexecutablescript15.8 KB
tests/agents/test-orchestrator.mdexecutablescript33.8 KB
tests/agents/test-researcher.mdexecutablescript21.0 KB
tests/agents/test-transcript-monitor.mdexecutablescript13.4 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.mdmarkdown1.1 KB
tests/fixtures/diluted_pass/legacy_no_phase.golden.summary.jsonconfig582 B
tests/fixtures/diluted_pass/legacy_no_phase.jsonconfig400 B
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.mdmarkdown2.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.golden.summary.jsonconfig1.0 KB
tests/fixtures/diluted_pass/legacy_scalar_phase.jsonconfig626 B
tests/fixtures/mining/real_campaign_bottleneck_analysis.mdmarkdown20.7 KB
tests/run_all.shexecutablescript1.6 KB
tests/scripts/test_transcript_filter.pyexecutablescript41.1 KB
tests/test_agent_runtime_contracts.pyexecutablescript1.8 KB
tests/test_ammo_state.pyexecutablescript71.7 KB
tests/test_artifact_layout_manifest.pyscript11.6 KB
tests/test_audit_started.pyscript18.2 KB
tests/test_correctness_comparator.pyscript12.6 KB
tests/test_diluted_pass.pyscript24.4 KB
tests/test_drift_compat.pyscript5.1 KB
tests/test_evidence_freshness.pyexecutablescript5.6 KB
tests/test_fail_closed.pyexecutablescript14.7 KB
tests/test_gpu_reservation.pyexecutablescript24.8 KB
tests/test_gpu_reservation_session_id.pyexecutablescript11.2 KB
tests/test_guidance_semantic_contract.pyscript24.9 KB
tests/test_hook_cmd_classify_parity.pyscript11.7 KB
tests/test_hook_semantics.pyexecutablescript95.3 KB
tests/test_mine_trace.pyscript24.2 KB
tests/test_mining_invalidated_schema.pyexecutablescript9.4 KB
tests/test_new_target_dp_ep.pyexecutablescript7.4 KB
tests/test_new_target_v2_scaffold.pyexecutablescript13.1 KB
tests/test_parse_artifacts.pyexecutablescript3.0 KB
tests/test_policy_convergence.pyscript797 B
tests/test_preflight_vllm_api.pyscript13.3 KB
tests/test_projection_accuracy_v2.pyexecutablescript4.1 KB
tests/test_reconcile_track_state.pyexecutablescript18.2 KB
tests/test_resolve_codex_transcript.pyexecutablescript3.1 KB
tests/test_state_schema_semantics.pyexecutablescript7.6 KB
tests/test_status_set_sync.pyexecutablescript9.1 KB
tests/test_sweep_baseline_from_arm.pyscript11.0 KB
tests/test_sweep_decode_capture.pyexecutablescript31.8 KB
tests/test_sweep_dp.pyexecutablescript45.4 KB
tests/test_sweep_fresh_cache.pyscript5.5 KB
tests/test_sweep_profiler_guard.pyscript22.6 KB
tests/test_sweep_real_prompts.pyscript20.1 KB
tests/test_sweep_refactor.pyexecutablescript37.6 KB
tests/test_sweep_round_slot.pyexecutablescript7.4 KB
tests/test_validation_report_phase_triage.pyexecutablescript10.0 KB
tests/test_validation_report_v2.pyexecutablescript6.2 KB
tests/test_verify_validation_gates.pyexecutablescript52.9 KB
tests/test_workload_matrix.pyexecutablescript19.3 KB

references/e2e-delta-math.md

View on GitHub
# E2E Delta Math (Stop‑Condition)

A big microbench speedup can still give a tiny end-to-end gain. The multiplier you put into the Amdahl formula must be the kernel's contribution to the **end‑to‑end wall clock**, not its share inside a single phase. This document defines that multiplier (`f_e2e`), shows how to derive it from sweep + profiling outputs, and gives the single boundary equation champions project with. Read it before you write any projected E2E number.

## f_e2e: The Correct Amdahl Multiplier

Let:
- `T_e2e` = total end-to-end wall time of one request (prefill + decode).
- `T_component` = wall-clock time spent in the target component **across the whole request**.
- `f_e2e = T_component / T_e2e` — the component's share of E2E wall time.
- `s` = component speedup factor (`T_component_old / T_component_new`, so `s ≥ 1` is faster).

Then the standard Amdahl form for E2E improvement is:

```text
E2E_improvement_fraction = f_e2e × (1 - 1/s)
E2E_improvement_pct = 100 × E2E_improvement_fraction
```

### Why `f_decode` is wrong for prediction

`f_decode` (component share within decode-step GPU time) is a **decode diagnostic**. Rank campaign opportunity by addressable production E2E impact — `f_e2e × removable_fraction` — and use `f_e2e` as the Amdahl multiplier. Two reasons:

1. Decode time is only a fraction of E2E (`decode_share_of_e2e < 1` whenever prefill > 0).
2. Decode wall-time is not all kernel-busy time (`decode_busy < 1` whenever inter-kernel overhead exists).

Using `f_decode` directly over-projects E2E gains by `1 / (decode_busy × decode_share_of_e2e)`. That is a **2.1×** over-projection in the worked example below, and it reaches **5–50×** on prefill-heavy or low-busy workloads.

`f_decode` stays in the ranking table as a diagnostic column; `f_e2e` is the column champions plug into Amdahl.

## Conversion Formula

```
f_e2e = f_decode × decode_busy × decode_share_of_e2e
```

| Variable | Definition | Data source |
|----------|-----------|-------------|
| `f_decode` | Component's share of decode-step GPU time | nsys trace; ranking metric only |
| `decode_busy` | `decode_kernel_time / decode_wall_time`. Fraction of decode wall-time that is actual GPU kernel work (vs scheduling, launch gaps). | Tier-0: `merged_busy_per_step × n_decode_steps / decode_avg_s`. Union overlapping kernel intervals (`merged_busy`); never sum concurrent kernels, and scale the per-step profiled window up to the full `decode_avg_s` before dividing. Tier-1: `Σ all_kernel_dur / (avg_latency × num_profiled_iters)` (sweep-level aggregate; **never** mix Chrome-trace timestamps with `RequestOutput.metrics` `time.monotonic` clocks). |
| `decode_share_of_e2e` | `decode_avg_s / avg_s`. Fraction of the authoritative E2E wall that is decode. | `e2e_latency_results.json` — never read the stored key (Denominator rule below). If phase metrics are unavailable, `OL / (IL + OL)` is a disclosed heuristic, not a timing bound. |

**Denominator rule.** Both factors denominate the authoritative E2E wall (`avg_s`). Never normalize a share to `prefill_avg + decode_avg`: that sum is a per-request-mean sub-interval, it discards the admission/ramp residual — real wall time, ~40% of the wall on prefill-heavy workloads — and it can attribute more E2E to decode kernels than decode physically occupies. `mine_trace.py` re-derives both shares from `avg_s` and `decode_avg_s` at point of use and WARNs when the stored key disagrees; never read `decode_share_of_e2e` from JSON. `decode_busy` likewise denominates the decode region's own wall taken from the **same** trace as its merged-busy numerator — never a per-step figure divided by a window from a different run or shape.

Consumers (`verify_validation_gates.py`, `generate_validation_report.py`, `ammo_state.py`) derive `decode_avg_s / avg_s` at point of use and never trust the stored key, because a persisted baseline outlives the emitter that wrote it.

**Ceiling and closure are transcription checks, not accuracy checks.** Under the mandated formulas the four-slice closure is an algebraic identity (`(avg−dec)/avg + busy·(dec/avg) + (1−busy)·(dec/avg) ≡ 1` for any inputs) and the ceiling reduces to `busy ≤ 1`. Both hold exactly for correct and badly wrong numbers alike. Run them — a violation is arithmetic proof of a denominator or transcription slip — but never cite passing them as evidence the measurement is right.

**Provenance is what validates the numbers.** Read the derivation, not just the identities:

- `decode_wall_s` and the ramp residual come from `avg_s` / `decode_avg_s` in the clean profiler-free sweep, never from a profiled run's inflated latency.
- `decode_busy`: the numerator is merged (union) busy over the decode region; the denominator is that region's wall in the same trace. Publish per-rank values — a spread wider than a few points means a wrong region boundary or a leaked warmup gap.
- `inter_kernel_share` must be corroborated by *measured* idle (summed true gaps from the merged sweep), not only by `(1−busy)·decode_share`. Disagreement means the busy figure is wrong.
- State the capture's shape. A reduced `--nsys-output-len` capture has a short, ragged decode tail: its prefill region is still valid (prefill depends on ISL and batch, not OSL), but it cannot supply decode step counts or decode shares.
- Segment phases by a per-step marker plus a phase-distinguishing kernel signature (e.g. attention `gridZ` at chunk size vs decode size), not by wall-clock gap size. Resolve the marker to a single kernel ID — name-substring matches can inflate the step count by an integer factor.

The researcher computes `decode_busy` and `decode_share_of_e2e` once per BS in Stage 2 and writes both into the `## Workload Dilution` table in `bottleneck_analysis.md`. Champions read those values; they do not re-derive them.

## Worked Example — Qwen3.6-27B-FP8, H100, BS=8

From the motivating campaign (see Appendix A of the design spec):

| Quantity | Value | Source |
|----------|-------|--------|
| `f_decode` (DeepGEMM gate_up) | 0.066 | nsys decode-region kernel time |
| `decode_busy` | 0.57 | `9.1 s / 15.9 s` from `## Workload Dilution` |
| `decode_share_of_e2e` | 0.82 | `15.9 s / 19.4 s` from `RequestOutput.metrics` |

```
f_e2e = 0.066 × 0.57 × 0.82 = 0.031
```

A 1.18× kernel speedup (the BW-utilization-limited ceiling for this DeepGEMM shape) projects:

```
E2E_improvement = 0.031 × (1 - 1/1.18) = 0.031 × 0.153 = 0.005  (0.47%)
```

…which may be below `min_e2e_improvement_pct` depending on the configured threshold. A naïve `f_decode`-based projection gives `0.066 × 0.153 = 1.0%`, over-projecting by **2.1×** and giving the champion false confidence the proposal is shippable. After conversion, the same evidence correctly classifies the proposal as marginal. Always compare against the campaign's configured `min_e2e_improvement_pct` (see `references/validation-defaults.md` for default).

## Conversion Is Always Required

Always convert, and use the measured `f_e2e` published for the exact workload bucket. Never decide that dilution is "small enough" to substitute `f_decode`; if the values are close, the conversion simply produces a close answer. When phase timing is missing, label any fallback heuristic and keep the resulting uncertainty or bound in the evidence scope.

## Prefill-Active Components — Why f_e2e Is a Lower Bound

The `f_e2e = f_decode × decode_busy × decode_share_of_e2e` formula handles **decode-only** kernels — kernels whose contribution to E2E is entirely through the decode phase. Some kernels run during BOTH prefill and decode (e.g., attention, embedding, RMSNorm). For these:

- The published `f_e2e` is a **lower bound** on the kernel's true E2E contribution.
- Actual gains from speeding up the kernel may *exceed* the projection because the kernel also accelerates prefill.
- The researcher flags such components with `prefill-active? = Yes` in the Top Components table; champions targeting them should treat their projection as conservative and note the upside in their proposal.

The decode-only bound may justify implementation when it clears the campaign floor; shipping still requires clean Gate 5.3b validation.

## Four-Slice Composition Model

Total E2E wall-time decomposes into four exclusive slices:

```
total_e2e = prefill_time + decode_kernel_time + decode_inter_kernel_time + other
```

Equivalently, in the dimensionless shares the researcher publishes:

```
1.0 = prefill_share_of_e2e + (decode_busy × decode_share_of_e2e) + inter_kernel_share + other_share
        prefill              decode_kernel                          decode_inter_kernel  other (small)
```

Where:

- `prefill_share_of_e2e` = `(total_e2e_s - decode_avg_s) / total_e2e_s` — the prefill/ramp region wall. Never `prefill_avg_s / total_e2e_s`: per-request `prefill_avg_s` (`first_token_ts - scheduled_ts`) excludes admission wait and the batch-fill ramp under chunked prefill, understating the slice several-fold on prefill-heavy workloads.
- `decode_busy × decode_share_of_e2e` — the slice that is actual decode-step kernel execution (sum of all `f_e2e` values for decode-only kernels equals this slice, modulo measurement noise).
- `inter_kernel_share = (1 - decode_busy) × decode_share_of_e2e` — decode wall time spent NOT in kernels (scheduling, launch gaps, host-side overhead).
- `other` — small residual (warmup, KV management, cleanup); typically < 0.02.

Each mechanism attacks a different slice; the slice decides which scope is EV-eligible (§ Pre-Implementation Magnitude — Slice Split).

**Window rule.** All three factors of a phase row (`f_phase`, `phase_busy`, `phase_share_of_e2e`) must use the **same** step window. A transient excluded from one side must be excluded from the other — the span from the wall, the kernel from the busy numerator; either alone is a silent ~10% error. Publish the step range and transient treatment beside the table, and verify the stated formula reproduces the published number.

**Family rule.** Define component families by **enumerated symbol lists**, not substring rules (prefix matchers under-count 15-30%). Completeness requires an **exhaustive partition** — every symbol assigned to one family, families summing to the window total with a stated residual; until one exists, family rows are lower bounds. Reconcile instance counts against per-step cadence (k× expected rate = k call sites). Publish the in-CUDA-graph fraction of any launch-overhead row. Derive secondary tables from the partition or delete them — never hand-maintain two tables over one window.

## Projection Authority — One Boundary Equation

There is one projection equation. Project the delta at the declared production boundary, occurrence-weighted, over baseline E2E:

```
E2E_improvement = (Δ_at_declared_boundary × occurrence_count) / baseline_E2E
```

**Non-overlap rule.** Never sum separately-measured async intervals: CPU API duration, GPU gaps, memcpy, and kernel time do NOT add. If the work you are crediting overlaps, enlarge the boundary to enclose it and do ONE A/B across the enlarged boundary — never add two independent measurements.

**Evidence ladder.** Only `bound`/`production_boundary`/`clean_e2e`-scope magnitude enters a projection; `proxy` magnitude never does (authority: `references/debate-rules.md` § Evidence-Scope Ladder).

**Frame limits.** Optimizations that change occurrence count (e.g. speculative decode) or realize benefit only under load (scheduler / throughput / memory-footprint) are OUT of this frame — instrument them with a phase / throughput measurement, not this equation.

## Pre-Implementation Magnitude — Slice Split

The EV-eligible pre-impl magnitude depends on which slice the mechanism attacks:

- **Kernel-work mechanisms** (kernel replacement, fusion, custom kernel, layout transform): the CUDA-graphed kernel A/B microbenchmark is EV-eligible only when it satisfies `debate-rules.md` § Production Baseline Provenance. A kernel harness that fails dispatch/identity confirmation is `proxy`, not `production_boundary`, and is EV-ineligible.
- **Host / dispatch / inter-kernel mechanisms**: the ordinary EV-eligible pre-impl magnitude is the device-work-removed zero-cost `bound`. Host-gap / dispatch savings become creditable only from an actual-runner boundary A/B. A host-only idea with no removable device work may receive one explicitly contingent implementation spike when a measured production host-slice ceiling meets or exceeds the campaign floor and feasibility is grounded. The ceiling is not expected magnitude and supplies no EV. The spike is the track's first Gate 5.2 execution and uses the normal Gate 5.2 artifacts/state fields; if its boundary remains unchanged, passing it satisfies Gate 5.2 and no duplicate A/B is run. Use a boundary-appropriate clock—host monotonic timestamps or NVTX+Nsys for CPU/dispatch intervals, CUDA events only for GPU intervals. Compare the occurrence-weighted E2E-equivalent improvement from the boundary equation above with the campaign floor, not the interval-local percentage. Stop immediately unless it meets or exceeds the floor; run no expensive clean E2E sweep first.

The micro-experiment must exercise the actual production code path; scheduler / preamble / collective mocks are insufficient.

## Inductor Baseline Parity (Gate 5.2 → E2E Translation)

Gate 5.2 measures the declared production boundary, including the production
compile/fusion behavior. An isolated CUDA-graph harness is binding only when
trace and dispatch evidence prove that its baseline and optimized boundaries
are identical to the compiled production boundary. Otherwise the isolated
result is `proxy` evidence: measure the compiled fused chain or enlarge the
boundary to an actual-runner interval before using its magnitude.

Reason: vLLM's Inductor pass pipeline (§ Which Passes to Check) can fuse the target chain into one op at compile time, so an unfused Gate 5.2 baseline may measure against code that never runs in production.

### How to Check for Inductor Fusion

Before projecting E2E from Gate 5.2 kernel speedup, verify the ACTUAL per-kernel time of the baseline chain inside the compiled CUDA graph:

1. Open the Stage 1 nsys trace (`rounds/{N}/profiling/nsys/baseline_bs{BS}.nsys-rep`)
2. Grep for your target kernel names in the decode window
3. If the unfused chain (e.g., separate `rms_norm` + `dynamic_scaled_fp8_quant`) does NOT appear — but a single fused kernel does (e.g., `rms_norm_dynamic_per_token_quant`) — then Inductor has already fused your target

### Corrected Projection

When Inductor has fused the baseline:

```
effective_speedup = inductor_fused_kernel_time / your_kernel_time   (from nsys)
E2E_improvement = f_e2e × (1 - 1/effective_speedup)
```

An isolated unfused proxy must not be substituted:

```
E2E_improvement = f_e2e × (1 - 1/proxy_unfused_speedup)   ← not Gate 5.2
```

Compare the corrected production-boundary projection with the campaign's configured floor; there is no universal kernel-speedup cutoff.

### Which Passes to Check

Enumerate the passes in the installed vLLM; do not trust a roster in this file:

```bash
grep -rn "class .*FusionPass" vllm/compilation/
```

Read each matched pass's pattern. Overlap classes to look for: norm+quant,
activation+quant, rope+KV-cache write, and collective+norm. If your target
chain matches a live pattern, your Gate 5.2 baseline is likely already fused in
production.

## Practical Usage in Phase 0 / Phase 4

1. Read `f_decode`, `decode_busy`, `decode_share_of_e2e`, `inter_kernel_share`, `prefill_share_of_e2e` from the `## Workload Dilution` table in `bottleneck_analysis.md`.
2. If the researcher already published `f_e2e` for your component (which they do for decode-only candidates in the Top Components table), use it directly — no conversion needed.
3. Otherwise compute `f_e2e` per the formula above.
4. Project with the one boundary equation (§ Projection Authority).
5. Solve for the kernel speedup `s` (or analogous variable) needed to clear the `min_e2e_improvement_pct` threshold.
6. If the required `s` is implausibly large given the BW/compute ceiling, switch to:
   - a different target where `f_e2e` is larger, OR
   - document the limitation and stop the proposal.

## Crossover Prediction from Kernel Data

When an optimization improves some batch sizes but regresses others, predict the crossover batch size from kernel-level measurements. That way you do not run expensive E2E benchmarks at every intermediate BS.

### Per-BS Delta Math

The standard formula generalizes to per-BS:

```
predicted_e2e_improvement(BS) = f_e2e(BS) × (1 - T_kernel_opt(BS) / T_kernel_base(BS))
```

Where `f_e2e(BS)` is the per-BS row in the `## Workload Dilution` table. **Use `f_e2e(BS)`, not `f_decode(BS)`** — see Conversion Is Always Required above.

### Why f Varies with Batch Size

The component share is NOT constant across batch sizes:
- At BS=1, `decode_busy` is often higher (smaller launch gaps relative to kernel work) and a kernel may be 8% of E2E (`f_e2e = 0.08`).
- At BS=32, the same kernel may be 3% (`f_e2e = 0.03`) because (a) the kernel scales superlinearly, (b) `decode_busy` may drop as more launches fit into a step, (c) decode_share may shift if prefill scales differently.

Extract per-BS `f_e2e` from the per-bucket `## Workload Dilution` rows. For an intermediate BS that was not profiled, interpolate linearly between the two rows that BRACKET it — the nearest profiled BS below and the nearest profiled BS above. This is approximate but sufficient for crossover prediction.

### Finding the Crossover

```
crossover_bs = max(BS) where predicted_e2e_improvement(BS) >= noise_tolerance_pct / 100
```

### When Kernel Prediction Is Unreliable

If the warm-cache vs cold-cache kernel speedup ratio exceeds 1.5× at any probed BS, the kernel prediction may not reflect production behavior (where L2 cache pressure from the full model pipeline dominates). In this case:
- For narrow BS ranges (< 15 values): fall back to E2E binary search.
- For wide BS ranges (>= 15 values): gate to exact PASS batch sizes only (skip probing).

See `references/crossover-probing.md` for the full probing protocol.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall