Ascend/msagent / skills/profiler/msagent-profiler-breakdown
msagent-profiler-breakdown
VerifiedDB格式Profiling数据场景化性能拆解:自动完成场景确认(框架/任务类型判定)、阶段拆解与单次执行边界拆解,并给出报告。Use when 需要对昇腾 Profiling DB 做阶段拆解、step 归因或生成性能报告, or when the user says "帮我按任务阶段拆解这个Profiling数据", "多层次性能耗时拆解", "分析这个 vllm/sglang/verl 数据,主要耗时都花费在哪里了"。
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 8.0 KB |
| references/breakdown-schema.md | markdown | 4.1 KB |
| references/check-rules.md | markdown | 1.5 KB |
| references/db-usage.md | markdown | 5.8 KB |
| references/frameworks/README.md | markdown | 5.5 KB |
| references/frameworks/verl.md | markdown | 11.9 KB |
| references/frameworks/vllm-instrumentation.md | markdown | 13.6 KB |
| references/frameworks/vllm-profiler-data.md | markdown | 4.6 KB |
| references/frameworks/vllm.md | markdown | 5.6 KB |
| references/report-template.md | markdown | 5.0 KB |
| references/scenario-taxonomy.md | markdown | 5.7 KB |
| resources/scenarios/inference/sglang.json | config | 549 B |
| resources/scenarios/inference/vllm.json | config | 912 B |
| resources/scenarios/rl/slime.json | config | 468 B |
| resources/scenarios/rl/verl.json | config | 1.4 KB |
| resources/scenarios/training/fsdp.json | config | 303 B |
| resources/scenarios/training/megatron.json | config | 556 B |
| resources/scenarios/training/mindspeed.json | config | 318 B |
| scripts/compare_runs.py | script | 12.4 KB |
| scripts/db_query.py | script | 12.7 KB |
| scripts/decompose.py | script | 47.7 KB |
references/frameworks/vllm-instrumentation.md
View on GitHub# vLLM / vLLM-Ascend 打点清单(record\_function\_or\_nullcontext · MSTX)
> 消费 skill:`msagent-profiler-breakdown`(decompose.py,阶段归因 + 单次执行边界)· `layer-decompose`(layer\_map)
> 性质:框架打点**事实清单**(非规则),供三处 `frameworks/vllm.md` 引用;引擎按 scope 名做边界/阶段匹配。
> 来源:代码静态梳理(vLLM 主仓库 + vLLM-Ascend)。**不含** MS Service Profiler、**不含**可观测性/OpenTelemetry traces 打点。
> 范围:仅 `record_function_or_nullcontext` 与 `MSTX`。`torch.profiler.record_function`(CUDA graph capture)与 `annotate_profile`(execute\_ 注解)属于另两条通路,见 §4。
> 互补资源:PyTorch Profiler 采集数据/字段(`torch.profiler` vs `torch_npu.profiler`)与 MSTX 落地口径,见同目录 `vllm-profiler-data.md`。
***
## 0. 机制与开关(先决事实)
| 项 | 说明 |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 唯一实现 | vLLM `vllm/v1/utils.py` 的 `record_function_or_nullcontext(name)`;vLLM-Ascend 直接 `import` 复用,不另造 |
| 开关 | `VLLM_CUSTOM_SCOPES_FOR_PROFILING=1` → `torch.autograd.profiler.record_function`;`VLLM_NVTX_SCOPES_FOR_PROFILING=1` → `nvtx.annotate`;全不设 → `nullcontext`(零开销不打点) |
| 字段 | 打点**只有** **`name`(scope 字符串)一个字段**,无 attributes;落地 DB 为 `PYTORCH_API` 表事件(`name` 经 `STRING_IDS` 反查) |
| shape 线索 | vLLM GPU 走 `torch.profiler.record_shapes` 才有 `Input Dims`;vLLM-Ascend 的 `torch_npu.profiler` **无 record\_shapes**,但 shape 仍可从 DB `PYTORCH_API.inputShapes/inputDtypes`(`STRING_IDS` 反查)取 |
| MSTX | vLLM-Ascend `_ExperimentalConfig(msprof_tx=False)` 显式关闭;`PYTORCH_API.type` 枚举含 `mstx`,仅 `msprof_tx=True` 时才产出,默认无 mstx 记录 |
***
## 1. vLLM 打点全清单(19 处)
### 1.1 gpu\_model\_runner.py —— 引擎主流程(10,一次 step 的时序骨架)
| scope name | 位置 | 语义 | 拆解维度 |
| ----------------------------------------------- | ----------------------------------------- | --------------------- | ---------------- |
| `gpu_model_runner: preprocess` | `vllm/v1/worker/gpu_model_runner.py:4309` | 状态更新 + 输入准备 | step 内阶段 |
| `gpu_model_runner: forward` | `vllm/v1/worker/gpu_model_runner.py:4544` | **模型前向**(最重,一次执行本体) | **step 边界 A 锚点** |
| `gpu_model_runner: postprocess` | `vllm/v1/worker/gpu_model_runner.py:4558` | hidden→logits / PP 广播 | step 内阶段 |
| `gpu_model_runner: sample` | `vllm/v1/worker/gpu_model_runner.py:4688` | 采样 | stage:sample |
| `gpu_model_runner: draft` | `vllm/v1/worker/gpu_model_runner.py:4713` | 投机 drafter 前向 | 投机推理 |
| `gpu_model_runner: bookkeep` | `vllm/v1/worker/gpu_model_runner.py:4816` | bookkeeping sync | step 内阶段 |
| `gpu_model_runner: eplb` | `vllm/v1/worker/gpu_model_runner.py:4860` | EPLB 负载均衡 | 通信/负载 |
| `gpu_model_runner: ModelRunnerOutput` | `vllm/v1/worker/gpu_model_runner.py:4867` | 组装输出 | step 内阶段 |
| `gpu_model_runner: AsyncGPUModelRunnerOutput` | `vllm/v1/worker/gpu_model_runner.py:4895` | 异步输出快照 | step 内阶段 |
| `gpu_model_runner: set_async_sampled_token_ids` | `vllm/v1/worker/gpu_model_runner.py:4924` | 异步 token 回填 | step 内阶段 |
### 1.2 llm\_engine.py —— 引擎层(4)
| scope name | 位置 |
| ---------------------------------- | ---------------------------------- |
| `llm_engine step: get_output` | `vllm/v1/engine/llm_engine.py:311` |
| `llm_engine step: process_outputs` | `vllm/v1/engine/llm_engine.py:315` |
| `llm_engine step: abort_requests` | `vllm/v1/engine/llm_engine.py:327` |
| `llm_engine step: record_stats` | `vllm/v1/engine/llm_engine.py:331` |
### 1.3 scheduler.py —— 调度层(4,CPU 侧)
| scope name | 位置 |
| ---------------------------------------- | -------------------------------------- |
| `schedule: allocate_slots` | `vllm/v1/core/sched/scheduler.py:701` |
| `schedule: get_num_common_prefix_blocks` | `vllm/v1/core/sched/scheduler.py:1281` |
| `schedule: make_cached_request_data` | `vllm/v1/core/sched/scheduler.py:1313` |
| `schedule: update_after_schedule` | `vllm/v1/core/sched/scheduler.py:1429` |
### 1.4 spec\_decode —— 投机推理(1)
| scope name | 位置 | 语义 |
| ---------------------------- | ----------------------------------------------- | ------------- |
| `ngram_proposer_gpu: kernel` | `vllm/v1/spec_decode/ngram_proposer_gpu.py:384` | ngram GPU 提议核 |
***
## 2. vLLM-Ascend 打点全清单(26 处)
### 2.1 model\_runner\_v1.py —— 主流程(7)
| scope name | 位置 | 备注 |
| -------------------- | -------------------------------------------- | ------------------------------- |
| `prepare input` | `vllm_ascend/worker/model_runner_v1.py:2141` | 对应 vLLM `preprocess` |
| `forward` | `vllm_ascend/worker/model_runner_v1.py:2399` | `ExitStack` 形式;**step 边界 A 锚点** |
| `post process` | `vllm_ascend/worker/model_runner_v1.py:2450` | 对应 vLLM `postprocess` |
| `sample_token` | `vllm_ascend/worker/model_runner_v1.py:2570` | 对应 vLLM `sample` |
| `draft_token` | `vllm_ascend/worker/model_runner_v1.py:2617` | 早 PP 分支,投机推理 |
| `draft_token` | `vllm_ascend/worker/model_runner_v1.py:2636` | 常规分支,投机推理 |
| `async_state_update` | `vllm_ascend/worker/model_runner_v1.py:2701` | `ExitStack` 形式 |
### 2.2 EPLB —— 负载均衡(3,Ascend 独有)
| scope name | 位置 |
| ------------------------ | ---------------------------------------------------------- |
| `EPLB generate p2p task` | `vllm_ascend/eplb/eplb_updator.py:111` |
| `EPLB gather moe load` | `vllm_ascend/eplb/eplb_updator.py:131` |
| `EPLB weight D2D wait` | `vllm_ascend/eplb/core/eplb_device_transfer_loader.py:104` |
### 2.3 schedule: 系列(4 套 scheduler 实现 × 各 4 个 = 16)
四组 scope 名完全一致,分布在 4 套 scheduler:
| scope name | patch\_balance\_schedule.py | scheduler\_profiling\_chunk.py | recompute\_scheduler.py | dyntra\_lb\_scheduler.py |
| ---------------------------------------- | --------------------------- | ------------------------------ | ----------------------- | ------------------------ |
| `schedule: allocate_slots` | `:295` | `:353` | `:377` | `:556` |
| `schedule: get_num_common_prefix_blocks` | `:747` | `:697` | `:940` | `:1068` |
| `schedule: make_cached_request_data` | `:770` | `:720` | `:963` | `:1091` |
| `schedule: update_after_schedule` | `:830` | `:758` | `:1042` | `:1194` |
- `patch/platform/patch_balance_schedule.py`:对 vLLM 上游 scheduler 的 platform patch(EPLB)。
- `core/scheduler_profiling_chunk.py` / `core/recompute_scheduler.py` / `core/dyntra_lb_scheduler.py`:Ascend 自研 scheduler 变体(profiling-chunk / recompute / 动态负载均衡)。
### 2.4 非打点(勿混入)
- `vllm_ascend/profiling_config.py:112-114` 的 `record_function_or_nullcontext` 是 **MS Service Profiler 的 symbol/handler 配置**,非本清单范围。
***
## 3. MSTX 打点(结论:均无实际采点)
| 仓库 | 结果 |
| ----------- | ---------------------------------------------------------------------------- |
| vLLM | 无任何 MSTX(CUDA 无此机制) |
| vLLM-Ascend | 仅 `profiler/torch_npu_profiler.py:53` 一处 `msprof_tx=False`(显式关闭),无实际 MSTX 采点 |
- 落地含义:默认落盘 `PYTORCH_API` 表中**不会**出现 `type='mstx'` 记录;step 边界改用 `STEP_TIME` 表 / `forward` scope(`record_function`)。
- 若要启用 MSTX 作 step/层边界:把 `msprof_tx` 改为 `True`(或做成配置),重采后 `PYTORCH_API.type='mstx'` 可作精确锚点。
***
## 4. 打点 → 拆解维度映射与关键澄清(纠正既有 profile 的错误假设)
| 拆解目标 | 可用的真实打点 | 说明 |
| ---------------------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **step 边界(boundary A)** | vLLM `gpu_model_runner: forward`;Ascend `forward`;或 DB `STEP_TIME` 表 | 每次模型执行出现一次,span 起始即一次 forward 起点;`STEP_TIME` 为首要来源 |
| **prefill/decode 判别(stage)** | `record_function_or_nullcontext` 内**无 prefill/decode 字样** | 判别不靠本清单的 scope 名,而靠:① vLLM GPU 的 `annotate_profile` 注解 `execute_context_X_generation_Y`(仅在 `gpu_worker.py`,**Ascend 无**);② `PYTORCH_API.inputShapes` 的 batch/token 维;③ attention kernel 类型(预填充 vs PagedAttention) |
| **投机推理** | vLLM `gpu_model_runner: draft` + `ngram_proposer_gpu: kernel`;Ascend `draft_token` | `draft` span 包住 drafter 前向,可单列投机开销 |
| **层边界(layer)** | **无默认分层打点**(无 `model.layers.{i}`) | 层归属靠 B 策略注入 `record_function`/MSTX、或序列对齐兜底、或 `with_modules`;不可假设已有层打点 |
| EPLB / 通信 | vLLM `gpu_model_runner: eplb`;Ascend `EPLB generate p2p task` / `EPLB gather moe load` / `EPLB weight D2D wait` | 通信在独立 comm stream,归通信子模块,见 `layer-map` |
| 调度(CPU 侧) | `schedule:*` 系列(vLLM 1 套 + Ascend 4 套) | 非 GPU 前向,调度/CPU 开销归因 |
### 修正既有 profile 的 3 处错误命名
1. `record_function("step")` → 实际是 `gpu_model_runner: forward`(vLLM)/`forward`(Ascend);`execute_model` 在 GPU 侧由 `annotate_profile`(`torch.profiler.record_function` 通路)注解,非 `record_function_or_nullcontext`。
2. `record_function("prefill")` / `record_function("decode")` → **代码中不存在**;prefill/decode 判别是 §4 表的②③两条,不可按这两个名字匹配。
3. `model.layers.{i}`(DecoderLayer 层边界)→ **默认不存在**,属 B 策略注入项,不是默认已有打点。
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |