Skip to content

Ascend/msagent / skills/operator/ops-performance-tuning

ops-performance-tuning

Verified

Compile, profile, diagnose, optimize, and compare Ascend NPU operators across CANN versions and A2/A3/A5 for Ascend C, CATLASS, Triton-Ascend, TileLang-Ascend, PyPTO, and SHMEM/MC2. Use when an operator already has a runnable implementation and the user asks for msOpProf collection, bottleneck analysis, source-level tuning, or a reproducible before/after performance report. Do not use as the primary workflow for operator creation, migration, or unresolved correctness failures.

FileKindSize
SKILL.mdskill md15.4 KB
evals/evals.jsonconfig3.6 KB
references/case-routing.mdmarkdown30.0 KB
references/cases/ascendc/pr_ascendc_allgathermm_scalar_align.mdmarkdown1.7 KB
references/cases/ascendc/pr_ascendc_cann_samples_perf_story.mdmarkdown3.9 KB
references/cases/ascendc/pr_ascendc_cgdr_vf_inverse.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_conv3ddx_tque_tbuf.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_layernorm_areg.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_logit_a5_microapi.mdmarkdown1.9 KB
references/cases/ascendc/pr_ascendc_matmul_case_18x.mdmarkdown1.5 KB
references/cases/ascendc/pr_ascendc_matmul_full_load_wiki.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_matmul_vcv_wiki.mdmarkdown1.8 KB
references/cases/ascendc/pr_ascendc_mc2_fusion.mdmarkdown1.4 KB
references/cases/ascendc/pr_ascendc_mhc_24x.mdmarkdown1.2 KB
references/cases/ascendc/pr_ascendc_msopprof_add_block_imbalance.mdmarkdown1.7 KB
references/cases/ascendc/pr_ascendc_quantize_a2_micro_opt.mdmarkdown2.0 KB
references/cases/ascendc/pr_ascendc_rotate_quant_blaze.mdmarkdown2.8 KB
references/cases/ascendc/pr_ascendc_tiling_l2_official.mdmarkdown1.6 KB
references/cases/catlass/pr_catlass_baseline.mdmarkdown1.3 KB
references/cases/catlass/pr_catlass_fixpipe_dualdst.mdmarkdown1.6 KB
references/cases/catlass/pr_catlass_grouped_matmul_tiling.mdmarkdown1.5 KB
references/cases/catlass/pr_catlass_mstuner_3p8x.mdmarkdown1.9 KB
references/cases/catlass/pr_catlass_swizzle_balance.mdmarkdown1.6 KB
references/cases/pypto/pr_pypto_gqa_decode_vector_merge.mdmarkdown2.4 KB
references/cases/pypto/pr_pypto_l2_hit_layout_s13.mdmarkdown1.9 KB
references/cases/pypto/pr_pypto_per_matmul_tile_shapes.mdmarkdown2.4 KB
references/cases/pypto/pr_pypto_small_shape_matmul.mdmarkdown2.3 KB
references/cases/pypto/pr_pypto_weight_none_cacheable.mdmarkdown2.6 KB
references/cases/shmem/pr_shmem_alltoallv_mr152.mdmarkdown2.4 KB
references/cases/shmem/pr_shmem_reduce_scatter_serial.mdmarkdown2.6 KB
references/cases/tilelang/pr_tilelang_benchmark_vs_ascendc.mdmarkdown1.7 KB
references/cases/tilelang/pr_tilelang_compiler_prs.mdmarkdown2.3 KB
references/cases/tilelang/pr_tilelang_fa_pr698.mdmarkdown1.5 KB
references/cases/tilelang/pr_tilelang_gqa_bwd_pr1494.mdmarkdown1.3 KB
references/cases/tilelang/pr_tilelang_persistent_mqa_decode.mdmarkdown2.2 KB
references/cases/tilelang/pr_tilelang_sfa_pr665.mdmarkdown914 B
references/cases/triton/pr_triton_atomic_simd_24x.mdmarkdown1.8 KB
references/cases/triton/pr_triton_autotune_background.mdmarkdown1.0 KB
references/cases/triton/pr_triton_constexpr_recompile.mdmarkdown1.1 KB
references/cases/triton/pr_triton_lgamma_simt.mdmarkdown1.6 KB
references/compile/compile-ascendc.mdmarkdown9.0 KB
references/compile/compile-catlass.mdmarkdown2.4 KB
references/compile/compile-pypto.mdmarkdown2.0 KB
references/compile/compile-shmem.mdmarkdown2.6 KB
references/compile/compile-tilelang.mdmarkdown2.7 KB
references/compile/compile-triton.mdmarkdown4.4 KB
references/diagnose/diagnose-l2-cache.mdmarkdown2.2 KB
references/diagnose/diagnose-occupancy.mdmarkdown4.0 KB
references/diagnose/diagnose-pipeline.mdmarkdown4.1 KB
references/diagnose/diagnose-roofline.mdmarkdown3.1 KB
references/optimize/optimize-api-usage.mdmarkdown1.7 KB
references/optimize/optimize-ascendc-regbase-vf.mdmarkdown5.9 KB
references/optimize/optimize-ascendc-tiling.mdmarkdown935 B
references/optimize/optimize-catlass.mdmarkdown3.0 KB
references/optimize/optimize-data-copy.mdmarkdown870 B
references/optimize/optimize-index.mdmarkdown3.6 KB
references/optimize/optimize-memory-hierarchy.mdmarkdown599 B
references/optimize/optimize-pipeline.mdmarkdown1.0 KB
references/optimize/optimize-pypto.mdmarkdown1.5 KB
references/optimize/optimize-shmem.mdmarkdown5.1 KB
references/optimize/optimize-tilelang.mdmarkdown1.7 KB
references/optimize/optimize-triton.mdmarkdown3.8 KB
references/profile/benchmark-harness.mdmarkdown2.2 KB
references/profile/profile-msopprof.mdmarkdown7.4 KB
references/troubleshooting.mdmarkdown5.8 KB
scripts/bound_analyzer.pyscript9.0 KB
scripts/delta_report.pyscript4.8 KB
scripts/env_check.shscript5.0 KB
scripts/result_saver.pyscript5.0 KB

references/optimize/optimize-triton.md

View on GitHub
# §4 Triton-Ascend

> 本文件覆盖 Triton-Ascend 的优化技术(Vector 类深度优化、30 优化点、SIMT→SIMD 原子操作、constexpr)。

## 4.1 Vector 类算子深度优化

| 技术 | 理论说明 | 代码/参数模式 |
|---|---|---|
| **多 Token 批量处理(首要优化点)** | NPU 访存弱计算强;单 Kernel UB 总容量 192KB,留余量仅用其 50%(85KB)以保证 Double Buffering | `N ≤ 85×1024 / S_token`(S_token=单 Token UB 峰值占用,如 BF16 一次 load+store 时 `hidden_size×2`);用整数除法 `//` 而非 `tl.cdiv` 防 UB 溢出 |
| 减少 kernel 内 Scalar 运算 | 与 pid/循环变量无关的计算外提 | 文字模式:循环不变量外提 |
| 非连续访问逐行读 | `index_select` 类只能逐行读(否则二维 mask 引入大量 Scalar) | 文字模式 |
| 加载与计算交织 | "加载一次、计算一次"隐藏访存延迟;多写入流边算边写 | 文字模式 |
| tl.load 规则 | 合并相同 load/计算/store;避免 `other` 参数(内部触发 `tl.where`,导致 load 无法并行) | 推荐先无掩码 load 再 `tl.where` 组合;连续访问用 `tl.insert_slice` |
| 索引与分支规范 | 用 `tl.arange` 生成二维索引;避免 `tl.where`;避免对同一 tensor 多次 `insert_slice`;if-else 分支同名变量 Shape 必须一致;tensor 必须 `.contiguous()`;不变入参声明 `tl.constexpr` | `tl.arange`、`tl.constexpr` |
| 规约维度选择 | 规约优先选最大维度 | 文字模式 |

## 4.2 30 个优化点

按 1→30 顺序命中式检查,一次迭代只用一个优化点:

- **通用类**:1 入参静态化(`tl.constexpr`)、2 Tiling(连续轴向量化)、3 分核(Grid 匹配核数)、4 离散访存、5 Scalar→Vector、6 避免向量 API 标量降级、7 Pass 消除合并、8 维度合并、9 libdevice、10 循环不变量外提、11 Load 重排序、12 Grid 多路径特化、13 Autotune、14 混合策略、17 冗余边界运算消除、18 Kernel 分裂(多 case 且 speedup<2.0 必查)、30 IR 分析(每轮最后必执行)。
- **结构类**:19 Cube/MTE3 分阶段解耦、20 Host 侧张量拼接、21 Workspace 物化解耦、22 Latency-Bound Tile 合并、23 Device-side Gather 连续化、24 Matmul 链中间 buffer dtype、25 输出预初始化。
- **算子专用**:15 归一化大 BLOCK、16 连续拷贝聚合、26 Interpolate、27 Pooling、28 Matmul Transpose、29 CV 融合(含 pingpong/tiling 3 个子文档)。
- **最终步骤**:Block Size Scaling 必做;多 case 不达标再做 Kernel 分裂。
- 各优化点的实现时只使用目标版本公开 API,并在修改记录中附源码位置与验证命令。

## 4.3 SIMT→SIMD 原子操作向量化(PR 案例)

- **理论说明**:SIMT 模式下带离散掩码的原子操作(atomic add/max/min/and/or/xor)逐元素串行发射,改用 SIMD 向量化路径。
- **性能对比**:shape 16,16,16,`atomic_add_3d INT32`:SIMD **3.309µs** vs SIMT **81.163µs**(约 24.5 倍);`atomic_and_3d INT32` 4.114 vs 96.141;FLOAT `atomic_min_3d` 3.372 vs 92.389;ROLLBACK 路径 3.5~4.3µs(triton-ascend PR #218:<https://github.com/triton-lang/triton-ascend/pull/218>; 该改动后被 revert,见 [cases/triton/pr_triton_atomic_simd_24x.md](cases/triton/pr_triton_atomic_simd_24x.md))。
- **代码示范**:**未提供可直接复用的版本无关实现;执行时必须核对目标版本 API、附源码位置并实测验证**。

## 4.4 constexpr 特化粒度控制

去除不必要的 `tl.constexpr` 修饰,避免运行时参数变化触发 Triton 重复 JIT 编译(vllm-ascend PR #7483:<https://github.com/vllm-project/vllm-ascend/pull/7483>;未附量化数字;案例全文见 [cases/triton/pr_triton_constexpr_recompile.md](cases/triton/pr_triton_constexpr_recompile.md))。
planning-and-task-breakdownaddyosmani/agent-skillsBreaks work into ordered tasks. Use when you have a spec or clear requirements and need to break work into implementable tasks. Use when a task feels too large to start, when you need to estimate scope, or when parallel work is possible.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
screenpipethedotmack/claude-memSearch screen and audio history recorded by Screenpipe and optionally save selected context into claude-mem. Use when the user asks to recall desktop activity, a meeting, or something seen in another app through Screenpipe.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
contact-widgetnexu-io/open-designSelf-contained floating chat widget with welcome screen, social links, meeting button, and message input. Single HTML file, zero dependencies.Stars: 108kUpdated Claude CodeCodexVerifiedInstall
meeting-notesnexu-io/open-designMeeting notes page — title bar with attendees, agenda checklist, decisions block, action items table with owners + dates, and a "next meeting" footer. Use when the brief mentions "meeting notes", "minutes", "1:1 notes", "all-hands recap", or "会议纪要".Stars: 108kUpdated Claude CodeCodexVerifiedInstall