Skip to content

Ascend/msagent / skills/operator/ops-performance-tuning

ops-performance-tuning

Verified

Compile, profile, diagnose, optimize, and compare Ascend NPU operators across CANN versions and A2/A3/A5 for Ascend C, CATLASS, Triton-Ascend, TileLang-Ascend, PyPTO, and SHMEM/MC2. Use when an operator already has a runnable implementation and the user asks for msOpProf collection, bottleneck analysis, source-level tuning, or a reproducible before/after performance report. Do not use as the primary workflow for operator creation, migration, or unresolved correctness failures.

FileKindSize
SKILL.mdskill md15.4 KB
evals/evals.jsonconfig3.6 KB
references/case-routing.mdmarkdown30.0 KB
references/cases/ascendc/pr_ascendc_allgathermm_scalar_align.mdmarkdown1.7 KB
references/cases/ascendc/pr_ascendc_cann_samples_perf_story.mdmarkdown3.9 KB
references/cases/ascendc/pr_ascendc_cgdr_vf_inverse.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_conv3ddx_tque_tbuf.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_layernorm_areg.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_logit_a5_microapi.mdmarkdown1.9 KB
references/cases/ascendc/pr_ascendc_matmul_case_18x.mdmarkdown1.5 KB
references/cases/ascendc/pr_ascendc_matmul_full_load_wiki.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_matmul_vcv_wiki.mdmarkdown1.8 KB
references/cases/ascendc/pr_ascendc_mc2_fusion.mdmarkdown1.4 KB
references/cases/ascendc/pr_ascendc_mhc_24x.mdmarkdown1.2 KB
references/cases/ascendc/pr_ascendc_msopprof_add_block_imbalance.mdmarkdown1.7 KB
references/cases/ascendc/pr_ascendc_quantize_a2_micro_opt.mdmarkdown2.0 KB
references/cases/ascendc/pr_ascendc_rotate_quant_blaze.mdmarkdown2.8 KB
references/cases/ascendc/pr_ascendc_tiling_l2_official.mdmarkdown1.6 KB
references/cases/catlass/pr_catlass_baseline.mdmarkdown1.3 KB
references/cases/catlass/pr_catlass_fixpipe_dualdst.mdmarkdown1.6 KB
references/cases/catlass/pr_catlass_grouped_matmul_tiling.mdmarkdown1.5 KB
references/cases/catlass/pr_catlass_mstuner_3p8x.mdmarkdown1.9 KB
references/cases/catlass/pr_catlass_swizzle_balance.mdmarkdown1.6 KB
references/cases/pypto/pr_pypto_gqa_decode_vector_merge.mdmarkdown2.4 KB
references/cases/pypto/pr_pypto_l2_hit_layout_s13.mdmarkdown1.9 KB
references/cases/pypto/pr_pypto_per_matmul_tile_shapes.mdmarkdown2.4 KB
references/cases/pypto/pr_pypto_small_shape_matmul.mdmarkdown2.3 KB
references/cases/pypto/pr_pypto_weight_none_cacheable.mdmarkdown2.6 KB
references/cases/shmem/pr_shmem_alltoallv_mr152.mdmarkdown2.4 KB
references/cases/shmem/pr_shmem_reduce_scatter_serial.mdmarkdown2.6 KB
references/cases/tilelang/pr_tilelang_benchmark_vs_ascendc.mdmarkdown1.7 KB
references/cases/tilelang/pr_tilelang_compiler_prs.mdmarkdown2.3 KB
references/cases/tilelang/pr_tilelang_fa_pr698.mdmarkdown1.5 KB
references/cases/tilelang/pr_tilelang_gqa_bwd_pr1494.mdmarkdown1.3 KB
references/cases/tilelang/pr_tilelang_persistent_mqa_decode.mdmarkdown2.2 KB
references/cases/tilelang/pr_tilelang_sfa_pr665.mdmarkdown914 B
references/cases/triton/pr_triton_atomic_simd_24x.mdmarkdown1.8 KB
references/cases/triton/pr_triton_autotune_background.mdmarkdown1.0 KB
references/cases/triton/pr_triton_constexpr_recompile.mdmarkdown1.1 KB
references/cases/triton/pr_triton_lgamma_simt.mdmarkdown1.6 KB
references/compile/compile-ascendc.mdmarkdown9.0 KB
references/compile/compile-catlass.mdmarkdown2.4 KB
references/compile/compile-pypto.mdmarkdown2.0 KB
references/compile/compile-shmem.mdmarkdown2.6 KB
references/compile/compile-tilelang.mdmarkdown2.7 KB
references/compile/compile-triton.mdmarkdown4.4 KB
references/diagnose/diagnose-l2-cache.mdmarkdown2.2 KB
references/diagnose/diagnose-occupancy.mdmarkdown4.0 KB
references/diagnose/diagnose-pipeline.mdmarkdown4.1 KB
references/diagnose/diagnose-roofline.mdmarkdown3.1 KB
references/optimize/optimize-api-usage.mdmarkdown1.7 KB
references/optimize/optimize-ascendc-regbase-vf.mdmarkdown5.9 KB
references/optimize/optimize-ascendc-tiling.mdmarkdown935 B
references/optimize/optimize-catlass.mdmarkdown3.0 KB
references/optimize/optimize-data-copy.mdmarkdown870 B
references/optimize/optimize-index.mdmarkdown3.6 KB
references/optimize/optimize-memory-hierarchy.mdmarkdown599 B
references/optimize/optimize-pipeline.mdmarkdown1.0 KB
references/optimize/optimize-pypto.mdmarkdown1.5 KB
references/optimize/optimize-shmem.mdmarkdown5.1 KB
references/optimize/optimize-tilelang.mdmarkdown1.7 KB
references/optimize/optimize-triton.mdmarkdown3.8 KB
references/profile/benchmark-harness.mdmarkdown2.2 KB
references/profile/profile-msopprof.mdmarkdown7.4 KB
references/troubleshooting.mdmarkdown5.8 KB
scripts/bound_analyzer.pyscript9.0 KB
scripts/delta_report.pyscript4.8 KB
scripts/env_check.shscript5.0 KB
scripts/result_saver.pyscript5.0 KB

references/cases/ascendc/pr_ascendc_cann_samples_perf_story.md

View on GitHub
# cann-samples 2_Performance 调优故事(13 个官方递进教程)

> 来源:<https://gitcode.com/cann/cann-samples/tree/master/Samples/2_Performance>(2026-08 验证可达)
> 类型:非 PR(官方递进教程,含可运行代码)

## 基本信息

| 属性 | 值 |
|------|-----|
| 算子类别 | vector-elementwise / matmul / attention / moe / norm / scatter / histogram |
| DSL/框架 | Ascend C |
| 平台 | Ascend 950 (dav-3510),CANN 9.1.0 |
| 总案例数 | 13 |

## 案例索引

| # | 案例 | 类型 | 关键技法 | 路径 |
|---|------|------|---------|------|
| 1 | gelu_eltwise_regbase | vector | MemBase→VF融合→循环拆分→展开→常量外提(5 Case) | `gelu_eltwise_regbase_story/` |
| 2 | flash_attn_lite | attention | CV双槽→AIC双缓冲→task IO双缓冲(v0~v5) | `flash_attn_lite_story/` |
| 3 | scalar_story | scalar-bound | icache预取+静态LocalTensor+消除指针解引用 | `scalar_story/` |
| 4 | simd_vf_story | vector | Broadcast/Elemwise/Reduce SIMD VF 范式 | `simd_vf_story/` |
| 5 | grouped_matmul | matmul | 分组tiling+数据搬运+MXFP4/MXFP8 | `grouped_matmul_story/` |
| 6 | matmul_story | matmul | baseline→SWAT→尾轮均衡→UnitFlag | `matmul_story/` |
| 7 | rms_norm_quant | norm | 多核并行+预加载+带宽+流水+硬件适配 | `rms_norm_quant_story/` |
| 8 | full_quant_fused_infer_attention | attention | per-block 全量化 FIA | `full_quant_fused_infer_attention_score_story/` |
| 9 | moe_init_routing | moe | 多核+带宽+流水+SIMT+硬件适配 | `moe_init_routing_story/` |
| 10 | moe_dispatch_and_combine | moe | MoE dispatch/combine 通信优化 | `moe_dispatch_and_combine_story/` |
| 11 | kv_rms_norm_rope_cache | norm+attn | RMSNorm+RoPE+cache融合, MemBase→RegBase | `kv_rms_norm_rope_cache_story/` |
| 12 | simt_scatter | scatter | SIMT直接GM不规则写+分组+单写者冲突解决 | `simt_scatter_story/` |
| 13 | simt_histogram | histogram | MTE+Vec→SIMT grid-stride→float4→launch_bounds | `simt_histogram_story/` |

## 按 bound 类型归类

| Bound | 对应案例 |
|-------|---------|
| VEC BOUND | gelu_eltwise_regbase, simd_vf_story, rms_norm_quant |
| CUBE BOUND | grouped_matmul, matmul_story |
| Pipeline BOUND | flash_attn_lite, moe_init_routing |
| SCALAR BOUND | scalar_story |
| MEM BOUND | kv_rms_norm_rope_cache, moe_dispatch_and_combine |
| SIMT (A5专有) | simt_scatter, simt_histogram |

## 性能数据(实测)

| 案例 | 版本 | 优化点 | 提升 |
|------|------|--------|------|
| flash_attn_lite | v0→v1 | L0C→UB 优化 | **1.96x** |
| gelu_eltwise_regbase | Case0→Case4 | Membase→RegBase 全流程 | 渐进提升 |
- 优化代码查看:<https://gitcode.com/cann/cann-samples> 仓内 `Samples/2_Performance/` 目录(样例代码直接可看;佐证提交 MR !362 <https://gitcode.com/cann/cann-samples/pull/362>, 已验证可达,验证日期 2026-08)
- 参考:Ascend C 官方"流水优化"技巧华为云博客原文 <https://bbs.huaweicloud.com/blogs/433234>( 已验证可达,验证日期 2026-08)

## 问题与瓶颈
`rms_norm_quant_story` 以渐进式版本演示优化路径——`3_vf`(simd_vf 向量化)、`4_double_buffer`(双缓冲)、`5_ub_utilization`(UB 利用率提升)、`6_binary_sum`(归约优化);另有 `kv_rms_norm_rope_cache_story` 融合算子调优样例。原文未附量化 profiling 数据。

## 优化方法(理论手段)
1. **向量化**(simd_vf);
2. **双缓冲**:`pipe.InitBuffer(inQueueX, 2, 256)` 开 double buffer 隐藏 MTE 搬运时间(与 Ascend C 官方"流水优化"技巧一致);
3. **UB 容量复用 / UB 利用率提升**;
4. **归约优化**(binary_sum)。

## 性能对比
原文未附量化数字。

## 适用范围与警示
- 佐证提交 MR !362 修复上述样例在 dav-3510 的编译错误,其中提到"实测若删除某标量重赋值,NPU 上精度从 100% 降至 ~15%"——说明样例在真实 NPU 上持续验证;也提示删改样例代码需重验精度。
planning-and-task-breakdownaddyosmani/agent-skillsBreaks work into ordered tasks. Use when you have a spec or clear requirements and need to break work into implementable tasks. Use when a task feels too large to start, when you need to estimate scope, or when parallel work is possible.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
screenpipethedotmack/claude-memSearch screen and audio history recorded by Screenpipe and optionally save selected context into claude-mem. Use when the user asks to recall desktop activity, a meeting, or something seen in another app through Screenpipe.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
contact-widgetnexu-io/open-designSelf-contained floating chat widget with welcome screen, social links, meeting button, and message input. Single HTML file, zero dependencies.Stars: 108kUpdated Claude CodeCodexVerifiedInstall
meeting-notesnexu-io/open-designMeeting notes page — title bar with attendees, agenda checklist, decisions block, action items table with owners + dates, and a "next meeting" footer. Use when the brief mentions "meeting notes", "minutes", "1:1 notes", "all-hands recap", or "会议纪要".Stars: 108kUpdated Claude CodeCodexVerifiedInstall