Skip to content

Ascend/msagent / skills/operator/ops-performance-tuning

ops-performance-tuning

Verified

Compile, profile, diagnose, optimize, and compare Ascend NPU operators across CANN versions and A2/A3/A5 for Ascend C, CATLASS, Triton-Ascend, TileLang-Ascend, PyPTO, and SHMEM/MC2. Use when an operator already has a runnable implementation and the user asks for msOpProf collection, bottleneck analysis, source-level tuning, or a reproducible before/after performance report. Do not use as the primary workflow for operator creation, migration, or unresolved correctness failures.

FileKindSize
SKILL.mdskill md15.4 KB
evals/evals.jsonconfig3.6 KB
references/case-routing.mdmarkdown30.0 KB
references/cases/ascendc/pr_ascendc_allgathermm_scalar_align.mdmarkdown1.7 KB
references/cases/ascendc/pr_ascendc_cann_samples_perf_story.mdmarkdown3.9 KB
references/cases/ascendc/pr_ascendc_cgdr_vf_inverse.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_conv3ddx_tque_tbuf.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_layernorm_areg.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_logit_a5_microapi.mdmarkdown1.9 KB
references/cases/ascendc/pr_ascendc_matmul_case_18x.mdmarkdown1.5 KB
references/cases/ascendc/pr_ascendc_matmul_full_load_wiki.mdmarkdown1.6 KB
references/cases/ascendc/pr_ascendc_matmul_vcv_wiki.mdmarkdown1.8 KB
references/cases/ascendc/pr_ascendc_mc2_fusion.mdmarkdown1.4 KB
references/cases/ascendc/pr_ascendc_mhc_24x.mdmarkdown1.2 KB
references/cases/ascendc/pr_ascendc_msopprof_add_block_imbalance.mdmarkdown1.7 KB
references/cases/ascendc/pr_ascendc_quantize_a2_micro_opt.mdmarkdown2.0 KB
references/cases/ascendc/pr_ascendc_rotate_quant_blaze.mdmarkdown2.8 KB
references/cases/ascendc/pr_ascendc_tiling_l2_official.mdmarkdown1.6 KB
references/cases/catlass/pr_catlass_baseline.mdmarkdown1.3 KB
references/cases/catlass/pr_catlass_fixpipe_dualdst.mdmarkdown1.6 KB
references/cases/catlass/pr_catlass_grouped_matmul_tiling.mdmarkdown1.5 KB
references/cases/catlass/pr_catlass_mstuner_3p8x.mdmarkdown1.9 KB
references/cases/catlass/pr_catlass_swizzle_balance.mdmarkdown1.6 KB
references/cases/pypto/pr_pypto_gqa_decode_vector_merge.mdmarkdown2.4 KB
references/cases/pypto/pr_pypto_l2_hit_layout_s13.mdmarkdown1.9 KB
references/cases/pypto/pr_pypto_per_matmul_tile_shapes.mdmarkdown2.4 KB
references/cases/pypto/pr_pypto_small_shape_matmul.mdmarkdown2.3 KB
references/cases/pypto/pr_pypto_weight_none_cacheable.mdmarkdown2.6 KB
references/cases/shmem/pr_shmem_alltoallv_mr152.mdmarkdown2.4 KB
references/cases/shmem/pr_shmem_reduce_scatter_serial.mdmarkdown2.6 KB
references/cases/tilelang/pr_tilelang_benchmark_vs_ascendc.mdmarkdown1.7 KB
references/cases/tilelang/pr_tilelang_compiler_prs.mdmarkdown2.3 KB
references/cases/tilelang/pr_tilelang_fa_pr698.mdmarkdown1.5 KB
references/cases/tilelang/pr_tilelang_gqa_bwd_pr1494.mdmarkdown1.3 KB
references/cases/tilelang/pr_tilelang_persistent_mqa_decode.mdmarkdown2.2 KB
references/cases/tilelang/pr_tilelang_sfa_pr665.mdmarkdown914 B
references/cases/triton/pr_triton_atomic_simd_24x.mdmarkdown1.8 KB
references/cases/triton/pr_triton_autotune_background.mdmarkdown1.0 KB
references/cases/triton/pr_triton_constexpr_recompile.mdmarkdown1.1 KB
references/cases/triton/pr_triton_lgamma_simt.mdmarkdown1.6 KB
references/compile/compile-ascendc.mdmarkdown9.0 KB
references/compile/compile-catlass.mdmarkdown2.4 KB
references/compile/compile-pypto.mdmarkdown2.0 KB
references/compile/compile-shmem.mdmarkdown2.6 KB
references/compile/compile-tilelang.mdmarkdown2.7 KB
references/compile/compile-triton.mdmarkdown4.4 KB
references/diagnose/diagnose-l2-cache.mdmarkdown2.2 KB
references/diagnose/diagnose-occupancy.mdmarkdown4.0 KB
references/diagnose/diagnose-pipeline.mdmarkdown4.1 KB
references/diagnose/diagnose-roofline.mdmarkdown3.1 KB
references/optimize/optimize-api-usage.mdmarkdown1.7 KB
references/optimize/optimize-ascendc-regbase-vf.mdmarkdown5.9 KB
references/optimize/optimize-ascendc-tiling.mdmarkdown935 B
references/optimize/optimize-catlass.mdmarkdown3.0 KB
references/optimize/optimize-data-copy.mdmarkdown870 B
references/optimize/optimize-index.mdmarkdown3.6 KB
references/optimize/optimize-memory-hierarchy.mdmarkdown599 B
references/optimize/optimize-pipeline.mdmarkdown1.0 KB
references/optimize/optimize-pypto.mdmarkdown1.5 KB
references/optimize/optimize-shmem.mdmarkdown5.1 KB
references/optimize/optimize-tilelang.mdmarkdown1.7 KB
references/optimize/optimize-triton.mdmarkdown3.8 KB
references/profile/benchmark-harness.mdmarkdown2.2 KB
references/profile/profile-msopprof.mdmarkdown7.4 KB
references/troubleshooting.mdmarkdown5.8 KB
scripts/bound_analyzer.pyscript9.0 KB
scripts/delta_report.pyscript4.8 KB
scripts/env_check.shscript5.0 KB
scripts/result_saver.pyscript5.0 KB

references/optimize/optimize-catlass.md

View on GitHub
# §3 CATLASS

> 本文件覆盖 CATLASS 模板库的优化技术(workspace 驻留 L2、容量约束、DispatchPolicy、诊断映射、Swizzle 等)。

## 3.1 头号优化:消除整块 C 的 HBM 往返(workspace 驻留 L2)

- **理论说明**:`MatmulActivation/MatmulEpilogue` 把整块 `[M,N]` fp32 C 写回 HBM 再读回(流量 2·M·N·4 B);换为"多级轮转 workspace + `MmadAtlasA2PreloadAsyncWithCallback`"后 C 驻留 L2,AIC/AIV 细粒度重叠;量化场景用 `QuantMatmulMultiStageWorkspace`。
- **性能对比**:C 驻留 L2 实测命中 **96–99%**。
- **禁止事项**:大 N 场景禁止"仍是整块 HBM 往返时只调 TileShape 就交付"。

## 3.2 容量约束与经验起点

- L1 占用 ≈ `(L1.M·L1.K + L1.K·L1.N)·sizeof·l1Stages ≤ ~512KB`;L0C ≈ `L1.M·L1.N·4 ≤ 128KB`。
- fp16 起点 `L1<128,256,256>` / `L0<128,256,64>`;int8 可到 K=512。

## 3.3 DispatchPolicy 调度策略(流水排布)

机制细节见 catlass 官方文档 `docs/dispatch_policies`。

| DispatchPolicy | 机制 |
|---|---|
| `MmadAtlasA2Pingpong` | L1/L0 双缓冲(STAGES 控制片数,ENABLE_UNIT_FLAG 控制 Mmad 与 L0C 拷出并行) |
| `MmadAtlasA2Preload` | + ShuffleK(重排 K 轴顺序优化 Cache 利用率)+ Block 间预加载 |
| `MmadAtlasA2PreloadAsync` | nBuffer 机制,L1/L0 可配不同缓冲片数,支持 group 间预加载(用于 Grouped Matmul) |

## 3.4 瓶颈诊断 → 调参映射表

| 症状 | 手段 |
|---|---|
| HBM 带宽高 + L2 命中低 | 换 workspace Kernel(见 §3.1) |
| MTE2 占比高 | Preload / ShuffleK / 调大 N-tile |
| Cube 高 Vector 闲 | 调大 K-tile |
| 任务块 < AIC 核数 | 调小 M-tile / SplitK |
| 小 shape | SmallMatmul Kernel |
| A 重读 | FullLoadA |
| 同步空泡 | workspaceStages |
| Cube ~90%+ | 即接近 roofline,再提速受硬件结构约束 |

## 3.5 Swizzle 负载均衡

- **理论说明**:BlockScheduler swizzle 调整 AI Core 间任务分配均衡性(提升流水线饱满度)。
- **代码/参数模式**:swizzle `<3,1>` → `<4,1>`。
- **性能对比**:性能从 40.6µs 提升到 35.3µs(出处:catlass `docs/catlass_optimize_guidance.md`,<https://gitcode.com/cann/catlass/blob/v1.2.0/docs/catlass_optimize_guidance.md>;经腾讯云社区转载 <https://cloud.tencent.com/developer/article/2612963>;案例全文见 [cases/catlass/pr_catlass_swizzle_balance.md](cases/catlass/pr_catlass_swizzle_balance.md))。

## 3.6 可调参数总表

TileShape(L1/L0 层)、DispatchPolicy、Swizzle 策略(含 Swizzle offset、Preload、Split-K、TLA 等待试项)、Kernel 类型。

## 3.7 tiling/swizzle 影响量级佐证

msTuner 在 Ascend 950 上的 GEMM tiling 寻优实测:初始 case `task_duration: 164.028 us`(256x256x128_256x256x32_swizzle3x1)→ Top-1 配置(128x128x128_128x128x64_swizzle3x1)**43.291 us**,约 3.8 倍差异(数据出自 catlass MR !966 测试节,<https://gitcode.com/cann/catlass/pull/966>;案例全文见 [cases/catlass/pr_catlass_mstuner_3p8x.md](cases/catlass/pr_catlass_mstuner_3p8x.md))。
planning-and-task-breakdownaddyosmani/agent-skillsBreaks work into ordered tasks. Use when you have a spec or clear requirements and need to break work into implementable tasks. Use when a task feels too large to start, when you need to estimate scope, or when parallel work is possible.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
screenpipethedotmack/claude-memSearch screen and audio history recorded by Screenpipe and optionally save selected context into claude-mem. Use when the user asks to recall desktop activity, a meeting, or something seen in another app through Screenpipe.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
contact-widgetnexu-io/open-designSelf-contained floating chat widget with welcome screen, social links, meeting button, and message input. Single HTML file, zero dependencies.Stars: 108kUpdated Claude CodeCodexVerifiedInstall
meeting-notesnexu-io/open-designMeeting notes page — title bar with attendees, agenda checklist, decisions block, action items table with owners + dates, and a "next meeting" footer. Use when the brief mentions "meeting notes", "minutes", "1:1 notes", "all-hands recap", or "会议纪要".Stars: 108kUpdated Claude CodeCodexVerifiedInstall