mohitmishra786/low-level-dev-skills / skills/gpu/cuda-tile
cuda-tile
VerifiedCUDA Tile skill for tile-based GPU kernels with cuTile Python and the CUDA Tile C++ API. Use when writing cuTile kernels with ct.load/ct.store/ct.mma, compiling C++ tile kernels with nvcc --enable-tile, exporting Tile IR or cubins ahead of time, choosing between tiles, SIMT CUDA and Triton, or debugging tileiras errors. Activates on queries about CUDA Tile, cuTile, cuda.tile, Tile IR, tileiras, __tile_global__, cuda::tiles, partition_view, ct.kernel, or ct.launch.
CUDA Tile
Purpose
Guide agents through NVIDIA's tile programming model (checked with CUDA 13.5 and cuda-tile 1.8.1): cuTile Python kernels, the C++ Tile API, ahead-of-time export, and when tiles are the right tool.
Triggers
- "How do I write a kernel with cuTile Python?"
- "What is CUDA Tile / Tile IR and how is it different from SIMT CUDA?"
- "How do I compile a C++ tile kernel with nvcc?"
- "Can I compile a cuTile kernel to a cubin without a GPU?"
- "Should I use cuTile or Triton?"
- "Which GPUs support CUDA Tile?"
Workflow
1. The model
In SIMT CUDA you write one thread's work and manage threads, warps and shared memory. In CUDA Tile you write one block's work on whole tiles (small arrays); the compiler maps tiles to threads, tensor cores, TMA and shared memory.
| SIMT CUDA | CUDA Tile | |
|---|---|---|
| Unit of code | One thread | One block, operating on tiles |
| Memory movement | Manual (shared memory, __syncthreads) | load/store of tiles; compiler stages data |
| Tensor cores | wmma/mma PTX or CUTLASS | ct.mma / ct::mma on tiles |
| IR | PTX | Tile IR (bytecode), lowered by tileiras |
| Hardware | All supported GPUs | Compute capability 8.0+ (Ampere and newer) |
New GPU kernel?
├── Dense tensor math (GEMM, attention, reductions) → CUDA Tile or Triton
├── Irregular control flow, per-thread logic, atomics everywhere → SIMT CUDA
├── Python stack, want NVIDIA's own compiler and Tile IR → cuTile Python
├── Existing C++/CUDA codebase → CUDA Tile C++ (nvcc --enable-tile)
└── Need AMD/Intel portability → Triton (also has a Tile IR backend for NVIDIA)
2. cuTile Python
pip install cuda-tile # with a CUDA Toolkit installed (see table below)
pip install "cuda-tile[tileiras]" # no toolkit: pulls tileiras, nvcc and NVVM wheels
pip install cupy-cuda13x # device arrays for the examples (not a cuda-tile dependency)
# Needs driver r580+, Linux x86_64/aarch64 or Windows x86_64, Python 3.11+
| GPU generation | Minimum toolkit / tileiras for Tile |
|---|---|
Blackwell (sm_100, sm_120) | 13.1 |
Ampere, Ada (sm_80, sm_86, sm_89) | 13.2 (FP8 on Ada: 13.5) |
Hopper (sm_90) | 13.3 |
import cupy as cp
import cuda.tile as ct
TILE = 1024
@ct.kernel
def vector_add(a, b, c, tile: ct.Constant[int]):
pid = ct.bid(0) # this block's index along axis 0
ta = ct.load(a, index=(pid,), shape=(tile,)) # load one tile from each input
tb = ct.load(b, index=(pid,), shape=(tile,))
ct.store(c, index=(pid,), tile=ta + tb) # element-wise add, store the tile
n = 1 << 20
a = cp.random.rand(n, dtype=cp.float32)
b = cp.random.rand(n, dtype=cp.float32)
c = cp.empty_like(a)
grid = (ct.cdiv(n, TILE), 1, 1) # one block per tile
ct.launch(cp.cuda.get_current_stream(), grid, vector_add, (a, b, c, TILE))
assert cp.allclose(c, a + b)
indexis in tile units:index=(pid,)withshape=(1024,)covers elementspid*1024 ... pid*1024+1023.ct.Constant[int]parameters are compile-time constants; each distinct value compiles a new variant.- Arrays are device arrays such as CuPy's (as in NVIDIA's quickstart); the runtime looks for
__cuda_array_interface__/__dlpack__.
3. Matrix multiply with tensor cores
@ct.kernel
def matmul(A, B, C, tm: ct.Constant[int], tn: ct.Constant[int], tk: ct.Constant[int]):
bm, bn = ct.bid(0), ct.bid(1) # output tile coordinates
acc = ct.full((tm, tn), 0, dtype=ct.float32) # fp32 accumulator tile
for k in range(ct.cdiv(A.shape[1], tk)):
# Zero-pad the partial last K tile so out-of-range elements add nothing
a = ct.load(A, index=(bm, k), shape=(tm, tk), padding_mode=ct.PaddingMode.ZERO)
b = ct.load(B, index=(k, bn), shape=(tk, tn), padding_mode=ct.PaddingMode.ZERO)
acc = ct.mma(a, b, acc) # tensor-core multiply-accumulate
ct.store(C, index=(bm, bn), tile=ct.astype(acc, C.dtype))
# fp16 inputs, 128x128 output tiles, K step 64
grid = (ct.cdiv(M, 128), ct.cdiv(N, 128), 1)
ct.launch(cp.cuda.get_current_stream(), grid, matmul, (A, B, C, 128, 128, 64))
Other tile operations: ct.sum, ct.max, ct.exp, ct.where, ct.matmul, ct.zeros,
ct.astype, ct.num_blocks(axis). Tile dimensions must be powers of two
(Dimension #0 of shape (1000,) is not a power of two).
4. Ahead-of-time export (no GPU needed)
import cuda.tile as ct
from cuda.tile.compilation import (ArrayConstraint, ConstantConstraint, KernelSignature,
CallingConvention, export_kernel)
def f32_1d():
return ArrayConstraint(ct.float32, 1, index_dtype=ct.int32, stride_lower_bound_incl=0,
alias_groups=(), may_alias_internally=False)
sig = KernelSignature([f32_1d(), f32_1d(), f32_1d(), ConstantConstraint(1024)],
CallingConvention.cutile_python_v1())
# Portable Tile IR bytecode (no gpu_code) needs bytecode version 13.3+, i.e. a 13.3+ toolkit;
# with 13.1/13.2, export a cubin for a specific target instead
export_kernel(vector_add, [sig], "vector_add.tilebc", output_format="tileir_bytecode")
export_kernel(vector_add, [sig], "vector_add_sm100.cubin", gpu_code="sm_100", output_format="cubin")
gpu_code accepts sm_80 and newer; exporting for sm_75 fails in tileiras. The bytecode
can be lowered later with tileiras --gpu-name=sm_120 file.tilebc -o file.cubin and
inspected with tileirdisasm file.tilebc.
5. CUDA Tile C++ (CUDA 13.3+)
// vadd_tile.cu — one block processes one 1024-element tile
#include <cstdio>
#include "cuda_tile.h"
__tile_global__ void vectorAdd(float* __restrict__ a, float* __restrict__ b,
float* __restrict__ out, std::size_t n)
{
namespace ct = cuda::tiles;
using namespace ct::literals;
a = assume_aligned(a, 16_ic); // lets the compiler use vector loads
b = assume_aligned(b, 16_ic);
out = assume_aligned(out, 16_ic);
auto idx = ct::bid().x; // tile index = block index
// View each array as a sequence of 1024-element tiles
auto view_a = ct::partition_view{ct::tensor_span{a, ct::extents{n}}, ct::shape{1024_ic}};
auto view_b = ct::partition_view{ct::tensor_span{b, ct::extents{n}}, ct::shape{1024_ic}};
auto view_out = ct::partition_view{ct::tensor_span{out, ct::extents{n}}, ct::shape{1024_ic}};
auto tile_a = view_a.load_masked(idx); // masked: the last tile may be partial
auto tile_b = view_b.load_masked(idx);
view_out.store_masked(tile_a + tile_b, idx);
}
int main()
{
const std::size_t n = 1 << 20;
float *a, *b, *out;
if (cudaMallocManaged(&a, n * sizeof(float)) != cudaSuccess ||
cudaMallocManaged(&b, n * sizeof(float)) != cudaSuccess ||
cudaMallocManaged(&out, n * sizeof(float)) != cudaSuccess) {
fprintf(stderr, "no usable GPU\n");
return 1;
}
for (std::size_t i = 0; i < n; i++) { a[i] = 1.0f; b[i] = 2.0f; }
unsigned tiles = (n + 1023) / 1024;
vectorAdd<<<tiles, 1>>>(a, b, out, n); // tile kernels launch with a block size of 1
cudaError_t err = cudaDeviceSynchronize();
if (err != cudaSuccess) {
fprintf(stderr, "kernel failed: %s\n", cudaGetErrorString(err));
return 1;
}
printf("out[0]=%f\n", out[0]);
return 0;
}
# Set -arch to your GPU: sm_80 (Ampere), sm_90 (Hopper), sm_100 / sm_120 (Blackwell)
nvcc -std=c++20 --enable-tile -arch=sm_100 -o vadd vadd_tile.cu
# Without --enable-tile: error: calling a __tile__ function ... from a __host__ function is not allowed
__tile_global__marks a tile kernel;_icliterals are compile-time integers (tile shapes).partition_viewsplits atensor_spaninto tiles;load/storetake tile indices.- 13.5 adds
ct::scaled_mma()(block-scaled MMA) andreturninside loops in tile code.
6. Debugging and profiling
# Print the Tile IR that cuTile generates (needs CUDA 13.4+)
CUDA_TILE_LOGS=TILEIR python app.py
# Write Tile IR (MLIR text) for each compiled kernel to a directory
CUDA_TILE_DUMP_TILEIR=/tmp/tileir python app.py
# Nsight Compute: the Details page has a "Tile Statistics" section for tile kernels
ncu --set full -o tile_report python app.py
| Symptom | Cause | Fix |
|---|---|---|
TileCompilerExecutionError when exporting or launching | tileiras rejected the target or the kernel | Use sm_80+; read the stderr from tileiras in the exception |
tileiras not found | No CUDA 13.1+ toolkit on PATH | pip install "cuda-tile[tileiras]" or install the toolkit |
C++ errors about __tile__ from __host__ | Missing --enable-tile | nvcc -std=c++20 --enable-tile |
| Many recompiles | A ct.Constant parameter takes many values | Make it a runtime argument, or bucket the values |
| Wrong results at array edges | Unmasked access on a partial tile | C++: load_masked/store_masked; Python: pad or use a size that is a multiple of the tile |
Related skills
- Use
skills/gpu/cudafor SIMT CUDA, nvcc targets and CUDA 13 architecture support - Use
skills/gpu/triton-langfor Triton, the main alternative tile DSL - Use
skills/gpu/cuda-profilingfor Nsight Compute and Nsight Systems - Use
skills/gpu/gpu-memory-modelfor the memory hierarchy tiles map onto
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |