siboehm/SGEMM_CUDA resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1294 stars · 212 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

An educational repository demonstrating step-by-step optimization of a CUDA SGEMM (matrix multiplication) kernel from a naive implementation to within ~94% of cuBLAS performance. It accompanies a detailed blog worklog explaining GPU performance concepts like memory coalescing, shared memory caching, and warptiling.

Use cases

• learn how to write fast CUDA matrix multiplication kernels
• understand GPU memory coalescing and shared memory optimization
• benchmark custom CUDA kernels against cuBLAS
• study GPU performance optimization for deep learning workloads
• learn CUDA kernel autotuning and warptiling techniques

When to choose

• you want to learn CUDA kernel optimization from first principles
• you need a worked example of optimizing matrix multiplication on NVIDIA GPUs
• you want to understand why cuBLAS is fast and what techniques it uses

When to avoid

• you need a production matrix multiplication library - use cuBLAS instead
• you need tensor core (TF32/BF16) support, which is not covered
• you work on non-NVIDIA GPUs, since the code is CUDA-specific

Facets

learning-resource · maturity stable

benchmarking math gpu-computing gpu-computing deep-learning performance tutorials cpp cuda sgemm matrix-multiplication kernel-optimization gpgpu educational algorithms linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="siboehm/SGEMM_CUDA")