MoonshotAI/FlashKDA

Health v2 · maintenance only

Flags: no_releases young

Adoption not part of the score

1229 stars · 118 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

FlashKDA is a set of high-performance CUDA kernels (built on CUTLASS) implementing Kimi Delta Attention, a linear attention mechanism, for NVIDIA SM90+ GPUs. It integrates as an auto-dispatched backend for the flash-linear-attention library's chunk_kda operation.

Use cases

• run kimi delta attention fast on gpu
• speed up linear attention kernels for llm training
• flash attention style kernel for delta rule attention
• backend for flash-linear-attention chunk_kda
• benchmark attention kernel performance on H100
• train models with gated linear attention efficiently

When to choose

• you need fast KDA forward passes on NVIDIA Hopper (SM90+) GPUs
• you already use flash-linear-attention and want a faster backend
• you train or serve models using Kimi Delta Attention with PyTorch 2.4+

When to avoid

• you have pre-Hopper GPUs (SM80 or earlier)
• you need a pure Triton or CPU implementation
• you don't use PyTorch or CUDA 12.9+

Facets

library · maturity active

llm-inference gpu-computing deep-learning machine-learning deep-learning large-language-models gpu-computing python cuda-kernels cutlass linear-attention flash-attention pytorch kimi-delta-attention hopper linux gpu cuda

Member repositories

For agents

markdown · JSON · MCP: product_card(name="MoonshotAI/FlashKDA")