MoonshotAI/Kimi-Linear

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1588 stars · 87 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Kimi Linear is a hybrid linear attention architecture (Kimi Delta Attention, based on Gated DeltaNet) released by Moonshot AI with 48B-parameter MoE model checkpoints (Base and Instruct) supporting 1M-token contexts. The repo provides the model implementation, KDA kernel integration with flash-linear-attention, and usage instructions for inference and fine-tuning.

Use cases

• run a long-context LLM with 1M token window
• use linear attention models instead of full attention transformers
• fine-tune a hybrid linear attention MoE model
• reduce KV cache memory for long-context inference
• benchmark linear attention vs full attention decoding throughput
• experiment with Kimi Delta Attention kernels

When to choose

• you need very long context (up to 1M tokens) with high decoding throughput
• you want to research or build on linear attention / finite-state RNN memory architectures
• you want lower KV cache memory usage in LLM serving

When to avoid

• you need a small model that fits on consumer hardware (48B total params)
• you need broad ecosystem/tooling support of mainstream transformer models
• you require short-context tasks where full attention models are equally good and better supported

Facets

library · maturity active

machine-learning deep-learning llm-inference llm-training large-language-models deep-learning artificial-intelligence python linear-attention kimi-delta-attention kda long-context moe flash-linear-attention model-checkpoints transformers gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="MoonshotAI/Kimi-Linear")