facebookresearch/xformers

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

10565 stars · 795 forks observed · 2026-10-14

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

xFormers is a PyTorch-based library of hackable, optimized Transformer building blocks with custom CUDA kernels for fast, memory-efficient attention and related operators. It lets researchers compose domain-agnostic components for vision, NLP, and other modalities without boilerplate.

Use cases

• speed up transformer attention with memory-efficient kernels
• build custom transformer architectures from composable blocks
• reduce GPU memory usage when training large models
• try bleeding-edge attention operators not yet in PyTorch
• accelerate stable diffusion and vision transformer inference

When to choose

• you need faster or more memory-efficient attention than stock PyTorch provides
• you want customizable transformer components for research across vision and NLP
• you are on Linux or Windows with a CUDA GPU and PyTorch 2.x

When to avoid

• you only need a high-level pretrained model API rather than low-level building blocks
• you need CPU-only or non-CUDA/non-ROCm hardware support
• you want a fully stable API guaranteed not to change between releases

Facets

library · maturity active

machine-learning deep-learning llm-training llm-inference gpu-computing deep-learning machine-learning large-language-models gpu-computing computer-vision python windows transformers attention-kernels pytorch memory-efficiency cuda-kernels research natural-language-processing linux gpu cuda

Member repositories

For agents

markdown · JSON · MCP: product_card(name="facebookresearch/xformers")