# facebookresearch/xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

Repository: https://github.com/facebookresearch/xformers
Canonical: https://ross.abutalabs.com/products/xformers
Homepage: https://facebookresearch.github.io/xformers/
Language: Python
License: NOASSERTION
License Family: other
Last push: 2026-08-07T12:33:49+00:00

## Health v2 (maintenance only)
Score: 88/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 96, release rhythm 71, longevity 100
- inputs: {"age_days": 1785, "days_push": 26, "days_rel": 194, "gap_med": 24.0, "n_releases_24m": 19}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 10542, forks 784 (observed 2026-08-28T04:10:42.297745+00:00)

## What it is
xFormers is a PyTorch-based library of hackable, optimized Transformer building blocks with custom CUDA kernels for fast, memory-efficient attention and related operators. It lets researchers compose domain-agnostic components for vision, NLP, and other modalities without boilerplate.

## Use cases
- speed up transformer attention with memory-efficient kernels
- build custom transformer architectures from composable blocks
- reduce GPU memory usage when training large models
- try bleeding-edge attention operators not yet in PyTorch
- accelerate stable diffusion and vision transformer inference

## When to choose
- you need faster or more memory-efficient attention than stock PyTorch provides
- you want customizable transformer components for research across vision and NLP
- you are on Linux or Windows with a CUDA GPU and PyTorch 2.x

## When to avoid
- you only need a high-level pretrained model API rather than low-level building blocks
- you need CPU-only or non-CUDA/non-ROCm hardware support
- you want a fully stable API guaranteed not to change between releases

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, deep-learning, llm-training, llm-inference, gpu-computing
- domain: deep-learning, machine-learning, large-language-models, gpu-computing, computer-vision
- platform: python, windows
- tags: transformers, attention-kernels, pytorch, memory-efficiency, cuda-kernels, research, natural-language-processing, linux, gpu, cuda

## Member repositories
- facebookresearch/xformers (main) score 88

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:10:42.297745+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:18:33.475573+00:00, confidence not recorded.
  - readme: https://github.com/facebookresearch/xformers (fetched 2026-08-28T04:10:42.297745+00:00, sha d7d3a2d0e099)
  - homepage: https://facebookresearch.github.io/xformers/ (fetched 2026-08-29T08:17:50.862071+00:00, sha 0fced8c6b6da)
- Data as of 2026-08-30T08:39:29.467469+00:00.