facebookincubator/AITemplate

Health v2 · maintenance only

Adoption not part of the score

4724 stars · 388 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

AITemplate is a Python framework that compiles deep neural networks into high-performance CUDA (NVIDIA) or HIP (AMD) C++ code for fast fp16 inference. It generates self-contained binaries with advanced horizontal, vertical, and memory kernel fusions without depending on third-party libraries like cuDNN or TensorRT.

Use cases

• serve fp16 transformer models like BERT or Stable Diffusion at near-roofline GPU performance
• compile neural networks into portable self-contained inference binaries
• run the same model efficiently on both NVIDIA and AMD GPUs
• fuse GEMM, LayerNorm, and elementwise ops into single GPU kernels
• deploy inference without PyTorch or CUDA runtime library dependencies
• add custom fused operators via simple Python codegen extensions

When to choose

• you need maximum fp16 inference throughput on NVIDIA or AMD GPUs
• you want self-contained binaries free of cuDNN/cuBLAS/TensorRT dependencies
• your models (ResNet, BERT, ViT, Stable Diffusion) benefit from aggressive kernel fusion
• you need one framework targeting both CUDA and HIP

When to avoid

• you need training or gradient support - it is inference-only
• you need fp32/int8 quantization or CPU inference
• you want a mature, actively evolving ecosystem - consider alternatives like TensorRT or torch.compile
• your workflow depends on dynamic shapes beyond its supported range

Facets

framework · maturity maintenance

deep-learning llm-inference compiler gpu-computing deep-learning machine-learning gpu-computing performance python cuda hip amd-gpu nvidia-gpu fp16 tensorcore inference-serving code-generation kernel-fusion linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="facebookincubator/AITemplate")