# cfregly/ai-performance-engineering

Code, labs, and resources for O'Reilly AI Systems Performance Engineering: GPU optimization, distributed training, inference scaling, and full-stack tuning.

Repository: https://github.com/cfregly/ai-performance-engineering
Canonical: https://ross.abutalabs.com/products/ai-performance-engineering
Language: Python
License: Apache-2.0
License Family: permissive
Last push: 2026-08-21T21:24:33+00:00

## Health v2 (maintenance only)
Score: 63/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 98, release rhythm 35, longevity 35
- inputs: {"age_days": 499, "days_push": 12, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1863, forks 260 (observed 2026-08-28T04:05:45.502950+00:00)

## What it is
Code, labs, and resources accompanying the O'Reilly book 'AI Systems Performance Engineering', covering GPU optimization, distributed training, and inference scaling. It includes thousands of lines of PyTorch and CUDA C++ examples plus profiling case studies for modern NVIDIA GPUs.

## Use cases
- learn how to profile and optimize GPU workloads with Nsight and PyTorch profilers
- understand distributed training parallelism strategies like FSDP, TP, PP, and MoE
- optimize high-throughput LLM inference with vLLM, SGLang, and TensorRT-LLM
- write custom GPU kernels with Triton and the PyTorch compiler stack
- reduce cost per token when serving large models
- hands-on labs for AI systems performance tuning

## When to choose
- you are an ML or systems engineer building or operating training/inference at scale
- you want empirical, profiling-driven GPU optimization techniques
- you need to learn disaggregated prefill/decode serving and KV-cache management
- you want a structured book plus runnable code examples

## When to avoid
- you need a production-ready tool or framework rather than educational material
- you work exclusively on non-NVIDIA hardware
- you are a beginner looking for basic ML tutorials rather than performance engineering

## Facets
- artifact type: learning-resource
- maturity: active
- function: benchmarking, gpu-computing, llm-inference, llm-training, machine-learning, deep-learning
- domain: gpu-computing, machine-learning, deep-learning, large-language-models, performance, developer-tools, tutorials
- platform: python, cross-platform
- tags: gpu-optimization, distributed-training, inference-serving, profiling, cuda, pytorch, triton, vllm, tensorrt-llm, oreilly-book, labs, gpu, linux

## Member repositories
- cfregly/ai-performance-engineering (main) score 63

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:45.502950+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:15:38.536484+00:00, confidence not recorded.
  - readme: https://github.com/cfregly/ai-performance-engineering (fetched 2026-08-28T04:05:45.502950+00:00, sha b11685b5a2a5)
- Data as of 2026-08-30T08:39:29.467469+00:00.