ai-dynamo/nixl

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

1227 stars · 426 forks observed · 2026-09-03

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

NVIDIA Inference Xfer Library (NIXL) is a C++/Python library that accelerates point-to-point communication in AI inference frameworks like NVIDIA Dynamo. It abstracts over various memory types (CPU, GPU) and storage backends (file, block, object store) through a modular plug-in architecture.

Use cases

• accelerate KV cache transfer between inference engine instances
• abstract point-to-point data transfer across CPU and GPU memory
• plug different transport backends like UCX into an inference framework
• benchmark memory transfer performance for LLM serving
• transfer tensors between distributed inference workers
• read and write to object or block storage from GPU code

When to choose

• building or extending AI inference frameworks on Linux with NVIDIA GPUs
• you need high-performance point-to-point transfers between heterogeneous memory and storage
• you want a pluggable abstraction over transports like UCX
• you're integrating with NVIDIA Dynamo or similar disaggregated serving stacks

When to avoid

• you need macOS or Windows support - NIXL is Linux-only
• you need general-purpose networking rather than inference-focused data transfer
• your project doesn't involve GPU or accelerator memory movement
• you need a stable API on a non-NVIDIA stack

Facets

library · maturity active

networking plugin-system benchmarking caching sdk machine-learning large-language-models microservices gpu-computing developer-tools cpp python data-transfer inference-infrastructure nvidia ucx kv-cache-transfer point-to-point-communication memory-abstraction cuda linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ai-dynamo/nixl")