zjhellofss/KuiperInfer resource

Health v2 · maintenance only

Adoption not part of the score

3496 stars · 372 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

KuiperInfer is an open-source course that guides learners through building a high-performance deep learning inference engine from scratch in modern C++, supporting models like ResNet, YOLOv5, UNet, and Llama-family LLMs. It includes a companion paid course on building an LLM inference framework with hand-written CUDA operators, CUDA acceleration, and Int8 quantization.

Use cases

• learn how deep learning inference engines work internally
• implement convolution and pooling operators from scratch in C++
• build a computational graph executor for neural networks
• run inference for resnet, yolov5, and unet models with a custom framework
• write CUDA kernels for llama and qwen large language model inference
• prepare for backend/ML systems job interviews with a substantial C++ project
• learn modern C++ project structure with CMake, unit tests, and benchmarks

When to choose

• you want to deeply understand inference engine internals rather than just use one
• you need a portfolio project for campus recruiting or interviews in C++/ML systems
• you want guided, step-by-step video lessons paired with a real codebase
• you want to learn CUDA operator development for LLM inference

When to avoid

• you need a production-ready inference engine for deployment (use ONNX Runtime, TensorRT, or llama.cpp instead)
• you want a framework to train models rather than run inference
• you prefer not to follow a course format and just need drop-in inference support
• you need broad model format support beyond PNNX/PyTorch-exported models

Facets

learning-resource · maturity active

llm-inference machine-learning deep-learning gpu-computing benchmarking testing deep-learning large-language-models machine-learning computer-vision education developer-tools cpp windows cross-platform inference-engine course tutorial cuda-kernels llama yolov5 unet resnet int8-quantization computational-graph modern-cpp cmake job-interview-prep linux macos gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="zjhellofss/KuiperInfer")