Tencent/hpc-ops

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1131 stars · 140 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

HPC-Ops is a production-grade C++/CUDA operator library for high-performance LLM inference, developed by Tencent's Hunyuan AI Infra team. It provides optimized kernels for attention, MoE, GEMM, sampling, normalization, and fused communication-compute on modern NVIDIA GPUs, with a Python API for integration into frameworks like vLLM and SGLang.

Use cases

• speed up LLM inference serving latency and throughput
• integrate optimized attention and MoE kernels into vLLM or SGLang
• run FP8 and BF16 mixed-precision inference on NVIDIA H20 GPUs
• benchmark custom CUDA kernels against FlashInfer, cuBLAS, and TensorRT-LLM
• learn to write production CUDA kernels with CuTe, CUTLASS, TMA, and PDL
• balance dynamic decode workloads with variable KV-cache lengths

When to choose

• you serve LLMs on NVIDIA Hopper-class GPUs (especially H20) and need SOTA kernels
• you want drop-in operators for popular inference frameworks like vLLM or SGLang
• you need FP8/quantized or mixed-precision kernels for accuracy-sensitive inference
• you want compact, production-quality CUDA examples for learning GPU kernel engineering

When to avoid

• you target non-NVIDIA GPUs or older CUDA versions below 12.8
• you need a full end-to-end inference engine rather than individual operators
• your workloads are small-scale or CPU-bound where kernel optimization matters little
• you require a permissive license for redistribution and cannot accept the custom license

Facets

library · maturity active

llm-inference gpu-computing machine-learning benchmarking deep-learning large-language-models gpu-computing performance python cpp cuda-kernels attention moe gemm fp8 inference-optimization nvidia-h20 cutlass gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="Tencent/hpc-ops")