bytedance/lightseq

Health v2 · maintenance only

Flags: archived no_license

Adoption not part of the score

3295 stars · 334 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

LightSeq is a high-performance CUDA-based library for training and inference of sequence models like Transformer, BERT, GPT, and BART, with fused custom kernels and fp16/int8 mixed-precision support. It also provides a serving backend built on TensorRT Inference Server for easy deployment.

Use cases

• accelerate transformer model training on gpu
• run fast transformer inference with beam search
• fine-tune bert with fp16 mixed precision
• deploy gpt or bart models with an inference server
• speed up machine translation training
• int8 quantized inference for transformer models

When to choose

• you need maximum GPU throughput for Transformer-family training or inference
• you want a deployable serving backend for sequence generation models
• you work with fairseq or Hugging Face models and want drop-in acceleration

When to avoid

• you need a general-purpose framework for non-Transformer architectures
• you require active maintenance and recent updates, as the last release was in 2023
• you need CPU-only inference

Facets

library · maturity maintenance

llm-inference llm-training machine-learning deep-learning nlp gpu-computing deep-learning machine-learning gpu-computing python cpp transformer cuda-kernels beam-search bert gpt mixed-precision inference-server machine-translation natural-language-processing linux gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="bytedance/lightseq")