PaddlePaddle/PaddleFormers

Health v2 · maintenance only

Adoption not part of the score

12986 stars · 2195 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

PaddleFormers is a Transformers-style library built on PaddlePaddle providing a model zoo of 100+ large language models and vision-language models with pretraining and post-training support. It offers high-performance distributed training (tensor, pipeline, expert parallelism, FP8, FlashMLA) that outperforms Megatron-LM on models like DeepSeek-V4 and GLM-4.5-Air, with Hugging Face-compatible Safetensors output.

Use cases

• pretrain large language models like DeepSeek-V4 on distributed GPU clusters
• fine-tune LLMs with SFT, LoRA, or DPO
• train vision-language models like Qwen3-VL and ERNIE-4.5-VL
• train MoE models with expert parallelism and FP8 precision
• train long-context models beyond 128K tokens
• export trained models in Hugging Face Safetensors format for vLLM or SGLang
• train function-call and thinking capabilities into models

When to choose

• you are in the PaddlePaddle ecosystem and need a Hugging Face Transformers equivalent
• you need maximum training throughput for DeepSeek, GLM, or Qwen models on GPUs or domestic accelerators
• you need distributed training strategies like tensor, pipeline, context, and expert parallelism
• you want pretraining and post-training (CPT/SFT/DPO) in one library

When to avoid

• your stack is PyTorch-based; use Hugging Face Transformers or Megatron-LM instead
• you only need inference rather than training
• you need broad community plugin support outside the Paddle ecosystem

Facets

library · maturity active

llm-training machine-learning deep-learning nlp large-language-models machine-learning deep-learning windows python paddlepaddle transformers distributed-training moe vlm sft lora dpo fp8 huggingface-compatible natural-language-processing linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="PaddlePaddle/PaddleFormers")