# PaddlePaddle/PaddleFormers

PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle.

Repository: https://github.com/PaddlePaddle/PaddleFormers
Canonical: https://ross.abutalabs.com/products/paddleformers
Language: Python
License: Apache-2.0
License Family: permissive
Topics: model
Last push: 2026-08-26T11:49:03+00:00

## Health v2 (maintenance only)
Score: 92/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 99, release rhythm 77, longevity 100
- inputs: {"age_days": 2812, "days_push": 7, "days_rel": 75, "gap_med": 68.5, "n_releases_24m": 7}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 12986, forks 2195 (observed 2026-08-28T04:11:00.324180+00:00)

## What it is
PaddleFormers is a Transformers-style library built on PaddlePaddle providing a model zoo of 100+ large language models and vision-language models with pretraining and post-training support. It offers high-performance distributed training (tensor, pipeline, expert parallelism, FP8, FlashMLA) that outperforms Megatron-LM on models like DeepSeek-V4 and GLM-4.5-Air, with Hugging Face-compatible Safetensors output.

## Use cases
- pretrain large language models like DeepSeek-V4 on distributed GPU clusters
- fine-tune LLMs with SFT, LoRA, or DPO
- train vision-language models like Qwen3-VL and ERNIE-4.5-VL
- train MoE models with expert parallelism and FP8 precision
- train long-context models beyond 128K tokens
- export trained models in Hugging Face Safetensors format for vLLM or SGLang
- train function-call and thinking capabilities into models

## When to choose
- you are in the PaddlePaddle ecosystem and need a Hugging Face Transformers equivalent
- you need maximum training throughput for DeepSeek, GLM, or Qwen models on GPUs or domestic accelerators
- you need distributed training strategies like tensor, pipeline, context, and expert parallelism
- you want pretraining and post-training (CPT/SFT/DPO) in one library

## When to avoid
- your stack is PyTorch-based; use Hugging Face Transformers or Megatron-LM instead
- you only need inference rather than training
- you need broad community plugin support outside the Paddle ecosystem

## Facets
- artifact type: library
- maturity: active
- function: llm-training, machine-learning, deep-learning, nlp
- domain: large-language-models, machine-learning, deep-learning
- platform: windows, python
- tags: paddlepaddle, transformers, distributed-training, moe, vlm, sft, lora, dpo, fp8, huggingface-compatible, natural-language-processing, linux, gpu

## Member repositories
- PaddlePaddle/PaddleFormers (main) score 92

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:11:00.324180+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:13:37.538933+00:00, confidence not recorded.
  - readme: https://github.com/PaddlePaddle/PaddleFormers (fetched 2026-08-28T04:11:00.324180+00:00, sha 486ed22cf16f)
  - registry_pypi: https://pypi.org/pypi/paddleformers/json (fetched 2026-08-29T08:09:27.744464+00:00, sha 539f8056b621)
- Data as of 2026-08-30T08:39:29.467469+00:00.