Tencent/TencentPretrain

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1091 stars · 148 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

TencentPretrain is a PyTorch-based toolkit for pre-training and fine-tuning transformer models across modalities such as text, vision, and audio. It includes a model zoo of pre-trained models (BERT, GPT-2, T5, CLIP, etc.) and supports distributed and DeepSpeed-based large model training.

Use cases

• pre-train a BERT or GPT-2 model from scratch in PyTorch
• fine-tune a pre-trained transformer for text classification
• download and use Chinese pre-trained language models
• train multimodal models on text and image data
• run distributed pre-training with DeepSpeed
• convert TensorFlow pre-trained models to PyTorch
• fine-tune models for NER and reading comprehension tasks

When to choose

• you need a modular PyTorch framework for pre-training or fine-tuning transformers
• you want ready-to-use pre-trained models, especially for Chinese NLP
• you need multimodal (text, vision, audio) pre-training support
• you want to train large models with DeepSpeed integration

When to avoid

• you only need inference with an off-the-shelf LLM API
• you prefer Hugging Face Transformers' broader ecosystem and community
• you need a framework with frequent updates and active maintenance
• you require non-PyTorch training stacks

Facets

library · maturity maintenance

machine-learning deep-learning llm-training nlp deep-learning machine-learning large-language-models python cross-platform pre-training fine-tuning model-zoo pytorch multimodal bert gpt t5 chinese-nlp deepspeed natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="Tencent/TencentPretrain")