# Tencent/TencentPretrain

Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo

Repository: https://github.com/Tencent/TencentPretrain
Canonical: https://ross.abutalabs.com/products/tencentpretrain
Homepage: https://github.com/Tencent/TencentPretrain/wiki
Language: Python
License: NOASSERTION
License Family: other
Topics: albert, bart, bert, chinese, classification, clue, elmo, fine-tuning, gpt, gpt-2, model-zoo, natural-language-processing, ner, pegasus, pre-training, pytorch, roberta, t5, unilm, xlm-roberta
Last push: 2024-08-04T11:53:43+00:00

## Health v2 (maintenance only)
Score: 32/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 0, release rhythm 35, longevity 100
- inputs: {"age_days": 1437, "days_push": 759, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1091, forks 148 (observed 2026-08-28T04:03:33.111683+00:00)

## What it is
TencentPretrain is a PyTorch-based toolkit for pre-training and fine-tuning transformer models across modalities such as text, vision, and audio. It includes a model zoo of pre-trained models (BERT, GPT-2, T5, CLIP, etc.) and supports distributed and DeepSpeed-based large model training.

## Use cases
- pre-train a BERT or GPT-2 model from scratch in PyTorch
- fine-tune a pre-trained transformer for text classification
- download and use Chinese pre-trained language models
- train multimodal models on text and image data
- run distributed pre-training with DeepSpeed
- convert TensorFlow pre-trained models to PyTorch
- fine-tune models for NER and reading comprehension tasks

## When to choose
- you need a modular PyTorch framework for pre-training or fine-tuning transformers
- you want ready-to-use pre-trained models, especially for Chinese NLP
- you need multimodal (text, vision, audio) pre-training support
- you want to train large models with DeepSpeed integration

## When to avoid
- you only need inference with an off-the-shelf LLM API
- you prefer Hugging Face Transformers' broader ecosystem and community
- you need a framework with frequent updates and active maintenance
- you require non-PyTorch training stacks

## Facets
- artifact type: library
- maturity: maintenance
- function: machine-learning, deep-learning, llm-training, nlp
- domain: deep-learning, machine-learning, large-language-models
- platform: python, cross-platform
- tags: pre-training, fine-tuning, model-zoo, pytorch, multimodal, bert, gpt, t5, chinese-nlp, deepspeed, natural-language-processing, gpu, linux

## Member repositories
- Tencent/TencentPretrain (main) score 32

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:03:33.111683+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T06:49:05.912881+00:00, confidence not recorded.
  - readme: https://github.com/Tencent/TencentPretrain (fetched 2026-08-28T04:03:33.111683+00:00, sha 88a58b01f61a)
  - homepage: https://github.com/Tencent/TencentPretrain/wiki (fetched 2026-08-29T12:51:16.806678+00:00, sha e45697da6000)
- Data as of 2026-08-30T08:39:29.467469+00:00.