huawei-noah/Pretrained-Language-Model

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

3165 stars · 637 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A collection of pretrained language models and optimization techniques from Huawei Noah's Ark Lab, including PanGu-α (200B-parameter Chinese autoregressive model), NEZHA, TinyBERT, and various BERT compression methods. It covers knowledge distillation, weight quantization/binarization, and dynamic architectures across TensorFlow, PyTorch, and MindSpore.

Use cases

• use a pretrained Chinese language model
• compress BERT with knowledge distillation
• quantize or binarize transformer weights for faster inference
• run a large-scale autoregressive Chinese model like PanGu-α
• build a byte-level tokenizer vocabulary
• fine-tune NEZHA on Chinese NLP tasks
• deploy tiny BERT models under strict latency budgets

When to choose

• you need Chinese-language pretrained models or BERT variants
• you want research-grade model compression techniques like TinyBERT, TernaryBERT, or BinaryBERT
• you work with MindSpore or Ascend hardware
• you need small, fast BERT models for edge or latency-constrained inference

When to avoid

• you need a maintained library with active support - releases are infrequent and there is no license file
• you want general-purpose English-centric LLM tooling like Llama-style models
• you need production training infrastructure rather than research code
• you require a permissively licensed dependency - the missing license complicates commercial use

Facets

library · maturity maintenance

machine-learning llm-training llm-inference nlp large-language-models machine-learning deep-learning python knowledge-distillation quantization model-compression chinese-nlp bert pangu mindspore tinybert pretrained-models natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="huawei-noah/Pretrained-Language-Model")