CLUEbenchmark/CLUECorpus2020 resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1018 stars · 83 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

CLUECorpus2020 is a 100GB cleaned Chinese text corpus (35 billion characters) derived from Common Crawl for pre-training language models, plus a smaller 14GB CLUECorpusSmall and a compact 8K-token Chinese vocabulary. It is distributed by the CLUE benchmark organization with an accompanying technical report and pre-trained models.

Use cases

• pretrain a Chinese language model like BERT or RoBERTa
• find a large Chinese text corpus for language modeling
• train a Chinese text generation model
• reduce tokenizer vocabulary size for Chinese NLP
• get cleaned Chinese web text for self-supervised learning
• fine-tune Chinese NLP models on a quality corpus

When to choose

• you need large-scale Chinese text for pretraining or language generation
• you want a smaller Chinese vocabulary to cut compute and memory costs
• you need pre-formatted training data with one sentence per line

When to avoid

• you need multilingual corpora beyond Chinese
• you cannot or will not apply via email and agree to the no-redistribution terms
• you need an actively updated corpus rather than a fixed 2020 snapshot

Facets

dataset · maturity stable

machine-learning nlp data-generation large-language-models machine-learning python cross-platform chinese-corpus pretraining language-modeling common-crawl vocabulary bert natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="CLUEbenchmark/CLUECorpus2020")