microsoft/DeBERTa

Health v2 · maintenance only

Adoption not part of the score

2239 stars · 240 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Official implementation of DeBERTa and DeBERTa V3, transformer-based language models with disentangled attention and decoding-enhanced BERT architecture. Includes pre-training and fine-tuning code plus pretrained models available on the Hugging Face hub.

Use cases

• fine-tune a pretrained language model for NLU tasks
• pre-train a transformer language model from scratch
• improve question answering accuracy over BERT or RoBERTa
• run natural language understanding benchmarks like SuperGLUE
• use a small efficient encoder model for text classification
• download DeBERTa v3 weights for use with Hugging Face transformers

When to choose

• you need a strong pretrained encoder for NLU tasks like MNLI, SQuAD, or SuperGLUE
• you want to reproduce DeBERTa research or pre-train with ELECTRA-style objectives
• you need high accuracy with a small parameter budget like DeBERTa-V3-XSmall

When to avoid

• you need text generation or decoder-only LLM capabilities
• you want a maintained general NLP toolkit rather than a specific model implementation
• you need a framework-agnostic solution outside PyTorch

Facets

library · maturity maintenance

machine-learning deep-learning nlp transformers deep-learning machine-learning python language-model pretrained-models transformer-encoder disentangled-attention pytorch bert huggingface natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/DeBERTa")