# microsoft/DeBERTa

The implementation of DeBERTa

Repository: https://github.com/microsoft/DeBERTa
Canonical: https://ross.abutalabs.com/products/deberta
Language: Python
License: MIT
License Family: permissive
Topics: bert, deeplearning, representation-learning, roberta, language-model, natural-language-understanding, self-attention, transformer-encoder
Last push: 2023-09-29T11:14:26+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 2277, "days_push": 1069, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2239, forks 240 (observed 2026-08-28T04:06:29.644009+00:00)

## What it is
Official implementation of DeBERTa and DeBERTa V3, transformer-based language models with disentangled attention and decoding-enhanced BERT architecture. Includes pre-training and fine-tuning code plus pretrained models available on the Hugging Face hub.

## Use cases
- fine-tune a pretrained language model for NLU tasks
- pre-train a transformer language model from scratch
- improve question answering accuracy over BERT or RoBERTa
- run natural language understanding benchmarks like SuperGLUE
- use a small efficient encoder model for text classification
- download DeBERTa v3 weights for use with Hugging Face transformers

## When to choose
- you need a strong pretrained encoder for NLU tasks like MNLI, SQuAD, or SuperGLUE
- you want to reproduce DeBERTa research or pre-train with ELECTRA-style objectives
- you need high accuracy with a small parameter budget like DeBERTa-V3-XSmall

## When to avoid
- you need text generation or decoder-only LLM capabilities
- you want a maintained general NLP toolkit rather than a specific model implementation
- you need a framework-agnostic solution outside PyTorch

## Facets
- artifact type: library
- maturity: maintenance
- function: machine-learning, deep-learning, nlp, transformers
- domain: deep-learning, machine-learning
- platform: python
- tags: language-model, pretrained-models, transformer-encoder, disentangled-attention, pytorch, bert, huggingface, natural-language-processing

## Member repositories
- microsoft/DeBERTa (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:29.644009+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:44:10.080676+00:00, confidence not recorded.
  - readme: https://github.com/microsoft/DeBERTa (fetched 2026-08-28T04:06:29.644009+00:00, sha e1a41e9e8284)
  - registry_pypi: https://pypi.org/pypi/deberta/json (fetched 2026-08-29T10:24:35.514774+00:00, sha 593b35af9281)
- Data as of 2026-08-30T08:39:29.467469+00:00.