# allenai/scibert

A BERT model for scientific text.

Repository: https://github.com/allenai/scibert
Canonical: https://ross.abutalabs.com/products/scibert
Homepage: https://arxiv.org/abs/1903.10676
Language: Python
License: Apache-2.0
License Family: permissive
Topics: bert, nlp, scientific-papers
Last push: 2022-02-22T19:57:07+00:00

## Health v2 (maintenance only)
Score: 32/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 35, longevity 100
- inputs: {"age_days": 2772, "days_push": 1653, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1712, forks 232 (observed 2026-08-28T04:05:25.816808+00:00)

## What it is
SciBERT is a BERT-based pretrained language model trained on a large multi-domain corpus of scientific publications, released by the Allen Institute for AI. It provides pretrained weights and code that improve performance on downstream scientific NLP tasks such as sequence tagging, sentence classification, and dependency parsing.

## Use cases
- extract named entities like chemicals and diseases from biomedical papers
- classify citation intent in scientific literature
- classify sentences in scientific abstracts by their role
- extract relations between scientific entities
- fine-tune a language model on scientific text with limited labeled data
- parse dependencies in scientific sentences

## When to choose
- you need strong baselines or embeddings for scientific or biomedical NLP tasks
- you have limited labeled scientific data and want transfer learning from a domain-specific pretrained model
- you want a well-cited, Apache-2.0 licensed BERT variant for academic text

## When to avoid
- you need a general-purpose language model for non-scientific text
- you need the latest large language models rather than BERT-era encoders
- you require actively maintained tooling, as the repo has seen few recent updates

## Facets
- artifact type: library
- maturity: maintenance
- function: nlp, machine-learning, parser
- domain: machine-learning, artificial-intelligence
- platform: python
- tags: bert, scientific-text, pretrained-model, transformers, named-entity-recognition, sentence-classification, natural-language-processing

## Member repositories
- allenai/scibert (main) score 32

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:25.816808+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:34:51.832396+00:00, confidence not recorded.
  - readme: https://github.com/allenai/scibert (fetched 2026-08-28T04:05:25.816808+00:00, sha af33585bef6b)
  - homepage: https://arxiv.org/abs/1903.10676 (fetched 2026-08-29T11:10:53.595738+00:00, sha c1cce416fb17)
  - site_page: https://info.arxiv.org/about/donate.html (fetched 2026-08-29T11:10:53.605244+00:00, sha cca9c3a11c56)
  - site_page: https://info.arxiv.org/about/ourmembers.html (fetched 2026-08-29T11:10:53.608963+00:00, sha 47cbc55ff1de)
  - site_page: https://info.arxiv.org/about (fetched 2026-08-29T11:10:53.610900+00:00, sha a1f16f915a9a)
  - site_page: https://info.arxiv.org/labs/index.html (fetched 2026-08-29T11:10:53.607157+00:00, sha b14a8d05a0ec)
- Data as of 2026-08-30T08:39:29.467469+00:00.