explosion/spacy-transformers

Health v2 · maintenance only

Adoption not part of the score

1409 stars · 181 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A spaCy v3 extension package that provides pipeline components for using pretrained transformer models like BERT, RoBERTa, XLNet, and GPT-2 via Hugging Face's transformers library. It handles tokenization alignment, multi-task learning, and model packaging so transformer embeddings can power spaCy NLP components.

Use cases

• use BERT embeddings in a spaCy NER pipeline
• fine-tune pretrained transformers for named entity recognition
• share one transformer model across multiple spaCy pipeline components
• train a custom spaCy pipeline with transformer features
• align Hugging Face tokenizer output with spaCy tokenization
• package and serialize a transformer-based NLP model

When to choose

• you already use spaCy and want state-of-the-art transformer accuracy
• you need multi-task learning with a shared transformer backbone
• you want spaCy v3 config-driven training with pretrained models

When to avoid

• you don't use spaCy and just want raw Hugging Face transformers
• you need lightweight CPU-only inference with minimal dependencies
• you're on spaCy v2.x without upgrading

Facets

library · maturity stable

nlp machine-learning transformers llm-training machine-learning deep-learning python cross-platform spacy huggingface bert transfer-learning pytorch named-entity-recognition multi-task-learning natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="explosion/spacy-transformers")