EleutherAI/gpt-neo

Health v2 · maintenance only

Flags: archived

Adoption not part of the score

8270 stars · 954 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

GPT-Neo is EleutherAI's implementation of model- and data-parallel GPT-3-style transformer language models built on mesh-tensorflow, with released pretrained checkpoints (125M to 2.7B) trained on The Pile. The codebase is archived and no longer maintained, having been superseded by GPT-NeoX.

Use cases

• train a GPT-3-style language model on TPUs
• run inference with pretrained open-source GPT models
• experiment with linear attention and mixture of experts in transformers
• study an open reproduction of GPT-3 architecture
• download open pretrained language model weights
• compare GPT-Neo against GPT-3 on benchmarks

When to choose

• you need the original GPT-Neo checkpoints or mesh-tensorflow-based TPU training code
• you are reproducing historical research results on GPT-3-like models
• you want to study local attention, linear attention, or axial positional embeddings implementations

When to avoid

• you want actively maintained LLM training code - use GPT-NeoX instead
• you just want to run pretrained GPT-Neo models - use the HuggingFace Transformers integration
• you need GPU-first or modern distributed training support

Facets

library · maturity abandoned

llm-training llm-inference transformers machine-learning deep-learning large-language-models deep-learning machine-learning artificial-intelligence python cloud gpt-3 gpt-2 mesh-tensorflow model-parallelism tpu language-model archived eleutherai pretrained-models the-pile natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="EleutherAI/gpt-neo")