minimaxir/aitextgen

Health v2 · maintenance only

Adoption not part of the score

1838 stars · 214 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

aitextgen is a Python library for training and generating text with GPT-2 and GPT Neo models, built on PyTorch, Hugging Face Transformers, and PyTorch Lightning. It supports finetuning pretrained models or training from scratch, with efficient dataset encoding, merging, and multi-GPU training.

Use cases

• finetune gpt-2 on my own text dataset
• generate text with a custom-trained language model
• train a small gpt model from scratch
• merge multiple text datasets for language model training
• run gpt-2 text generation on a gpu or colab notebook

When to choose

• you want a simple, high-level API for GPT-2/GPT Neo finetuning and text generation
• you need compatibility with Hugging Face Transformers for downstream NLP tasks
• you want to train on CPU, single GPU, or multiple GPUs with PyTorch Lightning

When to avoid

• you need modern LLM architectures like Llama or Mistral, which it does not support
• you need large-scale distributed training or the latest training techniques
• you want actively developed features; the project has seen limited recent updates

Facets

library · maturity maintenance

llm-training nlp machine-learning large-language-models machine-learning python cross-platform gpt-2 text-generation pytorch huggingface-transformers finetuning pytorch-lightning natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="minimaxir/aitextgen")