huggingface/pytorch-openai-transformer-lm

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1522 stars · 285 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch reimplementation of OpenAI's finetuned transformer language model (GPT-1) from the paper 'Improving Language Understanding by Generative Pre-Training', including a script to import OpenAI's TensorFlow pre-trained weights. It also provides a modified Adam optimizer with fixed weight decay and scheduled learning rate for fine-tuning on classification tasks.

Use cases

• load openai gpt-1 pretrained weights into pytorch
• run a transformer language model in pytorch
• fine-tune a pretrained transformer on text classification
• study the original gpt-1 model implementation
• convert tensorflow pretrained transformer weights to pytorch

When to choose

• you need the exact original GPT-1 architecture in PyTorch for research or reproduction
• you want to import OpenAI's original TensorFlow pre-trained weights into PyTorch

When to avoid

• you need modern pretrained models like GPT-2/3, BERT, or newer — use Hugging Face Transformers instead
• you want an actively maintained library with broad model support
• you need production-grade training tooling

Facets

library · maturity abandoned

machine-learning llm-training nlp deep-learning large-language-models python pytorch transformer gpt-1 pretrained-model language-model openai historical natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="huggingface/pytorch-openai-transformer-lm")