openai/finetune-transformer-lm

Health v2 · maintenance only

Flags: no_releases archived

Adoption not part of the score

2308 stars · 521 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Reference code and model for OpenAI's paper 'Improving Language Understanding by Generative Pre-Training' (the original GPT-1), implemented in Python with TensorFlow. It reproduces the ROCStories Cloze test results and is explicitly archived with no further updates.

Use cases

• reproduce the ROCStories cloze results from the GPT-1 paper
• study the original generative pre-training + fine-tuning approach
• fine-tune a pretrained transformer language model on a story completion task
• understand how GPT-1 was originally implemented

When to choose

• you need the exact historical code for the GPT-1 paper
• you are doing research on early transformer LM fine-tuning methods

When to avoid

• you want a maintained library for fine-tuning modern language models
• you need deterministic training results
• you want production-ready NLP tooling or current model architectures

Facets

library · maturity abandoned

machine-learning deep-learning nlp llm-training llm-inference machine-learning deep-learning large-language-models python gpt-1 transformer generative-pretraining fine-tuning research-code archived rocstories paper-code natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="openai/finetune-transformer-lm")