sapientinc/HRM-Text

Health v2 · maintenance only

Flags: no_releases young

Adoption not part of the score

1899 stars · 180 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

HRM-Text is a 1B-parameter text generation model based on the hierarchical recurrent HRM architecture, released with a complete pretraining framework. It enables training a foundation model from scratch for roughly $1000 using 130-600x less compute and 150-900x less data than conventional approaches.

Use cases

• pretrain a small language model from scratch on a limited budget
• train a 1B text generation model on a few H100 GPUs
• experiment with hierarchical recurrent architectures for language modeling
• reproduce benchmark results for a compact LLM
• convert and evaluate pretrained model checkpoints
• study compute-efficient LLM pretraining recipes

When to choose

• you need to pretrain a small foundation model cheaply with Hopper-class GPUs
• you want to research or extend the HRM architecture for text generation
• you need a reproducible FSDP2-based pretraining pipeline with evaluation tooling

When to avoid

• you only need to fine-tune or run inference with an existing LLM
• you lack H100-class GPUs since the attention path depends on FlashAttention 3
• you need a production-ready general-purpose LLM with broad ecosystem support

Facets

library · maturity active

llm-training machine-learning deep-learning benchmarking large-language-models deep-learning machine-learning gpu-computing python pretraining hierarchical-reasoning-model hrm fsdp flashattention foundation-model prefixlm gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="sapientinc/HRM-Text")