facebookresearch/large_concept_model

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2375 stars · 213 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Official PyTorch implementation of Meta's Large Concept Models (LCM), which perform language modeling by autoregressively predicting sentences in the SONAR sentence embedding space rather than at the token level. It includes training and finetuning recipes for 1.6B parameter MSE-regression and two-tower diffusion variants.

Use cases

• train a large concept model on sentence embeddings
• reproduce the LCM paper experiments
• generate text sentence-by-sentence in a multilingual embedding space
• experiment with diffusion-based sequence generation in SONAR space
• finetune a 1.6B concept model
• research language modeling beyond token-level prediction

When to choose

• you want to experiment with or reproduce Meta's Large Concept Model research
• you need sentence-level autoregressive generation in a multilingual embedding space
• you are researching alternatives to token-level language modeling

When to avoid

• you need a production-ready LLM for chat or inference
• you want a plug-and-play library rather than research code
• you cannot set up fairseq2 and GPU dependencies yourself

Facets

library · maturity experimental

machine-learning deep-learning nlp llm-training large-language-models machine-learning python language-models seq2seq sentence-embeddings sonar fairseq2 pytorch diffusion research-code natural-language-processing research linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="facebookresearch/large_concept_model")