# facebookresearch/large_concept_model

Large Concept Models: Language modeling in a sentence representation space

Repository: https://github.com/facebookresearch/large_concept_model
Canonical: https://ross.abutalabs.com/products/large_concept_model
Language: Python
License: MIT
License Family: permissive
Topics: language-models, nlp, pytorch, seq2seq, sequence-to-sequence
Last push: 2025-01-29T05:57:33+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 4, release rhythm 35, longevity 44
- inputs: {"age_days": 629, "days_push": 581, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2375, forks 213 (observed 2026-08-28T04:06:42.024097+00:00)

## What it is
Official PyTorch implementation of Meta's Large Concept Models (LCM), which perform language modeling by autoregressively predicting sentences in the SONAR sentence embedding space rather than at the token level. It includes training and finetuning recipes for 1.6B parameter MSE-regression and two-tower diffusion variants.

## Use cases
- train a large concept model on sentence embeddings
- reproduce the LCM paper experiments
- generate text sentence-by-sentence in a multilingual embedding space
- experiment with diffusion-based sequence generation in SONAR space
- finetune a 1.6B concept model
- research language modeling beyond token-level prediction

## When to choose
- you want to experiment with or reproduce Meta's Large Concept Model research
- you need sentence-level autoregressive generation in a multilingual embedding space
- you are researching alternatives to token-level language modeling

## When to avoid
- you need a production-ready LLM for chat or inference
- you want a plug-and-play library rather than research code
- you cannot set up fairseq2 and GPU dependencies yourself

## Facets
- artifact type: library
- maturity: experimental
- function: machine-learning, deep-learning, nlp, llm-training
- domain: large-language-models, machine-learning
- platform: python
- tags: language-models, seq2seq, sentence-embeddings, sonar, fairseq2, pytorch, diffusion, research-code, natural-language-processing, research, linux, gpu

## Member repositories
- facebookresearch/large_concept_model (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:42.024097+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:35:02.990490+00:00, confidence not recorded.
  - readme: https://github.com/facebookresearch/large_concept_model (fetched 2026-08-28T04:06:42.024097+00:00, sha 5c25ff0eb1cf)
- Data as of 2026-08-30T08:39:29.467469+00:00.