facebookresearch/MobileLLM

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1459 stars · 89 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Meta's training code for MobileLLM, a family of sub-billion parameter language models optimized for on-device use, published at ICML 2024. It implements design techniques like SwiGLU activation, deep-thin architectures, embedding sharing, and grouped-query attention, with follow-up MobileLLM-R1 reasoning models.

Use cases

• train a small language model under 1B parameters
• build an LLM that runs on mobile or edge devices
• reproduce MobileLLM or MobileLLM-R1 training recipes
• pretrain a compact LLM for math and coding tasks
• research efficient LLM architecture design
• deploy a reasoning model with limited compute budget

When to choose

• you need to train or fine-tune sub-billion parameter LLMs for on-device deployment
• you want SoTA small-model accuracy on commonsense reasoning, math, or coding benchmarks
• you want full training code, data prep, and recipes released by Meta research

When to avoid

• you just want to run inference with an existing model rather than train one
• you need a production serving framework or mobile SDK rather than research training code
• you lack multi-GPU training infrastructure, as the code assumes 8-GPU nodes

Facets

library · maturity active

llm-training machine-learning deep-learning large-language-models machine-learning mobile-development python on-device-llm small-language-models research-code pytorch icml-2024 model-training edge-ai gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="facebookresearch/MobileLLM")