# WebLLM

High-performance In-browser LLM Inference Engine

Repository: https://github.com/mlc-ai/web-llm
Canonical: https://ross.abutalabs.com/products/webllm
Homepage: https://webllm.mlc.ai
Language: TypeScript
License: Apache-2.0
License Family: permissive
Topics: deep-learning, llm, tvm, webgpu, webml, chatgpt, language-model
Last push: 2026-08-04T10:46:09+00:00
Link (homepage): https://webllm.mlc.ai
Link (site_page): https://webllm.mlc.ai/docs

## Health v2 (maintenance only)
Score: 85/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 96, release rhythm 69, longevity 88
- inputs: {"age_days": 1238, "days_push": 29, "days_rel": 131, "gap_med": 32.0, "n_releases_24m": 3}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 18603, forks 1352 (observed 2026-08-28T04:11:26.833865+00:00)

## What it is
WebLLM is a high-performance in-browser LLM inference engine that runs language models entirely client-side using WebGPU hardware acceleration. It provides an OpenAI API-compatible interface as an npm package, letting developers build privacy-preserving AI web applications without any server-side inference.

## Use cases
- run LLM chatbots entirely in the browser with no server
- build privacy-preserving AI assistants where user data never leaves the client
- integrate local LLM inference into web apps via OpenAI-compatible API
- generate structured JSON output from LLMs with schema constraints in the browser
- build a Chrome extension with local AI features
- offload inference to web workers to keep the UI responsive
- experiment with Llama, Phi, Gemma, Mistral, and Qwen models locally

## When to choose
- you need LLM inference fully client-side for privacy or zero server cost
- your users have modern browsers with WebGPU support
- you want OpenAI API compatibility with local open-source models
- you're building a web app or browser extension with AI features

## When to avoid
- you need to serve many users from a single backend or GPU cluster
- your target browsers lack WebGPU support
- you need very large models that exceed browser memory limits
- you need maximum inference throughput that native runtimes provide

## Facets
- artifact type: library
- maturity: active
- function: llm-inference, machine-learning, deep-learning, chatbot, sdk
- domain: large-language-models, artificial-intelligence, web-development, browser-extensions, privacy
- platform: browser, wasm
- tags: webgpu, in-browser-inference, openai-api-compatible, client-side-ai, local-llm, privacy-preserving, web-workers, -mode, function-calling, web-server, nodejs, typescript

## Member repositories
- mlc-ai/web-llm (main) score 85
- mlc-ai/web-llm-chat (frontend) score 55

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:11:26.833865+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:02:00.529029+00:00, confidence not recorded.
  - readme: https://github.com/mlc-ai/web-llm (fetched 2026-08-28T04:11:26.833865+00:00, sha f0cc4fc99799)
  - homepage: https://webllm.mlc.ai (fetched 2026-08-29T08:00:17.427141+00:00, sha 5f2308874c28)
  - site_page: https://webllm.mlc.ai/docs (fetched 2026-08-29T08:00:17.430249+00:00, sha 1445473a6b8b)
- Data as of 2026-08-30T08:39:29.467469+00:00.