WebLLM

Health v2 · maintenance only

Adoption not part of the score

18603 stars · 1352 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

WebLLM is a high-performance in-browser LLM inference engine that runs language models entirely client-side using WebGPU hardware acceleration. It provides an OpenAI API-compatible interface as an npm package, letting developers build privacy-preserving AI web applications without any server-side inference.

Use cases

• run LLM chatbots entirely in the browser with no server
• build privacy-preserving AI assistants where user data never leaves the client
• integrate local LLM inference into web apps via OpenAI-compatible API
• generate structured JSON output from LLMs with schema constraints in the browser
• build a Chrome extension with local AI features
• offload inference to web workers to keep the UI responsive
• experiment with Llama, Phi, Gemma, Mistral, and Qwen models locally

When to choose

• you need LLM inference fully client-side for privacy or zero server cost
• your users have modern browsers with WebGPU support
• you want OpenAI API compatibility with local open-source models
• you're building a web app or browser extension with AI features

When to avoid

• you need to serve many users from a single backend or GPU cluster
• your target browsers lack WebGPU support
• you need very large models that exceed browser memory limits
• you need maximum inference throughput that native runtimes provide

Facets

library · maturity active

llm-inference machine-learning deep-learning chatbot sdk large-language-models artificial-intelligence web-development browser-extensions privacy browser wasm webgpu in-browser-inference openai-api-compatible client-side-ai local-llm privacy-preserving web-workers -mode function-calling web-server nodejs typescript

Member repositories

For agents

markdown · JSON · MCP: product_card(name="mlc-ai/web-llm")