ngxson/wllama

Health v2 · maintenance only

Adoption not part of the score

1191 stars · 119 forks observed · 2026-09-03

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

wllama is a WebAssembly binding for llama.cpp that enables running LLM inference entirely in the browser, with WebGPU support, multimodal input, and tool calling. It ships as an npm package with an OpenAI-compatible typed API and runs inference in a web worker without any backend.

Use cases

• run LLM inference in the browser without a server
• generate text completions client-side with llama.cpp models
• compute embeddings in the browser for semantic search
• build a private chatbot that keeps data on-device
• run vision models on images directly in a web app
• add tool calling to an in-browser AI assistant

When to choose

• you want zero-backend, privacy-preserving LLM inference in a web app
• you need embeddings or completions computed client-side
• you want an OpenAI-compatible API over llama.cpp in the browser

When to avoid

• you need to run models larger than 2GB without splitting them
• you need high-throughput server-side inference with full GPU utilization
• your users' browsers lack WebAssembly SIMD or WebGPU support

Facets

library · maturity active

llm-inference sdk wasm chatbot rag large-language-models artificial-intelligence web-development frontend browser wasm llama-cpp webgpu in-browser-inference openai-compatible-api multimodal tool-calling embeddings typescript web-server nodejs

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ngxson/wllama")