microsoft/foundry-local

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

2587 stars · 378 forks observed · 2026-10-16

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Foundry Local is Microsoft's end-to-end local AI runtime and SDK suite (C#, JavaScript, Python, Rust) for running optimized models entirely on-device via ONNX Runtime. It bundles a curated model catalog, automatic NPU/GPU/CPU hardware acceleration, model lifecycle management, and an optional OpenAI-compatible local server.

Use cases

• run LLM chat completions locally without an API key or cloud backend
• transcribe audio on-device with Whisper
• ship offline AI features inside a desktop or mobile app
• point an existing OpenAI SDK app at a local endpoint
• serve local models to LangChain via an OpenAI-compatible server
• automatically pick the best execution provider for NPU, GPU, or CPU

When to choose

• you need on-device inference with data never leaving the user's machine
• you want zero network latency, offline operation, and no per-token costs
• you want automatic hardware acceleration and model management with minimal code
• your app already uses the OpenAI SDK and you want a drop-in local endpoint

When to avoid

• you need frontier-scale models that exceed consumer device hardware
• you require a permissive open-source license (license is custom/unspecified)
• you need multi-node or cloud-scale serving rather than single-device inference

Facets

library · maturity active

llm-inference speech-recognition sdk http-server machine-learning large-language-models artificial-intelligence speech-processing developer-tools windows cross-platform python rust dotnet on-device-ai onnx-runtime openai-compatible local-inference model-catalog hardware-acceleration offline-ai whisper macos nodejs gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/foundry-local")