# Michael-A-Kuykendall/shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

Repository: https://github.com/Michael-A-Kuykendall/shimmy
Canonical: https://ross.abutalabs.com/products/shimmy
Language: Rust
License: Apache-2.0
License Family: permissive
Topics: llama, llamacpp, llm-inference, ollama-api, command-line-tool, gguf, inference-server, local-ai, machine-learning, api-server, developer-tools, huggingface, huggingface-models, huggingface-transformers, openai-compatible, rust, rust-crate, transformers, webgpu, webgpu-shaders
Last push: 2026-08-26T22:05:24+00:00

## Health v2 (maintenance only)
Score: 84/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 99, longevity 26
- inputs: {"age_days": 370, "days_push": 7, "days_rel": 7, "gap_med": 1.0, "n_releases_24m": 37}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 5808, forks 559 (observed 2026-08-28T04:09:29.885232+00:00)

## What it is
Shimmy is a single-binary, OpenAI-compatible LLM inference server written in pure Rust, running GGUF models on a WebGPU-based engine (Airframe) with no Python or C++ dependencies. It serves as a lightweight (~5MB) drop-in alternative to Ollama for local, private model inference on any GPU.

## Use cases
- run local LLM inference server without Python or llama.cpp
- serve GGUF models with an OpenAI-compatible API
- replace Ollama with a lightweight single-binary server
- run inference on NVIDIA, AMD, Intel, or Apple Silicon GPUs via WebGPU
- point existing AI tools at a local private inference endpoint
- serve Hugging Face GGUF models locally

## When to choose
- you want a minimal, dependency-free local inference server in a single binary
- you need OpenAI/Ollama API compatibility for existing tooling
- you want deterministic outputs and WebGPU acceleration across GPU vendors
- you prefer Rust-based tooling without Python runtimes or C++ toolchains

## When to avoid
- you need model architectures or quantizations outside the certified GGUF set
- you require advanced features like multi-GPU sharding or fine-tuning
- you depend on llama.cpp-specific extensions or ecosystem integrations
- you need a mature, battle-tested server like vLLM for high-throughput production serving

## Facets
- artifact type: service
- maturity: active
- function: llm-inference, http-server, api-framework, gpu-computing
- domain: large-language-models, artificial-intelligence, self-hosted, developer-tools
- platform: windows, rust, cli, self-hosted, cross-platform
- tags: openai-compatible, gguf, webgpu, ollama-alternative, local-ai, single-binary, inference-server, huggingface, linux, macos

## Member repositories
- Michael-A-Kuykendall/shimmy (main) score 84

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:09:29.885232+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:52:37.784387+00:00, confidence not recorded.
  - readme: https://github.com/Michael-A-Kuykendall/shimmy (fetched 2026-08-28T04:09:29.885232+00:00, sha 22c4da2ba096)
  - registry_crates: https://crates.io/api/v1/crates/shimmy (fetched 2026-08-29T08:47:44.324121+00:00, sha 924686afbd40)
- Data as of 2026-08-30T08:39:29.467469+00:00.