# LlamaEdge/LlamaEdge

The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge

Repository: https://github.com/LlamaEdge/LlamaEdge
Canonical: https://ross.abutalabs.com/products/llamaedge
Homepage: https://llamaedge.com/
Language: Rust
License: Apache-2.0
License Family: permissive
Last push: 2026-02-08T03:32:28+00:00

## Health v2 (maintenance only)
Score: 65/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 66, release rhythm 57, longevity 75
- inputs: {"age_days": 1059, "days_push": 206, "days_rel": 206, "gap_med": 4, "n_releases_24m": 64}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1650, forks 149 (observed 2026-08-28T04:05:16.882152+00:00)

## What it is
LlamaEdge is a lightweight Rust and WasmEdge-based runtime for running open-source LLMs locally or on edge devices, with CLI chat apps and OpenAI-compatible API servers. It supports text generation, embeddings, speech-to-text, text-to-speech, and image generation models in a compact, dependency-free package.

## Use cases
- run open-source LLMs locally on my laptop
- serve an OpenAI-compatible API for a local model
- chat with a fine-tuned GGUF model from the command line
- host a private LLM API server without Python dependencies
- run LLM inference on edge devices with GPU acceleration
- create a chatbot web UI backed by a local model
- generate embeddings locally for a knowledge base

## When to choose
- you need a lightweight, portable local LLM runtime without Python or heavy dependencies
- you want OpenAI-compatible endpoints for open-source models on CPU or GPU
- you deploy LLM apps across heterogeneous edge hardware (CPUs, GPUs, NPUs)
- you want a single integrated runtime plus API server instead of gluing multiple tools

## When to avoid
- you need the broadest model support or ecosystem of Ollama or llama.cpp
- you rely on Python-based tooling and want to extend the runtime in Python
- you require multi-node distributed inference or large-scale serving
- you need features beyond the supported model types (e.g., advanced fine-tuning)

## Facets
- artifact type: application
- maturity: active
- function: llm-inference, http-server, cli, speech-recognition, tts, rag
- domain: large-language-models, artificial-intelligence, self-hosted, developer-tools
- platform: cross-platform, cli, self-hosted, wasm
- tags: openai-compatible-api, gguf, wasmedge, local-llm, edge-computing, chatbot-ui, gpu

## Member repositories
- LlamaEdge/LlamaEdge (main) score 65

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:16.882152+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:45:04.512568+00:00, confidence not recorded.
  - readme: https://github.com/LlamaEdge/LlamaEdge (fetched 2026-08-28T04:05:16.882152+00:00, sha 1ed0618bbfb4)
  - homepage: https://llamaedge.com/ (fetched 2026-08-29T11:18:07.644776+00:00, sha 6a2f155de4bd)
  - site_page: https://llamaedge.com/docs/intro (fetched 2026-08-29T11:18:07.648338+00:00, sha c3f6ed40aad1)
- Data as of 2026-08-30T08:39:29.467469+00:00.