# theroyallab/tabbyAPI

The official API server for Exllama. OAI compatible, lightweight, and fast.

Repository: https://github.com/theroyallab/tabbyAPI
Canonical: https://ross.abutalabs.com/products/tabbyapi
Language: Python
License: AGPL-3.0
License Family: copyleft
Last push: 2026-08-26T14:34:51+00:00

## Health v2 (maintenance only)
Score: 71/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 99, release rhythm 35, longevity 73
- inputs: {"age_days": 1027, "days_push": 7, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1324, forks 179 (observed 2026-08-28T04:04:22.426389+00:00)

## What it is
TabbyAPI is a FastAPI-based OpenAI-compatible API server for running large language models locally using the ExllamaV3 backend on NVIDIA GPUs. It serves as the official API backend for Exllama, supporting text generation and tool calling.

## Use cases
- serve local LLMs with an OpenAI-compatible API
- run ExllamaV3 models on a GPU
- self-host a lightweight LLM inference server
- add tool calling to a local language model
- generate text via HTTP API from local models
- replace OpenAI endpoints with a self-hosted backend

## When to choose
- you want a fast, lightweight OpenAI-compatible server for Exllama models
- you run local LLM inference on NVIDIA GPUs
- you need tool calling support with a local backend
- you prefer a hobbyist-friendly rolling-release server over heavyweight production stacks

## When to avoid
- you need a production-grade, high-concurrency inference server
- you want to run GGUF models (use its sister project YALS instead)
- you need ExLlamaV2 support on the main branch
- you lack an NVIDIA GPU

## Facets
- artifact type: application
- maturity: active
- function: llm-inference, http-server, api-framework
- domain: large-language-models, artificial-intelligence, self-hosted
- platform: windows, python, self-hosted
- tags: openai-compatible, exllama, fastapi, text-generation, local-llm, tool-calling, linux, docker, gpu

## Member repositories
- theroyallab/tabbyAPI (main) score 71

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:04:22.426389+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T04:47:02.253267+00:00, confidence not recorded.
  - readme: https://github.com/theroyallab/tabbyAPI (fetched 2026-08-28T04:04:22.426389+00:00, sha 52e110efb05a)
- Data as of 2026-08-30T08:39:29.467469+00:00.