ulab-uiuc/LLMRouter

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2547 stars · 259 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

LLMRouter is an open-source Python library that intelligently routes queries to the most suitable LLM based on task complexity, cost, and performance requirements. It provides 16+ router models across single-round, multi-round, multimodal, agentic, and personalized categories, plus a unified CLI for training, inference, and chat with a Gradio UI.

Use cases

• route llm queries to the cheapest model that meets quality requirements
• train a custom llm router on my own query data
• compare different llm routing strategies under a shared benchmark
• reduce llm api costs by dynamically selecting models per query
• build a cost-aware cascade across multiple llm providers
• evaluate routing policies on quality and token cost
• chat with routed models through a web ui

When to choose

• you serve many diverse queries and want to balance model quality against inference cost
• you want to develop, train, or benchmark LLM routing methods with a unified framework
• you need a ready-made pool of router algorithms (KNN, SVM, BERT-based, graph-based, Elo, etc.)
• you want a CLI and data generation pipeline for building routing supervision

When to avoid

• you only call a single LLM and have no need for model selection
• you need a production API gateway with rate limiting, auth, and load balancing rather than a research routing library
• your candidate model pool is too small for routing to pay off

Facets

library · maturity active

llm-inference machine-learning cli benchmarking rag large-language-models machine-learning artificial-intelligence developer-tools python cli cross-platform llm-routing model-selection cost-optimization router-training gradio-ui benchmark

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ulab-uiuc/LLMRouter")