function: llm-inference

3362 products, primary matches first, then adoption-weighted; health v2 shown.

Product	Health v2	Stars	Maturity
Hermes Agent Hermes Agent is a self-improving AI agent framework by Nous Research that builds skills from experience, maintains curated memory, and mode…	85	236895	active
anomalyco/opencode OpenCode is an open source AI coding agent that helps developers write code from the terminal, desktop app, or IDE extension. It supports 7…	82	201714	active
Significant-Gravitas/AutoGPT AutoGPT is an open-source platform for building, deploying, and running AI agents that execute complete workflows, via natural-language cha…	93	186906	active
Ollama Ollama is an open-source tool for downloading, running, and managing large language models locally on macOS, Windows, and Linux, with a sim…	92	179521	active
huggingface/transformers Hugging Face Transformers is a Python library that serves as the model-definition framework for state-of-the-art machine learning models ac…	95	164475	stable
langflow-ai/langflow Langflow is an open-source, Python-based visual builder for creating, testing, and deploying AI agents and workflows. It provides a drag-an…	98	153719	stable
Open WebUI Open WebUI is a self-hosted, extensible web interface for interacting with large language models, supporting Ollama, OpenAI-compatible APIs…	90	150036	active
langchain-ai/langchain LangChain is an open-source Python framework for building LLM-powered agents and applications, providing a configurable agent harness (crea…	95	145076	stable
ggml-org/llama.cpp llama.cpp is a C/C++ library and toolset for running large language model inference locally on CPUs and GPUs, built on the ggml tensor libr…	93	125803	stable
openai/codex OpenAI's Codex CLI is a lightweight coding agent that runs locally in your terminal, helping you write, review, and refactor code with AI a…	82	118785	active
harry0703/MoneyPrinterTurbo MoneyPrinterTurbo is an AI-powered application that generates high-definition short videos from a topic or keyword. It automatically writes…	92	116915	active
deepseek-ai/DeepSeek-V3 Official repository for DeepSeek-V3, a 671B-parameter Mixture-of-Experts language model (37B activated per token) with open weights, infere…	29	104416	active
JuliusBrussee/caveman Caveman is an efficiency stack for AI coding agents that reduces token usage through a terse-writing skill, a payload-compression engine, a…	81	101200	active
TauricResearch/TradingAgents TradingAgents is a Python framework that orchestrates multiple LLM-powered agents to analyze financial markets and make trading decisions. …	83	100743	active
earendil-works/pi Pi is a TypeScript AI agent toolkit providing a unified multi-provider LLM API, an agent runtime with tool calling, a terminal UI library, …	80	97788	active
deepseek-ai/DeepSeek-R1 DeepSeek-R1 is a family of open-weight large language models trained with large-scale reinforcement learning for reasoning, including DeepS…	24	92038	active
vllm-project/vllm vLLM is a high-throughput, memory-efficient inference and serving engine for large language models, featuring PagedAttention, continuous ba…	93	90149	active
ChatGPTNextWeb/NextChat NextChat is a lightweight, fast open-source AI chat assistant web UI supporting multiple LLM providers like OpenAI GPT-4, Claude, Gemini, D…	76	88649	active
OpenHands OpenHands (Agent Canvas) is a self-hosted developer control center for running AI coding agents like OpenHands, Claude Code, Codex, and Gem…	87	85199	active
bytedance/deer-flow DeerFlow is an open-source SuperAgent harness from ByteDance that orchestrates a lead agent with sub-agents, memory, tools, skills, and Doc…	72	80958	active
unclecode/crawl4ai Crawl4AI is an open-source Python library that crawls websites with a headless browser and converts pages into clean, LLM-ready Markdown fo…	89	79471	active
rtk-ai/rtk RTK (Rust Token Killer) is a CLI proxy that intercepts shell commands and compresses their output before it reaches an LLM agent's context …	78	77533	active
GPT4All GPT4All is an open-source desktop application that lets users run large language models locally on everyday hardware without GPUs or API ca…	42	77397	active
AppFlowy AppFlowy is an open-source AI collaborative workspace and Notion alternative for projects, wikis, docs, and task management, built with Flu…	98	75984	active
Unsloth Unsloth is a desktop application for running and fine-tuning LLMs, diffusion, embedding, and audio models locally, with support for NVIDIA,…	94	74883	active
abi/screenshot-to-code An AI-powered application that converts screenshots, mockups, Figma designs, and screen recordings into clean functional code (HTML/Tailwin…	70	74677	active
binary-husky/gpt_academic GPT Academic is a self-hosted Python application providing a practical chat interface for LLMs like GPT, GLM, Qwen, DeepSeek, and Claude, w…	57	71245	active
FoundationAgents/MetaGPT MetaGPT is an open-source multi-agent framework that assigns roles like product manager, architect, and engineer to LLM-powered agents, sim…	48	70050	active
openinterpreter/openinterpreter Open Interpreter is a Rust-based terminal coding agent optimized for low-cost open models like Kimi K3, DeepSeek, and Qwen. It emulates pro…	95	68163	active
cline/cline Cline is an open-source autonomous coding agent available as a VS Code extension, JetBrains plugin, CLI, and Node.js SDK. It reads and writ…	86	66919	active
xtekky/gpt4free GPT4Free (g4f) is a Python library and application that aggregates many free/reverse-engineered LLM providers behind an OpenAI-compatible A…	93	66605	active
mem0ai/mem0 Mem0 is a memory layer for AI agents and LLM applications that extracts, stores, and retrieves persistent user context across sessions. It …	92	64121	active
virattt/ai-hedge-fund An educational AI-powered hedge fund simulator where LLM-driven agents modeled after famous investors analyze stocks and make simulated tra…	86	63060	active
FoundationAgents/OpenManus OpenManus is an open-source framework for building general-purpose AI agents, created by the MetaGPT team as an open alternative to Manus. …	66	58085	active
CrewAI CrewAI is an open-source Python framework for orchestrating role-playing, autonomous AI agents that collaborate on complex tasks. It offers…	89	57650	active
karpathy/nanochat nanochat is a minimal, hackable full-stack LLM training harness that covers tokenization, pretraining, finetuning, evaluation, and inferenc…	60	57548	active
zylon-ai/private-gpt PrivateGPT is an open-source Python API layer that turns local or OpenAI-compatible LLM servers into production AI applications, providing …	93	57471	active
diegosouzapw/OmniRoute OmniRoute is a free, MIT-licensed AI gateway that exposes a single OpenAI-compatible endpoint routing to 350+ LLM providers (90+ with free …	78	56223	active
aaif-goose/goose goose is an open-source, extensible general-purpose AI agent built in Rust that runs locally as a desktop app, CLI, and API. It connects to…	85	53534	active
ggml-org/whisper.cpp A high-performance C/C++ port of OpenAI's Whisper automatic speech recognition model with no external dependencies. It supports CPU, GPU (M…	99	53203	stable
LlamaIndex LlamaIndex is a Python (with a TypeScript variant) framework for building LLM-powered applications, centered on ingesting, indexing, and qu…	95	51883	stable
jamiepine/voicebox Voicebox is a free, open-source, local-first AI voice studio desktop app that combines voice cloning, text-to-speech across 7 engines, and …	75	51551	active
Cherry Studio Cherry Studio is a cross-platform desktop AI productivity application that provides smart chat, autonomous agents, and 300+ prebuilt assist…	87	51102	active
Alishahryar1/free-claude-code An open-source Python tool that lets you run coding agents like Claude Code, Codex, Pi, and OpenCode against a catalog of 50 ToS-friendly p…	60	50378	active
hugohe3/ppt-master PPT Master is an open-source AI application that turns documents or topics into natively editable PowerPoint (.pptx) decks with real shapes…	83	49604	active
Aider-AI/aider Aider is an open-source CLI tool that lets developers pair program with LLMs (Claude, GPT-4o, DeepSeek, Gemini, local models) directly in t…	69	48517	active
moeru-ai/airi Project AIRI is an open-source, self-hosted AI companion (waifu/virtual character) platform inspired by Neuro-sama, supporting Live2D and V…	86	48451	active
exo-explore/exo exo is a Python application that turns a collection of devices (Macs, GPUs, etc.) into a distributed AI cluster for running large language …	84	47081	active
zhayujie/CowAgent CowAgent (formerly chatgpt-on-wechat) is an open-source AI assistant and Agent Harness that plans tasks, runs tools and skills, and self-ev…	99	46692	active
QuantumNous/new-api New API is a self-hosted LLM gateway and AI asset management system that aggregates 30+ model providers behind a single unified endpoint. I…	90	46459	active
janhq/jan Jan is an open-source desktop application that serves as a ChatGPT alternative, letting users download and run large language models fully …	93	44202	active
danielmiessler/Fabric Fabric is an open-source framework for augmenting humans with AI by organizing crowdsourced AI prompts (called patterns) into a modular, re…	86	43541	active
DeepSpeed DeepSpeed is a deep learning optimization library from Microsoft that makes distributed training and inference of large models fast, memory…	98	43003	stable
chatboxai/chatbox Chatbox is a cross-platform desktop and mobile AI client for chatting with LLMs like ChatGPT, Claude, Gemini, DeepSeek, and local models vi…	95	41574	active
chatanywhere/GPT_API_free A hosted API forwarding/proxy service that provides free daily-quota access to GPT, DeepSeek, Claude, Gemini, and other major LLMs via a un…	72	41190	active
langchain-ai/langgraph LangGraph is a low-level orchestration framework and runtime for building, managing, and deploying long-running, stateful AI agents in Pyth…	91	40503	stable
microsoft/BitNet bitnet.cpp is Microsoft's official inference framework for 1-bit large language models (BitNet b1.58), providing optimized CPU kernels with…	65	40203	active
AstrBotDevs/AstrBot AstrBot is a multi-platform LLM chatbot and agentic AI assistant development framework written in Python. It integrates many instant messag…	94	39649	active
mindsdb/mindshub MindsHub Cowork is an open-source AI workspace where users delegate multi-step projects (apps, reports, research, scheduled operations) to …	88	39649	active
Wei-Shaw/sub2api Sub2API is a self-hosted AI API gateway written in Go that converts Claude, OpenAI, Gemini, and Grok subscriptions into unified API access.…	83	39526	active
QuivrHQ/quivr Quivr is an opinionated RAG (Retrieval-Augmented Generation) library, available as the quivr-core Python package, that lets developers inge…	45	39423	active
HKUDS/LightRAG LightRAG is a Python framework for retrieval-augmented generation that integrates graph structures into text indexing and retrieval, using …	85	39204	active
RSSNext/Folo Folo is an open-source AI-powered RSS reader that aggregates and summarizes content feeds across desktop and mobile platforms. It integrate…	86	38862	active
google/langextract LangExtract is a Python library that uses LLMs to extract structured information from unstructured text based on user-defined instructions …	82	38490	active
tinyhumansai/openhuman OpenHuman is an open-source personal AI desktop application built in Rust that acts as a 'second brain' with a local-first memory engine, a…	81	38212	active
stanfordnlp/dspy DSPy is a Python framework from Stanford NLP for programming language model systems with declarative signatures and modules instead of hand…	94	37612	active
HKUDS/DeepTutor DeepTutor is an open-source agentic AI tutoring framework that combines citation-grounded problem tutoring with difficulty-calibrated quest…	78	37574	active
musistudio/claude-code-router Claude Code Router (CCR) is a self-hosted local model gateway and control plane that routes AI coding agent requests (Claude Code, Codex, G…	87	36893	active
songquanpeng/one-api One API is a self-hosted LLM API gateway and key management system that unifies OpenAI, Azure, Anthropic Claude, Google Gemini, DeepSeek, a…	55	36591	stable
ItzCrazyKns/Vane Vane is a self-hosted, privacy-focused AI answering engine that combines web search (via SearxNG) with LLMs to deliver cited answers. It su…	74	36489	active
MadsLorentzen/ai-job-search An AI-powered job application framework built on Claude Code that runs locally: it evaluates job postings, tailors CVs, writes cover letter…	81	36468	active
PDFMathTranslate/PDFMathTranslate PDFMathTranslate (pdf2zh) is a tool that translates scientific PDF documents while preserving the original layout, including formulas, char…	69	36368	active
reworkd/AgentGPT AgentGPT is a web application for assembling, configuring, and deploying autonomous AI agents directly in the browser. Users name a custom …	10	36303	active
VectifyAI/PageIndex PageIndex is a Python SDK and framework for vectorless, reasoning-based RAG that replaces vector similarity search with a hierarchical tree…	87	35333	active
DayuanJiang/next-ai-draw-io A Next.js web application that combines AI chat with draw.io diagram editing, letting users create and modify diagrams through natural lang…	81	35313	active
esengine/DeepSeek-Reasonix Reasonix (DeepSeek-Reasonix) is a DeepSeek-native AI coding agent delivered as a single Go binary with a terminal TUI, desktop app, browser…	77	35192	active
anthropics/financial-services A collection of reference AI agents, skills, and data connectors for financial-services workflows like investment banking, equity research,…	59	34533	active
AlexsJones/llmfit llmfit is a Rust-based terminal tool (TUI and CLI) that matches LLM models to your hardware by detecting RAM, CPU, and GPU and scoring mode…	78	34236	active
OpenBMB/ChatDev ChatDev 2.0 (DevAll) is a zero-code multi-agent orchestration platform where users configure agents, workflows, and tasks to build LLM-powe…	80	34126	active
TabbyML/tabby Tabby is a self-hosted, open-source AI coding assistant that serves as an on-premises alternative to GitHub Copilot. It provides LLM-powere…	82	33838	active
linshenkx/prompt-optimizer Prompt Optimizer is an AI prompt optimization tool that helps users write, refine, test, and evaluate prompts for better LLM outputs. It sh…	87	33385	active
cursor/cursor Cursor is an AI-powered code editor and coding agent (a fork of VS Code) that lets developers delegate implementation tasks to AI agents, w…	67	33180	active
SillyTavern/SillyTavern SillyTavern is a locally installed, self-hosted web UI for interacting with text generation LLMs, image generation engines, and TTS voice m…	87	32691	active
lyogavin/airllm AirLLM is a Python library that dramatically reduces LLM inference memory usage by streaming model layers (and MoE experts) from disk, enab…	96	32573	active
SGLang SGLang is a high-performance open-source serving framework for large language models and multimodal models, offering low-latency, high-thro…	93	32504	stable
HKUDS/Vibe-Trading Vibe-Trading is an open-source AI finance research agent that turns natural-language prompts into runnable market research, backtests, and …	81	31791	active
openai/openai-python The official Python client library for the OpenAI REST API, generated from OpenAI's OpenAPI specification. It provides typed, synchronous a…	95	31467	stable
alchaincyf/nuwa-skill Nuwa.skill is an Agent Skills-compatible package that distills how a person thinks—mental models, decision heuristics, and expression style…	59	31438	active
hsliuping/TradingAgents-CN A Chinese-enhanced fork of the TradingAgents multi-agent LLM framework for stock analysis and trading research, supporting A-shares, Hong K…	61	31393	active
stanford-oval/storm STORM is an LLM-powered system that researches a topic via internet search and generates full-length, cited, Wikipedia-like reports. It is …	38	31135	active
Gitlawb/openclaude OpenClaude is an open-source terminal-based coding agent CLI that reads and edits files, runs shell commands, searches codebases, and talks…	81	30843	active
ScrapeGraphAI/Scrapegraph-ai ScrapeGraphAI is a Python library that uses LLMs and direct graph logic to build web scraping pipelines for websites and local documents (X…	88	29959	active
Zackriya-Solutions/meetily Meetily is a privacy-first, open-source AI meeting assistant that records, transcribes, and summarizes meetings entirely locally using Para…	79	29940	active
sipeed/picoclaw PicoClaw is an ultra-lightweight, self-hosted personal AI assistant written in Go that ships as a single binary running in under 10MB of RA…	79	29910	active
AgentScope AgentScope is a production-ready multi-agent framework for building LLM-powered agents with ReAct reasoning, tool execution, MCP integratio…	93	29706	active
tobi/qmd QMD is a local, on-device CLI search engine that indexes markdown notes, docs, meeting transcripts, and knowledge bases. It combines BM25 f…	82	29259	active
modular/modular Modular Platform hosting the MAX AI serving framework and the Mojo systems programming language. It provides an OpenAI-compatible inference…	92	29225	active
huggingface/smolagents smolagents is a lightweight Python library from Hugging Face for building LLM-powered agents in a few lines of code, with first-class suppo…	84	29005	active
openai/openai-agents-python OpenAI Agents SDK is a lightweight Python framework for building multi-agent LLM workflows with agents, handoffs, tools, guardrails, sessio…	82	28982	active
alibaba/page-agent Page Agent is a JavaScript library that embeds an AI GUI agent directly into any web page with a single script, letting users control the i…	82	28847	active