function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| RUC-NLPIR/WebThinker WebThinker is a research framework that empowers large reasoning models (like DeepSeek-R1 and QwQ) with deep research capability, letting t… | 45 | 1467 | active |
| QuantaAlpha/QuantaAlpha QuantaAlpha is an LLM-driven framework for mining quantitative alpha factors using a trajectory-based self-evolving paradigm. Users describ… | 56 | 1466 | active |
| fictionco/fiction Fiction is a next-generation CMS and application framework for building AI-generated content, personal websites, marketing funnels, and new… | 44 | 1465 | active |
| Meta-Harness Meta-Harness is a Python framework for automated end-to-end search over task-specific model harnesses — the code surrounding a fixed base L… | 56 | 1464 | active |
| NousResearch/Hermes-Function-Calling A repository providing function-calling capabilities for the Hermes Pro large language models, including stock price and financial data too… | 51 | 1457 | active |
| HexmosTech/git-lrc git-lrc is a free, open-source tool that runs micro AI code reviews automatically on every git commit. It is written in Go and integrates w… | 82 | 1454 | active |
| shenminglinyi/PlotPilot PlotPilot (墨枢) is an open-source narrative engine kernel for long-form AI-assisted fiction and script writing, built in Python with FastAPI… | 77 | 1454 | active |
| vitalops/openvibe Openvibe is a Python implementation of opencode, an open-source AI coding agent that runs terminal commands, edits files, and executes agen… | 61 | 1450 | active |
| gabotechs/MusicGPT MusicGPT is a Rust application that runs music generation AI models (currently Meta's MusicGen) locally, generating music from natural lang… | 28 | 1450 | active |
| opencx-labs/OpenChat OpenChat is a self-hosted chatbot console that lets everyday users create and manage unlimited custom chatbots powered by GPT-3/GPT-4 with … | 10 | 5237 | maintenance |
| qwersyk/Newelle Newelle is a GTK4/GNOME virtual assistant application for Linux that connects to local (Ollama, Llama.cpp) and cloud LLM providers. It supp… | 96 | 1449 | active |
| sauravpanda/BrowserAI BrowserAI is a TypeScript library for running LLMs, speech recognition, text-to-speech, and audio separation models directly in the browser… | 78 | 1449 | active |
| Paritok-official/paritok-4b-v1 Paritok is a non-destructive compression gateway that sits as a drop-in proxy between AI coding agents (Claude Code, Cursor, Codex, OpenHan… | 57 | 1449 | active |
| langchain-ai/openwork Openwork is an Electron-based desktop interface for deepagentsjs, providing a harness for building and running deep AI agents with filesyst… | 71 | 1448 | active |
| wordware-ai/twitter A Next.js web application that generates AI-powered personality profiles and compatibility analyses from Twitter/X handles using an LLM age… | 67 | 1446 | active |
| morsoli/aimangastudio AIMangaStudio is a web application that uses AI (Google GenAI) to create manga/comics end-to-end, including script generation, storyboard l… | 62 | 1446 | active |
| dust-tt/dust Dust is an open-source platform for building, deploying, and managing custom AI agents powered by large language models. It combines a web … | 99 | 1442 | active |
| Nutlope/pdftochat PDFToChat is an open-source Next.js web application that lets users upload PDFs and chat with them using AI. It uses Together AI's Mixtral … | 68 | 1441 | active |
| xhongc/ai_story AI Story is a self-hosted platform that automates story video production: given a topic, it generates scripts, storyboards, AI images, came… | 60 | 1441 | active |
| valentinfrlch/ha-llmvision LLM Vision is a Home Assistant integration (installed via HACS) that uses multimodal large language models to analyze images, videos, live … | 90 | 1440 | active |
| bramses/chatgpt-md An Obsidian community plugin that integrates ChatGPT-style AI chat directly into notes, supporting wiki-link context, AI agents, and tool c… | 96 | 1439 | active |
| fawney19/Aether Aether is a self-hosted AI API gateway written in Rust that provides a unified entry point for Claude, OpenAI, Gemini, and their CLI client… | 80 | 1439 | active |
| Danau5tin/multi-agent-coding-system A Python multi-agent AI coding system where an orchestrator agent dispatches explorer and coder subagents with shared knowledge artifacts, … | 33 | 1438 | active |
| wannabespace/conar Conar (formerly Tamery) is an open-source, AI-powered desktop database client built with Electron for PostgreSQL, MySQL, MSSQL, and ClickHo… | 81 | 1436 | active |
| xuanyustudio/LocalMiniDrama LocalMiniDrama is an open-source Electron desktop application that generates AI short dramas and animated dramas end-to-end, from story and… | 78 | 1436 | active |
| edwko/OuteTTS OuteTTS is a Python interface for running OuteAI's text-to-speech models, supporting multiple inference backends including llama.cpp, Huggi… | 52 | 1436 | active |
| salesforce/CodeGen CodeGen is a family of open-source large language models (350M to 16B parameters) from Salesforce AI Research for program synthesis, genera… | 70 | 5179 | maintenance |
| eumemic/ai-legion AI Legion is a TypeScript framework for running LLM-powered autonomous agents that collaborate to accomplish tasks. Agents are managed thro… | 40 | 1435 | active |
| jekalmin/extended_openai_conversation A Home Assistant custom component that acts as a conversation agent powered by OpenAI. It uses OpenAI's function calling to control Home As… | 77 | 1431 | active |
| google/oss-fuzz-gen A Google framework that uses large language models to automatically generate fuzz targets for real-world C/C++, Java, and Python projects, … | 58 | 1431 | active |
| Darkatse/TauriTavern TauriTavern is a native rewrite of SillyTavern built with Tauri v2 and a Rust backend, providing the classic LLM chat/roleplay frontend as … | 85 | 1427 | active |
| six2dez/burp-ai-agent Custom AI Agent (formerly Burp AI Agent) is a Burp Suite extension written in Kotlin that integrates LLMs into web security workflows via l… | 82 | 1427 | stable |
| jakobhoeg/nextjs-ollama-llm-ui A fully-featured, ChatGPT-inspired web interface for chatting with local Ollama LLMs, built with Next.js, React, and Tailwind. It runs full… | 30 | 1426 | active |
| andrew-waters/orchard Orchard is a native Swift macOS GUI for Apple's container tooling, serving as a Docker Desktop replacement for managing containers, Linux V… | 85 | 1425 | active |
| ammaarreshi/gemma-chat An open-source Electron app that runs Google's Gemma 4 models locally on Apple Silicon via Apple's MLX framework, providing a chat interfac… | 50 | 1424 | active |
| deepseek-ai/EPLB EPLB is DeepSeek's open-source Expert Parallelism Load Balancer for Mixture-of-Experts models. It computes balanced expert replication and … | 26 | 1424 | active |
| shobrook/wut wut is a Python CLI tool that sends the output of your last terminal command to an LLM (OpenAI, Claude, or a local Ollama model) and return… | 22 | 1424 | active |
| AgentAlphaAGI/Idea2Paper Idea2Paper is an end-to-end multi-agent framework that decomposes the research process into structured phases, turning raw research ideas i… | 49 | 1421 | active |
| acon96/home-llm A Home Assistant custom integration plus fine-tuned small language models that let you control your smart home entirely with a local LLM, n… | 87 | 1419 | active |
| WPeace-HcH/WPeGPT WPeGPT is an IDA Pro plugin that integrates LLM models (OpenAI, DeepSeek, or any OpenAI-compatible API) into binary analysis workflows. It … | 75 | 1418 | active |
| InternScience/InternAgent InternAgent-1.5 is a Python-based unified agentic framework for long-horizon autonomous scientific discovery, orchestrating multi-agent sys… | 62 | 1415 | active |
| myshell-ai/AIlice AIlice is a fully autonomous, general-purpose AI agent built on open-source LLMs, designed as a JARVIS-like standalone assistant. It uses a… | 34 | 1412 | active |
| Dryxio/auto-re-agent auto-re-agent is an open-source AI reverse-engineering agent that combines Ghidra binary analysis with LLMs (Claude, Codex, OpenAI-compatib… | 78 | 1410 | active |
| RahulSChand/gpu_poor A web-based calculator that estimates GPU memory requirements and inference/finetuning throughput (token/s) for any LLM. It supports quanti… | 18 | 1405 | active |
| XiaoMi/mace MACE (Mobile AI Compute Engine) is a deep learning inference framework optimized for mobile heterogeneous computing on Android, iOS, Linux … | 23 | 5046 | maintenance |
| joewongjc/type4me Type4Me is a macOS voice input method that transcribes speech in real time (streaming, local or cloud ASR engines) and refines text with LL… | 81 | 1404 | active |
| hotovo/aider-desk AiderDesk is an open-source desktop application (Electron) that provides a GUI and orchestration layer for AI-assisted software development… | 84 | 1401 | active |
| 0nutation/SpeechGPT SpeechGPT is a series of speech large language models that can perceive and generate speech, supporting cross-modal instruction following a… | 29 | 1401 | active |
| awaescher/OllamaSharp OllamaSharp is a .NET client library providing bindings for the entire Ollama API, including chat, generation, embeddings, and model manage… | 86 | 1400 | active |
| explosion/spacy-llm spacy-llm is a Python library that integrates Large Language Models into spaCy NLP pipelines via a serializable llm component. It provides … | 67 | 1394 | active |
| callstackincubator/ai A collection of on-device AI primitives for React Native offering Vercel AI SDK-compatible APIs for running LLMs, embeddings, transcription… | 76 | 1387 | active |
| robertpiosik/CodeWebChat Code Web Chat (CWC) is a free, privacy-first VS Code extension toolkit for AI-assisted coding that builds unified prompts with static conte… | 66 | 1387 | active |
| lamm-mit/PDF2Audio A Gradio-based web application that converts PDF documents into audio podcasts, lectures, and summaries using OpenAI GPT models for text ge… | 30 | 1383 | active |
| milanglacier/minuet-ai.nvim Minuet AI is a Neovim plugin that provides AI-powered code completion as you type, using LLM providers such as OpenAI, Claude, Gemini, Code… | 88 | 1380 | active |
| aws-samples/generative-ai-use-cases Generative AI Use Cases (GenU) is a well-architected sample application from AWS that demonstrates business use cases for generative AI, bu… | 92 | 1379 | active |
| QwenAudio/ThinkSound ThinkSound is a PyTorch implementation of a NeurIPS 2025 framework that generates and edits audio from video, text, or audio inputs using C… | 52 | 1378 | active |
| 1rgs/jsonformer Jsonformer is a Python library that wraps Hugging Face causal language models to generate syntactically correct JSON conforming to a given … | 30 | 4934 | maintenance |
| Vizards/deepseek-v4-for-copilot A VS Code extension that adds DeepSeek V4 Flash, Pro, and Flash Vision Exp models to the GitHub Copilot Chat model picker. It lets develope… | 81 | 1377 | active |
| agentenv/agentflow AgentFlow is a Python library for orchestrating large numbers of LLM agents (Codex, Claude, Kimi, Pi) as programmatic dependency graphs wit… | 59 | 1377 | active |
| mpociot/chatgpt-vscode A Visual Studio Code extension that integrates ChatGPT into the editor, letting users ask questions, query selected code snippets, and inse… | 32 | 4925 | maintenance |
| Zefan-Cai/KVCache-Factory KVCache-Factory is a unified Python framework implementing many KV cache compression, retrieval, merging, and quantization methods for long… | 68 | 1375 | active |
| YuvDwi/Steve Steve is a Minecraft Forge mod that adds autonomous AI agents ('Cursor for Minecraft') that play the game alongside the player. Players giv… | 59 | 1375 | active |
| AtomicBot-ai/Atomic-Chat Atomic Chat is an open-source desktop and mobile application with a built-in inference engine for running open-weight LLMs fully locally an… | 81 | 1374 | active |
| Meituan-AutoML/MobileVLM MobileVLM is a family of compact vision language models (1.4B-3B parameters) designed to run efficiently on mobile devices, combining small… | 17 | 1370 | active |
| egoist/chatwise-releases ChatWise is a fast, privacy-friendly desktop AI chatbot client that connects to multiple LLM providers (OpenAI, Anthropic, Google, Groq, Ol… | 72 | 1369 | active |
| BICLab/SpikingBrain-7B SpikingBrain-7B is a brain-inspired large language model that combines hybrid efficient attention, MoE modules, and spike encoding, with a … | 54 | 1369 | active |
| ali-vilab/TeaCache TeaCache is a training-free caching approach that accelerates inference for video diffusion models by estimating output differences across … | 33 | 1369 | active |
| OpenPPL/ppl.nn PPLNN is a high-performance deep-learning inference engine written in C++ that runs ONNX models on x86 CPUs and NVIDIA GPUs, with a dedicat… | 32 | 1367 | active |
| xandergos/terrain-diffusion Terrain Diffusion is a Python framework that uses diffusion models as a learned, deterministic replacement for Perlin noise, generating inf… | 64 | 1365 | active |
| NVIDIA-AI-IOT/torch2trt torch2trt is a Python library that converts PyTorch models to TensorRT engines using the TensorRT Python API, with a simple single-function… | 23 | 4878 | maintenance |
| ThousandBirdsInc/chidori Chidori is an agent framework that runs async TypeScript agents on an embedded pure-Rust JavaScript engine, recording every side effect (LL… | 94 | 1363 | active |
| mishl-dev/Discord-AI-Chatbot A self-hosted Discord AI chatbot application written in Python that uses the Groq API for fast LLM responses. It supports slash and text co… | 10 | 1362 | active |
| ntegrals/10x 10x is an open-source terminal-based AI coding agent that chains multiple LLM models through multi-step 'Superpowers' workflows with smart … | 49 | 1360 | active |
| wrtnlabs/autobe AutoBE is an AI-powered backend builder that turns natural language requirements into complete NestJS/TypeScript backend applications with … | 70 | 1359 | active |
| OpenRaiser/NanoResearch NanoResearch is an autonomous AI research assistant that runs an end-to-end pipeline from a research idea to a complete LaTeX paper. It exe… | 62 | 1358 | active |
| elevenyellow/handcrafted-persona-engine Persona Engine is a Windows desktop application that drives a Live2D avatar with an AI pipeline: microphone speech recognition, an LLM guid… | 73 | 1357 | active |
| BillHuang2001/evogit EvoGit is a decentralized multi-agent framework that treats software development as an evolutionary process, deploying a population of inde… | 68 | 1357 | active |
| SakanaAI/ShinkaEvolve ShinkaEvolve is a Python framework that combines LLMs with evolutionary algorithms to automatically evolve and improve scientific code. It … | 79 | 1355 | active |
| felixrieseberg/clippy A desktop chat application that runs local LLMs (GGUF models via llama.cpp) behind a nostalgic 1990s Microsoft Office Clippy assistant inte… | 45 | 1351 | active |
| raznem/parsera Parsera is a lightweight Python library for scraping websites using LLMs, letting users define elements to extract with natural-language de… | 54 | 1350 | active |
| HazyResearch/minions Minions is a communication protocol and Python framework that lets small on-device LLMs collaborate with frontier cloud LLMs to cut cloud i… | 52 | 1350 | active |
| FreedomIntelligence/HuatuoGPT-o1 HuatuoGPT-o1 is a medical large language model project for complex medical reasoning, with open-sourced models (7B-72B), fine-tuning data, … | 22 | 1349 | active |
| vercel-labs/gemini-chatbot An open-source Next.js chatbot template powered by Google Gemini and the Vercel AI SDK, with streaming chat, generative UI, chat history pe… | 63 | 1348 | active |
| JoasASantos/NeuroSploit NeuroSploit is an AI-powered penetration testing framework written in Rust that turns a URL, repository, app, or host into an autonomous se… | 85 | 1347 | active |
| Kuingsmile/word-GPT-Plus Word GPT Plus is a Microsoft Word add-in that integrates LLMs and AI agents directly into Word for generating, translating, summarizing, an… | 84 | 1345 | active |
| jbilcke-hf/ai-comic-factory AI Comic Factory is a web application that generates comic panels and pages from a single text prompt by combining an LLM for story/dialogu… | 10 | 1345 | active |
| Abraxas-365/langchain-rust LangChain for Rust is the Rust language implementation of the LangChain framework for building LLM-powered applications through composabili… | 72 | 1344 | active |
| traycerai/traycer Traycer is an open-source desktop AI orchestration app that lets developers connect existing coding agents (Claude Code, Codex, Cursor, etc… | 91 | 1343 | active |
| cccyd2003-qwq/pinkbin Pinkbin is an open-source Windows disk analyzer and cleaner built with Tauri 2, React, and Rust. It scans entire drives in seconds via dire… | 61 | 1343 | active |
| ParisNeo/lollms-webui LoLLMs WebUI is a local, single-user web interface for running large language models and multimodal AI systems, supporting hundreds of mode… | 71 | 4785 | maintenance |
| hao-ai-lab/LookaheadDecoding A Python library implementing Lookahead Decoding, an exact parallel decoding algorithm that accelerates LLM inference without a draft model… | 31 | 1342 | active |
| louisho5/picobot Picobot is a lightweight, self-hosted AI agent distributed as a single ~9MB Go binary with persistent memory, tool calling, skills, and mes… | 62 | 1340 | active |
| elie222/rakazo Rakazo is an open-source, self-hosted platform for running persistent AI teammates (bots) that can use a browser, terminal, and desktop, re… | 66 | 1339 | active |
| google-gemma/gemma-translator An on-device, fully offline voice translator application powered by Gemma 4 via LiteRT-LM, with a React web frontend optimized for small ha… | 56 | 1337 | active |
| 1692775560/deepseek_project A collection of DeepSeek-related tools including a WeChat chatbot that uses the DeepSeek API for automated replies, a local document upload… | 65 | 1335 | active |
| MoonshotAI/Kimi-Dev Kimi-Dev-72B is an open-source coding LLM from Moonshot AI specialized in software engineering tasks like issue resolution and bug fixing, … | 38 | 1335 | active |
| Lapis0x0/obsidian-yolo YOLO is an agent-native AI assistant plugin for Obsidian that combines chat, writing assistance, knowledge base interaction, and agent orch… | 81 | 1330 | active |
| Nativ Nativ is a free, MIT-licensed macOS desktop application for running open AI models locally on Apple Silicon Macs, built on MLX-VLM. It prov… | 80 | 1330 | active |
| andimarafioti/faster-qwen3-tts A Python library for real-time text-to-speech inference with Qwen3-TTS using manual CUDA graph capture, requiring no Flash Attention, vLLM,… | 78 | 1330 | active |
| ServiceNow/BrowserGym BrowserGym is a Gym-style environment library for web task automation, providing a unified interface to web agent benchmarks like MiniWoB, … | 78 | 1329 | active |