function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| Tencent/hpc-ops HPC-Ops is a production-grade C++/CUDA operator library for high-performance LLM inference, developed by Tencent's Hunyuan AI Infra team. I… | 59 | 1131 | active |
| auto-novel/auto-novel AutoNovel is a website application that automatically machine-translates light novels (web novels, bunko novels, and local files) using LLM… | 77 | 1130 | active |
| Dobiasd/frugally-deep frugally-deep is a lightweight header-only C++ library for running inference (forward passes) on Keras/TensorFlow models without linking ag… | 85 | 1128 | active |
| NativeMindBrowser/NativeMindExtension NativeMind is an open-source browser extension that provides a fully private, on-device AI assistant by connecting to local LLMs via Ollama… | 66 | 1128 | active |
| atiilla/GeoIntel GeoIntel is a Python tool that uses Google's Gemini API to estimate where a photo was taken through AI-powered geolocation analysis. It off… | 58 | 1128 | active |
| futantan/OpenGpt OpenGpt is an open-source web platform for instantly running and creating small ChatGPT-powered prompt applications, with a browsable marke… | 42 | 3897 | maintenance |
| mrphrazer/reverser_ai ReverserAI is a Binary Ninja plugin that provides automated reverse engineering assistance using locally-hosted large language models runni… | 68 | 1127 | active |
| EuniAI/Prometheus Prometheus is a knowledge-graph-driven AI agent platform that maps, understands, and repairs complex codebases through reasoning rather tha… | 66 | 1127 | active |
| K-Dense-AI/k-dense-byok K-Dense BYOK is a free, open-source desktop application providing 'Kady', an AI research assistant (co-scientist) that runs locally and use… | 78 | 1126 | active |
| espressif/esp-dl ESP-DL is Espressif's lightweight neural network inference framework for ESP-series chips, with a custom .espdl model format, quantization … | 72 | 1125 | active |
| m1guelpf/chatgpt-telegram A Go application that runs a Telegram bot letting users chat with OpenAI's ChatGPT from a single command or Docker container. It handles au… | 22 | 3886 | maintenance |
| Quorinex/Kiro-Go A Go service that converts Kiro accounts into OpenAI- and Anthropic-compatible API endpoints, with multi-account pooling, round-robin load … | 58 | 1124 | active |
| PromtEngineer/Verbi Verbi is a modular Python voice assistant application for experimenting with state-of-the-art transcription, LLM response generation, and t… | 48 | 1124 | active |
| joeseesun/qiaomu-mondo-poster-design An AI-powered poster and cover design tool that generates professional-grade posters, book covers, album covers, and social media graphics … | 47 | 1123 | active |
| AGenUI/AGenUI AGenUI is a high-performance A2UI SDK written in C++ that renders AI-generated, streaming user interfaces as native components on iOS, Andr… | 82 | 1122 | active |
| massgen/MassGen MassGen is an open-source multi-agent scaling system that runs in the terminal, orchestrating multiple frontier LLM agents to work on the s… | 72 | 1122 | active |
| mtymek/opencode-obsidian An Obsidian plugin that embeds the OpenCode AI assistant's web view directly into Obsidian's sidebar, launching it via Node.js child proces… | 60 | 1122 | active |
| plasma-umass/ChatDBG ChatDBG is an AI-assisted debugging tool that integrates large language models into standard debuggers (pdb, lldb, gdb) for C/C++/Python/Ru… | 71 | 1118 | active |
| vamplabAI/sgr-agent-core SGR Agent Core is an open-source Python framework for building research agents using Schema-Guided Reasoning, with an extensible BaseAgent … | 79 | 1117 | active |
| HarryR/z80ai Z80-μLM is a 2-bit quantized micro language model designed to run on 8-bit Z80 processors with only 64KB of RAM, exportable as ~40KB CP/M .… | 51 | 1117 | active |
| microsoft/onnxruntime-genai ONNX Runtime GenAI is a C++ library with Python, C#, C/C++, and Java APIs for running generative AI models (LLMs, Whisper, vision-language … | 93 | 1116 | active |
| axoviq-ai/synthadoc Synthadoc is an open-source, domain-agnostic LLM knowledge compilation engine that transforms raw documents into structured, human-readable… | 82 | 1116 | active |
| Doriandarko/make-it-heavy A Python framework that emulates Grok Heavy-style deep analysis by orchestrating multiple specialized AI agents in parallel via OpenRouter'… | 32 | 1116 | active |
| shroominic/codeinterpreter-api A Python library providing an open-source implementation of the ChatGPT Code Interpreter, built on LangChain. It lets LLM agents execute sa… | 19 | 3848 | maintenance |
| XYZ-AI-Lab/AxisAgentic AxisAgentic is an extensible Python runtime for long-horizon AI agents that handles multi-turn execution, tool orchestration, context manag… | 55 | 1114 | active |
| bevibing/tutor-skills A pair of Claude Code skills that convert documents (PDF, MD, HTML, EPUB) or source codebases into structured Obsidian study vaults with in… | 46 | 1114 | active |
| mrwadams/stride-gpt STRIDE GPT is an AI-powered threat modeling tool that uses LLMs to generate STRIDE-based threat models, attack trees, and MITRE ATT&CK/ATLA… | 84 | 1112 | active |
| index-labs/readpilot Read Pilot is a web application that analyzes online articles and generates Q&A cards from them using OpenAI models. It is built with Next.… | 63 | 1112 | active |
| KoljaB/RealtimeVoiceChat A Python client-server application enabling natural spoken conversations with LLMs, streaming browser audio via WebSockets through Realtime… | 33 | 3832 | maintenance |
| gabber-dev/gabber Gabber is an open-source engine for building real-time multimodal AI applications that can see, hear, and speak, using graph-based orchestr… | 44 | 1111 | active |
| jina-ai/discoart DiscoArt is a Python library that wraps Disco Diffusion (CLIP-guided diffusion) so generative artists and developers can create AI artworks… | 23 | 3826 | maintenance |
| yoheinakajima/babyagi BabyAGI is an experimental Python framework for building self-building autonomous agents, built around a 'functionz' system that stores, ma… | 59 | 22356 | experimental |
| TheAiSingularity/graphrag-local-ollama A local adaptation of Microsoft's GraphRAG that runs LLM and embedding extraction entirely through Ollama models like llama3, mistral, gemm… | 60 | 1108 | active |
| prometheus-eval/prometheus-eval Prometheus-Eval is a Python library for evaluating LLM generation outputs using the Prometheus family of open evaluator models and GPT-4 as… | 23 | 1107 | active |
| dp-archive/archive Skill Compose is an open-source agent builder and runtime platform for creating skill-powered AI agents, written in Python. It deliberately… | 60 | 1105 | active |
| fxy2311-youyou/expression-trainer A local desktop app (Electron) that trains spoken expression skills by combining fully offline real-time speech recognition (Sherpa-ONNX) w… | 55 | 1105 | active |
| alishobeiri/thread-notebook Thread is an AI-powered Jupyter Notebook alternative that integrates an AI copilot into the notebook editing experience. It runs locally an… | 61 | 1104 | active |
| canwhite/Krebs Krebs is a self-hosted AI gateway with a general-purpose agent focused on attention management. It provides multi-turn dialogue with tool e… | 60 | 1104 | active |
| KroMiose/nekro-agent NekroAgent is a cross-platform AI agent framework for multi-user chat scenarios that generates and executes code in sandboxed containers, w… | 90 | 1103 | active |
| FutureUniant/WorkShadow WorkShadow is a local-first desktop work journal app built with Tauri, React, and Rust, offering rich-text logging, keyword and semantic se… | 74 | 1103 | active |
| PharMolix/OpenBioMed OpenBioMed is an open-source toolkit and agent platform for biomedicine and life science, offering multimodal models like BioMedGPT-Mol, 45… | 72 | 1103 | active |
| workweave/router Workweave Router is a model router for agentic AI systems that routes each prompt to the most cost-effective model capable of handling it, … | 67 | 1103 | active |
| vifreefly/kimuraframework Kimuraframework (Kimurai) is a Ruby web scraping framework with an AI-assisted DSL: an LLM generates XPath selectors from a schema on first… | 61 | 1102 | active |
| Deep-ai-inc/ch.at A lightweight LLM chat service in Go, deployable as a single static binary, accessible via web, curl, SSH, DNS TXT queries, and an OpenAI-c… | 52 | 1102 | active |
| moltlaunch/cashclaw CashClaw is an autonomous AI agent that connects to the Moltlaunch onchain marketplace, where it evaluates tasks, quotes prices, executes w… | 47 | 1102 | active |
| pireel/pireel Pireel Studio is an open-source, browser-based AI video editor for talking heads, shorts, demos, and montages, with canvas/timeline editing… | 58 | 1101 | active |
| jackdark425/aigroupapp AI Group is an open-source Android mobile assistant app that integrates multiple large language models such as OpenAI, Claude, Gemini, ERNI… | 41 | 1101 | active |
| openresponses/openresponses Open Responses is an open-source specification for multi-provider, interoperable LLM interfaces modeled on the OpenAI Responses API, defini… | 57 | 1100 | active |
| cbh123/stickerbaker StickerBaker is an open-source web application that generates AI stickers from text prompts or face uploads, powered by Replicate models (f… | 25 | 1100 | active |
| vocodedev/vocode-core Vocode is an open-source Python library for building real-time, voice-based LLM applications and agents. It orchestrates streaming transcri… | 21 | 3785 | maintenance |
| agentlas-ai/Agentlas-OS Agentlas OS (with the Hephaestus engine) is an open-source agent framework that lets users create portable AI agents from plain-language re… | 77 | 1099 | active |
| zjunlp/LightMem LightMem is a lightweight Python framework for adding long-term memory to LLM applications and AI agents, providing memory storage, retriev… | 64 | 1099 | active |
| MrSibe/KnowNote KnowNote is a local-first, open-source desktop application built with Electron that serves as an alternative to Google NotebookLM. It lets … | 58 | 1099 | active |
| Gen-Verse/LatentMAS LatentMAS is a training-free multi-agent reasoning framework that lets LLM agents collaborate in continuous latent space by passing hidden-… | 56 | 1099 | active |
| xiaobright/dsh-anchored-standard A community plugin providing DeepSeek Harness (DSH) agent presets that bootstrap a session with a minimal tool schema and then promote to a… | 57 | 3775 | maintenance |
| HswAI2026/JuZhou-V1 JuZhou 1.0 is an ultra-lightweight 0.387B-parameter text-to-image foundation model designed for fully offline, on-device execution on mobil… | 54 | 1097 | active |
| paean-ai/deeptide DeepTide is an AI coding agent tuned for DeepSeek models, shipped as a Swift-native macOS app plus cross-platform terminal CLIs (TypeScript… | 58 | 1096 | active |
| boona13/image-extender An open-source Next.js web app for AI image outpainting that extends images in any direction using Gemini models via OpenRouter, with Poiss… | 52 | 1096 | active |
| synth-inc/onit Onit SideKick is an open-source macOS AI chat sidebar that docks to any application, letting users chat with LLMs using highlighted text or… | 82 | 1095 | active |
| YOOTeam/OpenPPT OpenPPT is a web-based online presentation (PPT) editor based on ChatPPT, supporting the full workflow of creating, importing, editing, bea… | 36 | 1095 | active |
| yandex/YaLM-100B YaLM-100B is a GPT-like pretrained language model with 100 billion parameters, trained by Yandex on English and Russian text using DeepSpee… | 32 | 3757 | maintenance |
| wilpel/caveman-compression A Python tool that compresses text for LLM contexts by stripping predictable grammar while preserving factual content, reducing token usage… | 41 | 1092 | active |
| welltop-cn/ComfyUI-TeaCache A ComfyUI plugin integrating TeaCache, a training-free caching method that accelerates diffusion model inference by exploiting output diffe… | 35 | 1092 | active |
| google-ai-edge/litert-torch LiteRT Torch is a Python library that converts PyTorch models into .tflite format for running with LiteRT on Android, iOS, and IoT devices.… | 87 | 1091 | active |
| flagos-ai/FlagGems FlagGems is a high-performance operator library for large language models written in the Triton language, providing backend-neutral GPU ker… | 89 | 1089 | active |
| hinthornw/trustcall A Python library that improves LLM tool calling reliability by having models generate JSON patch operations instead of full JSON blobs. It … | 38 | 1089 | active |
| yeahhe365/Gemini-Nexus Gemini Nexus is a Chrome extension (Manifest V3) that adds an AI assistant layer to the browser, integrating Gemini Web, the Gemini API, an… | 83 | 1088 | active |
| xuzhougeng/wisp-science Wisp Science is an open-source, local-first desktop AI research workbench that combines an LLM agent with persistent Python/R kernels, MCP-… | 81 | 1087 | active |
| amruth-sn/kong Kong is an LLM-orchestrated reverse engineering CLI that plugs AI into Ghidra's analysis engine to recover function names, types, and struc… | 63 | 1087 | active |
| lastmile-ai/aiconfig AIConfig is an open-source framework that manages generative AI prompts, models, and model parameters as JSON-serializable configs, separat… | 48 | 1087 | active |
| QwenLM/Qwen2.5-Math Qwen2.5-Math is a series of math-specialized large language models (1.5B/7B/72B base and instruct variants plus a 72B reward model) built o… | 23 | 1087 | active |
| rhymes-ai/Aria Aria is an open multimodal native Mixture-of-Experts (MoE) model with 25.3B total parameters (3.9B activated per token) and a 64K multimoda… | 23 | 1087 | active |
| nobodywho-ooo/nobodywho NobodyWho is an open-source (EUPL-1.2) on-device LLM inference engine written in Rust, built on llama.cpp, with SDKs for Kotlin, Swift, Pyt… | 86 | 1086 | active |
| HacxGPT-Official/HacxGPT-CLI HacxGPT-CLI is a Python command-line chat client for accessing uncensored, unrestricted language models, including HacxGPT's own paid API m… | 79 | 1086 | active |
| Uncensored Local Studio Uncensored-Local-Studio is a portable, fully offline AI assistant stack (Ollama + Llama 3 / other GGUF models + AnythingLLM) that runs enti… | 58 | 1086 | active |
| ndif-team/nnsight nnsight is a Python library for interpreting and intervening on the internals of deep learning models, built on PyTorch. It lets researcher… | 88 | 1085 | active |
| piotrostr/listen Listen is a Rust toolkit and framework for algorithmic trading on Solana, evolving into a framework for AI-driven crosschain portfolio mana… | 47 | 1085 | active |
| xataio/agent Xata Agent is an open-source AI agent that acts as a PostgreSQL expert, monitoring database logs and metrics, diagnosing root causes of iss… | 10 | 1085 | active |
| LiuMengxuan04/MiniCode MiniCode is a lightweight terminal-based AI coding assistant that implements a Claude Code-like agent loop with tool calling, diff review, … | 59 | 1084 | active |
| NVlabs/Fast-dLLM NVIDIA's official implementation of Fast-dLLM, a family of training-free and fine-tuning-based acceleration techniques for diffusion-based … | 57 | 1082 | active |
| BruceMacD/chatd Chatd is an Electron desktop application that lets you chat with your local documents using a local large language model (Mistral-7B) via O… | 18 | 1082 | active |
| GradientHQ/symphony-coord Symphony-Coord is a decentralized multi-agent framework that treats agent selection as an online multi-armed bandit problem, allowing roles… | 54 | 1080 | active |
| techjarves/Uncensored-Local-AI-Multiplatform A cross-platform Flutter application that runs uncensored open-source GGUF language models entirely on-device for Android, iOS, and desktop… | 67 | 1079 | active |
| spring-ai-alibaba/Lynxe Lynxe (formerly JManus) is a Java-based, code-free 'Prompt Programming' studio implementing a Manus-style multi-agent system with high exec… | 76 | 1077 | active |
| ikechan8370/chatgpt-plugin A ChatGPT/LLM chat plugin for Yunzai-Bot (Miao-Yunzai) QQ bots, built on the Chaite kernel. It provides multi-model channels, presets, grou… | 74 | 1077 | active |
| XInTheDark/raycast-g4f A Raycast extension that provides free access to GPT-4, Claude, Llama, Gemini and other AI models via GPT4Free providers, with full support… | 57 | 1076 | active |
| Pokee-AI/PokeeResearchOSS An open-source repository for Pokee's 7B-parameter DeepResearch agent that performs multi-turn web search and content reading to answer com… | 38 | 1076 | active |
| zubair-trabzada/ai-sales-team-claude A collection of 14 Claude Code skills and 5 parallel agents that turn Claude Code into an AI-powered sales team. It researches prospects, q… | 48 | 1075 | active |
| FalkorDB/QueryWeaver QueryWeaver is an open-source Text2SQL tool that converts plain-English questions into SQL queries using graph-powered schema understanding… | 85 | 1073 | active |
| Muesli-HQ/muesli Muesli is an open-source native macOS app that combines hotkey-driven AI dictation with local meeting transcription, running speech-to-text… | 82 | 1072 | active |
| mit-han-lab/streaming-vlm StreamingVLM is a vision-language model framework from MIT Han Lab for real-time understanding of effectively infinite video streams. It ma… | 38 | 1072 | active |
| mskayyali/nodepad nodepad is a spatial note-taking web application where notes are placed on a canvas and AI quietly classifies them, infers connections, and… | 56 | 1071 | active |
| sipeed/TinyMaix TinyMaix is a tiny neural network inference library for microcontrollers (TinyML), with core code under 400 lines and a .text section under… | 34 | 1071 | active |
| rockbenben/subtitle-translator A free, browser-based batch subtitle translation tool for .srt, .ass, .vtt, and .lrc files that strips timing locally and sends only dialog… | 83 | 1070 | active |
| tryAGI/LangChain LangChain .NET is a C#/.NET implementation of the LangChain framework for building LLM-powered applications through composability. It provi… | 65 | 1070 | active |
| codeacme17/examor Examor is a self-hosted web application that generates exams and review questions from your knowledge notes using LLMs like GPT-4 and Claud… | 31 | 1070 | active |
| dynamiq-ai/dynamiq Dynamiq is an open-source Python orchestration framework for building agentic AI and LLM applications, covering agents, RAG pipelines, work… | 85 | 1066 | active |
| Dataherald/dataherald Dataherald is an open-source natural language-to-SQL engine for enterprise question answering over relational databases, exposing an API th… | 19 | 3646 | maintenance |
| metaskills/experts Experts.js is a JavaScript library that simplifies creating and deploying OpenAI Assistants via the Assistants API, hiding Run object manag… | 24 | 1065 | active |
| sybil-solutions/codex-shim A local Python/aiohttp server that exposes an OpenAI Responses-compatible endpoint on loopback, letting Codex Desktop use any BYOK model (O… | 58 | 1064 | active |