function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| patterns-ai-core/langchainrb Langchain.rb is a Ruby library for building LLM-powered applications, providing a unified interface to many LLM providers (OpenAI, Anthropi… | 76 | 1991 | active |
| lightseekorg/tokenspeed TokenSpeed is a high-performance LLM inference engine designed for agentic workloads, aiming for TensorRT-LLM-level performance with vLLM-l… | 68 | 1989 | active |
| NVIDIA/Stable-Diffusion-WebUI-TensorRT An NVIDIA extension for the Stable Diffusion Web UI (Automatic1111) that accelerates image generation using TensorRT-optimized engines on R… | 18 | 1989 | active |
| 777genius/agent-teams-ai A free Electron desktop app for orchestrating teams of AI agents that handle tasks autonomously, message each other, and review each other'… | 80 | 1986 | active |
| chaidiscovery/chai-lab Chai-1 is a state-of-the-art multi-modal foundation model for biomolecular structure prediction, handling proteins, small molecules, DNA, R… | 65 | 1986 | active |
| coder/xum Xum (formerly Mux) is an open-source desktop application for running multiple AI coding agents in parallel, each in its own isolated worksp… | 79 | 1985 | active |
| antirez/iris.c Iris is a pure C inference pipeline that generates images from text prompts using open-weights diffusion transformer models like FLUX.2 Kle… | 46 | 1983 | active |
| nhaouari/obsidian-textgenerator-plugin An open-source Obsidian plugin that brings generative AI text generation into the note-taking workflow, supporting providers like OpenAI, A… | 92 | 1981 | active |
| openlm-research/open_llama OpenLLaMA is a permissively licensed (Apache-2.0) open reproduction of Meta AI's LLaMA, releasing 3B, 7B, and 13B pretrained model weights … | 30 | 7528 | maintenance |
| openakita/openakita OpenAkita is an open-source multi-agent AI assistant framework in Python that orchestrates multiple cooperating agents with skills, plugins… | 76 | 1973 | active |
| alexgreensh/token-optimizer A Claude Code plugin/skill that finds and removes wasted 'ghost tokens' in agent context windows, helping sessions survive compaction witho… | 78 | 1971 | active |
| FireRedTeam/FireRedASR FireRedASR is a family of open-source industrial-grade automatic speech recognition models supporting Mandarin, Chinese dialects, and Engli… | 52 | 1971 | active |
| bigcode-project/starcoder StarCoder is a 15B-parameter code language model trained on 80+ programming languages, and this repository hosts the fine-tuning and infere… | 30 | 7506 | maintenance |
| run-llama/notebookllama NotebookLlaMa is an open-source, Python-based alternative to Google's NotebookLM, backed by LlamaCloud for document ingestion and retrieval… | 52 | 1967 | active |
| StanfordBDHG/HealthGPT HealthGPT is an experimental open-source iOS app built on the Stanford Spezi framework that lets users query their Apple Health data using … | 83 | 1964 | active |
| siliconflow/onediff OneDiff is an out-of-the-box acceleration library for diffusion models, providing PyTorch compilation tools and optimized GPU kernels. It i… | 48 | 1964 | active |
| SaiAkhil066/CORTEX-AI-SUPER-RAG CORTEX RAG is a local-first, agentic retrieval-augmented generation application that lets users upload PDFs and ask questions with cited an… | 61 | 1962 | active |
| j4wg/interview-coder-withoupaywall-opensource A free, open-source desktop application (CodeInterviewAssist) that provides AI-powered coding interview assistance, including problem analy… | 38 | 1961 | active |
| OpenMotionLab/MotionGPT MotionGPT is a unified motion-language model that treats 3D human motion as a foreign language by converting motion into discrete motion to… | 32 | 1961 | active |
| showlab/computer_use_ootb An out-of-the-box desktop GUI agent that lets vision-language models like Claude 3.5 Computer Use, ShowUI, and UI-TARS control Windows and … | 32 | 1960 | active |
| huggingface/chat-macOS HuggingChat macOS is a native Swift chat application for macOS that provides access to open-source AI chat models from Hugging Face. It off… | 10 | 1959 | active |
| firecrawl/fireplexity Fireplexity is an open-source, Perplexity-style AI search engine built with TypeScript that delivers answers with real-time citations, stre… | 36 | 1958 | active |
| Yuliang-Liu/Monkey Monkey is a large multi-modal model (LMM) research project from CVPR 2024 that improves image understanding via higher input resolution and… | 65 | 1951 | active |
| langgraph4j/langgraph4j LangGraph4j is a Java library for building stateful, multi-agent applications with LLMs, inspired by the Python LangGraph project. It lets … | 86 | 1947 | active |
| rynfar/meridian Meridian is a proxy that bridges Anthropic's Claude Agent SDK to the standard Anthropic API format, letting a Claude Max subscription power… | 78 | 1941 | active |
| akdeb/ElatoAI ElatoAI is an open-source platform for running realtime voice AI conversations on ESP32/Arduino hardware, supporting 100+ STT, LLM, and TTS… | 64 | 1933 | active |
| LingyiChen-AI/JadeAI JadeAI is an open-source, AI-powered resume builder with drag-and-drop editing, 50+ professional templates, and real-time AI optimization i… | 82 | 1931 | active |
| Tencent-Hunyuan/HunyuanOCR HunyuanOCR-1.5 is a lightweight end-to-end OCR vision-language model from Tencent, with a unified inference environment, llama.cpp PC-side … | 59 | 1930 | active |
| carlrobertoh/ProxyAI ProxyAI is an open-source AI coding copilot plugin for JetBrains IDEs like IntelliJ and PyCharm. It connects to models from OpenAI, Anthrop… | 80 | 1928 | active |
| RightNow-AI/picolm PicoLM is a pure C11 LLM inference engine that runs 1-billion parameter models in GGUF format on extremely constrained hardware like $10 bo… | 46 | 1921 | active |
| yomorun/yomo YoMo is an open-source serverless LLM function calling framework for building fast, geo-distributed AI agents, written in Rust with a QUIC-… | 98 | 1920 | active |
| kevinhermawan/Ollamac Ollamac is a native macOS application that provides a chat interface for interacting with local Ollama models. It supports all Ollama model… | 34 | 1913 | active |
| deepseek-ai/DeepSeek-LLM DeepSeek LLM is a family of open-source large language models (7B and 67B, Base and Chat variants) trained from scratch on 2 trillion token… | 27 | 7257 | maintenance |
| rosemarycox5334-debug/PA_Agent A desktop AI-assisted Price Action analysis tool for discretionary traders. It ingests candlestick data from MT5, TradingView, yfinance, or… | 58 | 1912 | active |
| alexpinel/Dot Dot is a standalone Electron desktop application for chatting with your documents using fully local LLMs and Retrieval Augmented Generation… | 16 | 1911 | active |
| arcsin1/oh-my-ppt Oh My PPT is a local-first, open-source desktop application (Electron + React + TypeScript) that uses AI to generate editable HTML-based pr… | 81 | 1907 | active |
| NVIDIA-NeMo/labs-OO-Agents NVIDIA Object Oriented Agents (NOOA) is a model-agnostic Python framework for building AI agents where state, tools, prompts, and typed int… | 79 | 1906 | active |
| veasion/AiPPT AiPPT (文多多) is a commercial-grade AI PowerPoint generation project that creates presentations from topics, files, or URLs, and can parse PP… | 49 | 1906 | active |
| melih-unsal/DemoGPT DemoGPT is a Python framework and autonomous agent that generates LangChain-based LLM agent applications from natural-language prompts, bun… | 53 | 1905 | active |
| bhaskatripathi/pdfGPT pdfGPT is an open-source Python application that lets users chat with the contents of PDF files using GPT capabilities. It implements a sim… | 62 | 7167 | maintenance |
| ivanfioravanti/chatbot-ollama An open-source web-based chat UI for Ollama, forked from chatbot-ui, that lets users chat with locally hosted LLMs. It supports document (P… | 49 | 1891 | active |
| CaviraOSS/PageLM PageLM is an open-source, AI-powered education platform inspired by NotebookLM that transforms study materials like PDFs and notes into int… | 62 | 1884 | active |
| kyuz0/amd-strix-halo-toolboxes A collection of pre-built Toolbx/Docker containers ('toolboxes') for running llama.cpp LLM inference on AMD Ryzen AI Max 'Strix Halo' integ… | 62 | 1881 | active |
| IPADS-SAI/MobiAgent MobiAgent is a systematic framework for building customizable GUI agents that operate mobile phones, comprising the MobiMind agent model fa… | 60 | 1880 | active |
| rednote-machine-learning/RedKnot RedKnot is a long-context LLM inference acceleration library built on SGLang, using head-classified KV reuse, offline KV storage with RoPE … | 58 | 1880 | active |
| SqueezeAILab/LLMCompiler LLMCompiler is a Python framework from an ICML 2024 paper that orchestrates parallel function calling with LLMs. It uses a planner, task fe… | 26 | 1880 | active |
| jiujiu532/grok2api A self-hosted Python/FastAPI gateway that exposes grok.com and console.x.ai chat, image, and video capabilities through OpenAI- and Anthrop… | 55 | 1878 | active |
| THUDM/LongWriter LongWriter is a research project from Tsinghua's THUDM group providing fine-tuned LLMs (based on GLM-4-9B and Llama-3.1-8B) plus training a… | 35 | 1875 | active |
| nndeploy/nndeploy nndeploy is an easy-to-use, high-performance AI deployment framework written in C++ with Python bindings. It provides a visual drag-and-dro… | 87 | 1868 | active |
| NVIDIA-AI-IOT/Lidar_AI_Solution NVIDIA's collection of GPU-accelerated Lidar AI inference solutions for autonomous driving, including optimized implementations of PointPil… | 72 | 1867 | active |
| MixLabPro/comfyui-mixlab-nodes A ComfyUI custom-nodes plugin that adds workflow-to-web-app conversion, screen sharing, floating video, GPT/LLM integration, 3D, speech rec… | 67 | 1863 | active |
| SMNETSTUDIO/WeChat-AI A self-hosted WeChat roleplay chatbot service written in TypeScript that connects directly to Tencent iLink, stores data in remote Redis, a… | 68 | 1857 | active |
| webpilot-ai/Webpilot Webpilot is a free, open-source browser extension that acts as a 'Copilot for the web', letting users converse with web page content withou… | 75 | 1855 | active |
| Kav-K/GPTDiscord GPTDiscord is a self-hosted Discord bot providing an all-in-one GPT interface with ChatGPT-style conversations, DALL-E image generation, AI… | 62 | 1855 | active |
| zakirkun/guardian-cli Guardian is a Python CLI tool that automates penetration testing workflows using LLM providers (OpenAI, Claude, Gemini, Ollama, and others)… | 69 | 1853 | active |
| Tele-AI/Telechat TeleChat is a family of open-source bilingual (Chinese-English) large language models (1B, 7B, 12B) developed by China Telecom's AI team, r… | 67 | 1853 | active |
| filiksyos/gitreverse GitReverse is a web application that turns any public GitHub repository into a single synthetic prompt that could have been used to vibe-co… | 59 | 1851 | active |
| AutoFigure AutoFigure-Edit is a Python application that converts scientific paper method sections into fully editable SVG figures using large language… | 56 | 1845 | active |
| dphnAI/sonar Sonar is a large-scale LLM inference engine for Hugging Face-compatible language and multimodal models, based on vLLM (formerly the Aphrodi… | 94 | 1843 | active |
| mlc-ai/xgrammar XGrammar is an open-source library for efficient, flexible, and portable structured generation in LLM inference via constrained decoding. I… | 88 | 1843 | active |
| YangLing0818/RPG-DiffusionMaster Official implementation of RPG (Recaption, Plan, Generate), a training-free framework that uses multimodal LLMs as prompt recaptioners and … | 28 | 1842 | active |
| moorcheh-ai/memanto Memanto is a companion 'memory agent' that manages long-term semantic memory for AI agents — deciding what to keep, resolving contradiction… | 81 | 1841 | active |
| RanFeng/clipsketch-ai ClipSketch AI is a web-based AI content creation workbench that imports videos from Bilibili and Xiaohongshu links, lets users frame-accura… | 43 | 1840 | active |
| aallam/openai-kotlin A Kotlin client library for the OpenAI API built on Kotlin multiplatform and coroutines, using Ktor as the HTTP engine. It provides typed a… | 63 | 1838 | active |
| MaximeRivest/riddle A diary-style application for the reMarkable Paper Pro e-ink tablet that turns handwritten pages into conversations with an LLM. You write … | 74 | 1837 | active |
| handy-computer/transcribe.cpp A C/C++ speech-to-text inference library built on the ggml runtime that runs 16+ ASR model families (Whisper, Parakeet, Canary, Moonshine, … | 81 | 1836 | active |
| abinthomasonline/repo2txt A browser-based tool that converts GitHub, GitLab, Azure DevOps repositories, local folders, or ZIP files into a single formatted text file… | 65 | 1835 | active |
| dauparas/ProteinMPNN ProteinMPNN is a PyTorch-based tool that designs amino acid sequences for given protein backbone structures using a message-passing neural … | 23 | 1834 | stable |
| sunmh207/AI-Codereview-Gitlab A self-hosted automated code review application that uses large language models (DeepSeek, OpenAI, Anthropic, ZhipuAI, Qwen, Ollama) to rev… | 83 | 1826 | active |
| agentjido/jido Jido is an autonomous agent framework for Elixir that runs agents as supervised BEAM processes with fault tolerance, signal-based coordinat… | 83 | 1822 | active |
| GCWing/BitFun BitFun is a cross-platform desktop AI agent application with a high-performance Rust agent runtime that writes code, produces documents, an… | 82 | 1817 | active |
| jacoblee93/fully-local-pdf-chatbot A Next.js web application that lets users chat with uploaded PDF documents entirely locally, performing chunking, embedding, vector storage… | 52 | 1817 | active |
| potamides/DeTikZify DeTikZify is a Python library and research tool that uses multimodal large language models to synthesize TikZ/LaTeX graphics programs from … | 49 | 1817 | active |
| LingyiChen-AI/AIComicBuilder AI Comic Builder is a self-hosted Next.js web application that turns scripts into fully animated comic videos through an automated AI pipel… | 66 | 1814 | active |
| ROCm/FastFlowLM FastFlowLM (FLM) is an NPU-first LLM inference runtime purpose-built and deeply optimized for AMD Ryzen AI NPUs (XDNA2), offering an Ollama… | 85 | 1809 | active |
| jd-opensource/JoyAI-VL-Interaction JoyAI-VL-Interaction is an open 8B-scale vision-language interaction model with a complete deployable real-time streaming system, including… | 58 | 1806 | active |
| Robbyant/lingbot-va LingBot-VA is an autoregressive diffusion framework that unifies video world modeling and robot policy learning in a single interleaved vid… | 56 | 1806 | active |
| SunoAI-API/Suno-API An unofficial Python/FastAPI-based API wrapper for Suno AI that lets you generate songs and lyrics programmatically. It includes automatic … | 33 | 1806 | active |
| GordenSun/GordenSuperPPTSkills A set of Codex agent skills that use GPT image generation and vision to create image-format PPT slides and convert them into fully editable… | 52 | 1805 | active |
| undertheseanlp/underthesea Underthesea is an open-source Python toolkit for Vietnamese natural language processing that has evolved into an agentic AI toolkit with mu… | 94 | 1804 | active |
| evalplus/evalplus EvalPlus is a Python framework and benchmark suite for rigorously evaluating LLM-synthesized code, providing HumanEval+ and MBPP+ datasets … | 41 | 1804 | active |
| granawkins/rawdog Rawdog is a CLI assistant that responds to natural-language prompts by generating and auto-executing Python scripts. It uses a recursive se… | 33 | 1804 | active |
| APIParkLab/APIPark APIPark is an open-source, cloud-native AI gateway and API developer portal for managing, integrating, and deploying AI and REST APIs. It u… | 66 | 1803 | active |
| Xilinx/Vitis-AI AMD Vitis AI is an integrated development environment and stack for accelerating AI inference on AMD/Xilinx adaptable platforms, including … | 71 | 1800 | active |
| generative-computing/mellea Mellea is a Python library for writing generative programs, where LLM calls are first-class operations with type-annotated outputs, verifia… | 83 | 1799 | active |
| JHubi1/ollama-app A modern, easy-to-use Flutter client app for Ollama that connects to an Ollama server over the local network to chat with local LLMs. It ru… | 58 | 1799 | active |
| inference-labs-inc/inference-network Inference Network is a Proof of Inference system built on EigenLayer that provides trustless verification of AI inference results via Solid… | 49 | 1799 | active |
| zai-org/CogVLM CogVLM is an open-source visual language model (17B) combining a vision encoder with a pretrained language model for image understanding an… | 28 | 6744 | maintenance |
| AI-Citizen/SolidGPT SolidGPT is an AI-powered semantic search assistant for developers that lets you chat with your codebase and Notion workspace. It ships as … | 19 | 1796 | active |
| OS-Copilot/OS-Copilot OS-Copilot is an open-source Python library for building generalist AI agents that interface with operating system elements like the web, t… | 16 | 1796 | active |
| QwenLM/Qwen-VL Official repository for Qwen-VL, Alibaba Cloud's large vision-language model family, including the pretrained Qwen-VL and instruction-tuned… | 28 | 6726 | maintenance |
| hitsz-ids/airda airda (Air Data Agent) is a Python-based multi-agent system for data analysis that understands natural language data requirements and gener… | 17 | 1785 | active |
| neuml/paperai paperai is an AI application for medical and scientific papers that runs bulk LLM inference and RAG pipelines over article repositories to … | 67 | 1779 | active |
| justlovemaki/CloudFlare-AI-Insight-Daily A Cloudflare Workers-based content aggregation and generation platform that daily curates AI industry news, trending open-source projects, … | 63 | 1779 | active |
| Emu Series Emu3 is a suite of state-of-the-art multimodal models from BAAI trained solely with next-token prediction, tokenizing images, text, and vid… | 57 | 1778 | active |
| aieditor-team/AiEditor AiEditor is an open-source, AI-powered rich text WYSIWYG editor built on Web Components (ProseMirror/Tiptap-based) that integrates with any… | 61 | 1774 | active |
| Nutlope/twitterbio A Next.js web application that generates Twitter/X bios using AI models (Qwen 3.5 9B and GPT OSS 20B) served via the Together AI API with s… | 71 | 1770 | active |
| vercel-labs/coding-agent-template A deployable Next.js web application template for building multi-agent AI coding platforms that run coding agents like Claude Code, Codex C… | 62 | 1769 | active |
| signerlabs/Klee Klee is a native macOS AI chat application that runs large language models entirely on-device using Apple's MLX framework on Apple Silicon.… | 56 | 1762 | active |
| AIdea AIdea is a fully open-source cross-platform mobile and desktop app built with Flutter that integrates mainstream large language models (GPT… | 51 | 1762 | active |