function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| AmintaCCCP/GithubStarsManager An AI-powered desktop and web application for managing GitHub stars, forks, and Gists with semantic vector search, automatic categorization… | 85 | 3409 | active |
| pashpashpash/vault-ai OP Vault is a self-hosted application combining a Go backend with a React frontend that lets users upload documents (PDF, txt, epub) into a… | 43 | 3388 | active |
| i-am-bee/beeai-framework BeeAI Framework is an open-source, Linux Foundation-hosted framework for building production-grade multi-agent AI systems in Python and Typ… | 85 | 3385 | active |
| Intelligent-Internet/ii-agent II-Agent is an open-source (Apache-2.0) AI agent framework for building and deploying intelligent assistants that execute complex multi-ste… | 65 | 3381 | active |
| deepseek-ai/DeepSeek-OCR-2 DeepSeek-OCR 2 is an open-source vision-language model and inference toolkit implementing 'Visual Causal Flow' for optical character recogn… | 44 | 3379 | active |
| alibaba/ROLL ROLL is an open-source reinforcement learning library from Alibaba for training large language models at scale, supporting algorithms like … | 78 | 3374 | active |
| nv-tlabs/kimodo Kimodo is NVIDIA's official implementation of a kinematic motion diffusion model trained on 700 hours of motion capture data to generate hi… | 56 | 3365 | active |
| Lagrange-Labs/deep-prove DeepProve is a Rust framework for generating zero-knowledge proofs of neural network inference, including end-to-end proving of full LLM fo… | 60 | 3357 | active |
| xororz/local-dream A free, open-source Android app for running Stable Diffusion locally with Snapdragon NPU acceleration, also supporting CPU/GPU inference. I… | 82 | 3346 | active |
| OpenGVLab/Ask-Anything VideoChat/Ask-Anything is a family of multimodal chat models and demos that combine video understanding with large language models, letting… | 72 | 3346 | active |
| synthetic-sciences/openscience OpenScience is an open-source AI workbench for scientific research that runs the full research loop: literature review, hypothesis formatio… | 80 | 3342 | active |
| google-ai-edge/LiteRT LiteRT is Google's successor to TensorFlow Lite, an on-device runtime for high-performance ML and GenAI inference on edge platforms. It pro… | 85 | 3339 | active |
| QiuYannnn/Local-File-Organizer A privacy-focused desktop application that uses on-device AI models (Llama3.2 3B and Llava v1.6 via Nexa SDK) to scan, understand, and reor… | 13 | 3329 | active |
| Open-Less/openless OpenLess is an open-source cross-platform voice input application that lets users hold a hotkey, speak, and have AI-polished text inserted … | 81 | 3321 | active |
| johnbean393/Sidekick Sidekick is a native macOS app for chatting with local LLMs powered by a built-in llama.cpp backend, with RAG over local files, folders, an… | 69 | 3314 | active |
| RKNN-Toolkit2 RKNN-Toolkit2 is Rockchip's SDK for converting trained neural network models into RKNN format and deploying them on Rockchip NPU chips like… | 36 | 3313 | active |
| nicobailon/pi-subagents A Pi extension that lets the Pi coding agent delegate tasks to focused child subagents with async execution, truncation, artifacts, and ses… | 78 | 3311 | active |
| FireRedTeam/FireRed-OpenStoryline FireRed-OpenStoryline is an AI video editing agent that converts natural language instructions into directed video edits via LLM planning a… | 58 | 3310 | active |
| Mesh-LLM/mesh-llm Mesh LLM is a Rust-based distributed LLM inference runtime that pools GPUs and memory across machines into a single OpenAI-compatible API. … | 77 | 3305 | active |
| yaojingang/GEOFlow GEOFlow is an open-source content engineering and multi-site distribution system built for GEO (Generative Engine Optimization), combining … | 79 | 3303 | active |
| ResearAI/DeepScientist DeepScientist is a local-first autonomous research studio that runs the full scientific research loop on your machine, from baselines and e… | 74 | 3296 | active |
| gptscript-ai/gptscript GPTScript is a Go-based framework that lets LLMs operate and interact with systems such as local executables, OpenAPI endpoints, SDKs, and … | 10 | 3280 | active |
| zai-org/ChatGLM3 ChatGLM3 is a series of open bilingual (Chinese-English) conversational large language models from Zhipu AI and Tsinghua University, includ… | 28 | 13655 | maintenance |
| microsoft/lida LIDA is a Python library from Microsoft that uses large language models to automatically generate, execute, edit, explain, evaluate, and re… | 30 | 3273 | active |
| jehna/humanify A Rust CLI tool that unminifies and deobfuscates JavaScript by using LLMs (OpenAI, Anthropic, Gemini, OpenRouter, or local Ollama) to sugge… | 92 | 3272 | active |
| pezzolabs/pezzo Pezzo is an open-source, cloud-native LLMOps platform for managing the lifecycle of AI-powered features, including prompt design, versionin… | 64 | 3266 | active |
| ANative-Lab/EvoAgentX EvoAgentX is an open-source Python framework for building, evaluating, and automatically evolving LLM-based agents and agentic workflows. I… | 82 | 3261 | active |
| SawyerHood/draw-a-ui A Next.js web app that lets users draw UI wireframes on a tldraw canvas and generates HTML with Tailwind CSS from the sketch using the GPT-… | 42 | 13583 | maintenance |
| aandrew-me/tgpt tgpt is a cross-platform command-line tool written in Go that lets users interact with AI chatbots (GPT, Llama, Mixtral, etc.) directly fro… | 96 | 3252 | active |
| itsOwen/CyberScraper-2077 CyberScraper 2077 is an AI-powered web scraping application with a Streamlit GUI that uses OpenAI, Gemini, or local Ollama models to intell… | 67 | 3245 | active |
| nicedreamzapp/claude-code-local An MLX-native server for Apple Silicon that exposes an Anthropic-API-compatible endpoint so Claude Code and similar agents can run entirely… | 77 | 3240 | active |
| cita-777/metapi Metapi is a self-hosted meta-aggregation gateway that unifies multiple AI relay/proxy sites (New API, One API, OneHub, DoneHub, Veloera, et… | 72 | 3240 | active |
| KouriChat/KouriChat KouriChat is a Python-based emotional companionship program powered by LLMs that lets users chat with realistic AI personas. It integrates … | 51 | 3237 | active |
| langchain-ai/langgraphjs LangGraph.js is a TypeScript/JavaScript framework for building stateful, long-running LLM agents as graphs of nodes and edges, mixing deter… | 89 | 3236 | active |
| guaguaguaxia/weekly_report A web application that generates complete weekly work reports from brief descriptions of work content, powered by the OpenAI GPT-3.5 API. I… | 73 | 3234 | active |
| Jittor/jittor Jittor is a high-performance deep learning framework from Tsinghua University based on just-in-time (JIT) compilation and meta-operators, w… | 67 | 3229 | active |
| MisoLabsAI/MisoTTS Miso TTS 8B is an open-source text-to-speech model based on an RVQ Transformer architecture with a Llama 3.2-style 8B backbone, designed fo… | 52 | 3224 | active |
| alexrudall/ruby-openai A Ruby gem providing a client for the OpenAI API, supporting chat completions, the Responses API, streaming, vision, audio (Whisper), image… | 72 | 3222 | active |
| zai-org/GLM-4-Voice GLM-4-Voice is an end-to-end bilingual (Chinese/English) speech dialogue model from Zhipu AI, built on GLM-4-9B with a speech tokenizer and… | 22 | 3222 | active |
| op7418/NanoBanana-PPT-Skills An AI-powered tool that generates high-quality presentation slides as images using Google's Nano Banana Pro (Gemini image model), with opti… | 44 | 3218 | active |
| wendy7756/AI-Video-Transcriber An open-source AI tool that transcribes, summarizes, and archives videos and podcasts from 30+ platforms (YouTube, TikTok, Bilibili, etc.) … | 62 | 3215 | active |
| Josh-XT/AGiXT AGiXT is an open-source AI agent automation platform that orchestrates instruction management and complex task execution across multiple AI… | 87 | 3211 | active |
| crawlab-team/artipub ArtiPub is a self-hosted article publishing platform that automatically distributes content to multiple media channels such as CSDN and Seg… | 52 | 3204 | active |
| av/harbor Harbor is a CLI tool that spins up a complete, pre-wired local LLM stack with a single command, using Docker Compose to orchestrate hundred… | 85 | 3191 | active |
| google-antigravity/antigravity-sdk-python A Python SDK for building AI agents powered by Google Antigravity and Gemini, abstracting the agentic loop behind a stateful, secure infras… | 59 | 3185 | active |
| gcui-art/suno-api An unofficial open-source API wrapper for Suno.ai's AI music generation service, exposing endpoints for generating songs, lyrics, and audio… | 56 | 3184 | active |
| aipoch/open-science Open Science is an open-source, local-first, model-agnostic AI research workbench for reproducible scientific research. It combines a coord… | 80 | 3182 | active |
| imraywang/wewrite WeWrite is a Python-based AI agent skill that automates the full WeChat Official Account content pipeline: topic selection from trending ne… | 79 | 3182 | active |
| google-gemini/computer-use-preview A Python reference implementation of Gemini's computer-use agent that lets the model control a browser (via Playwright or Browserbase) to c… | 61 | 3181 | active |
| MiniMax-AI/MiniMax-M1 MiniMax-M1 is an open-weight, large-scale hybrid-attention reasoning language model released by MiniMax under Apache-2.0. The repository pr… | 32 | 3180 | active |
| CatchTheTornado/text-extract-api A self-hosted FastAPI-based API that converts PDFs, Office documents, and images into Markdown or structured JSON using OCR engines (EasyOC… | 45 | 3175 | active |
| nesaorg/nesa Nesa is a privacy-preserving AI inference platform that runs models like Llama, Mistral, and Stable Diffusion with end-to-end encryption us… | 24 | 3167 | active |
| qdrant/fastembed FastEmbed is a lightweight Python library for generating text embeddings using ONNX Runtime instead of PyTorch, requiring no GPU and minima… | 83 | 3166 | active |
| tekaratzas/RustGPT A transformer-based large language model implemented entirely in pure Rust with no external ML frameworks, using only ndarray for matrix op… | 38 | 3157 | active |
| HolmesGPT/holmesgpt HolmesGPT is an open-source AI agent (CNCF Sandbox project) that investigates production incidents and finds root causes by querying live o… | 87 | 3148 | active |
| BayLing-Models/BayLing-Speech LLaMA-Omni is an end-to-end speech interaction model built on Llama-3.1-8B-Instruct that generates simultaneous text and speech responses f… | 32 | 3146 | active |
| matthartman/ghost-pepper A free, open-source macOS menu bar app providing fully on-device speech-to-text dictation and meeting transcription using local Whisper, Pa… | 78 | 3140 | active |
| edison7009/EchoBird EchoBird is a cross-platform desktop application (Tauri + Rust) that provides one-click installation, launching, and model configuration fo… | 78 | 3130 | active |
| Letta Code Letta (formerly MemGPT) is an open-source platform for building stateful AI agents with persistent memory that learn and self-improve over … | 79 | 3125 | active |
| SuperCmdLabs/SuperCmd SuperCmd is an open-source macOS launcher combining Raycast-compatible extensions, hold-to-speak dictation, text-to-speech, AI chat and age… | 75 | 3124 | active |
| spinabot/brigade Brigade is a self-hosted, MIT-licensed multi-agent runtime where a crew of AI agents on an org chart delegate work, switch models mid-task,… | 76 | 3119 | active |
| cocktailpeanut/dalai Dalai is a self-hosted application that makes it simple to run LLaMA (and Alpaca) language models locally with a one-command installer and … | 21 | 12896 | maintenance |
| MiroFlow MiroFlow is a performance-first open-source agent framework for building, orchestrating, and benchmarking LLM-powered agents, with MiroThin… | 59 | 3103 | active |
| dirk1983/deepseek A lightweight PHP demo application that calls DeepSeek/OpenAI-compatible chat APIs with streaming (EventSource) output, supporting Markdown… | 36 | 3103 | active |
| AutoCodeRoverSG/auto-code-rover AutoCodeRover is an autonomous software engineering agent that combines LLMs with code structure analysis to automatically resolve GitHub i… | 34 | 3100 | active |
| SciSharp/BotSharp BotSharp is an open-source multi-agent AI framework written in C# for .NET, providing an agent abstraction layer, conversation state manage… | 79 | 3098 | active |
| HeyWillow/willow Willow is an open source, self-hosted voice assistant platform for ESP32-S3-BOX hardware, designed as a privacy-focused alternative to Amaz… | 86 | 3097 | active |
| ikawrakow/ik_llama.cpp A fork of llama.cpp offering state-of-the-art quantization types and improved CPU/CUDA inference performance for large language models. It … | 59 | 3091 | active |
| ridgerchu/matmulfreellm A Python implementation of MatMul-Free LM, a language model architecture that eliminates matrix multiplication operations using ternary wei… | 49 | 3089 | active |
| simonlin1212/TradingAgents-astock A Python multi-agent investment research framework for China's A-share market, deeply forked from TauricResearch/TradingAgents. It runs 7 s… | 80 | 3080 | active |
| cosmicstack-labs/mercury-agent Mercury Agent is a soul-driven AI assistant that runs 24/7 from the CLI, Telegram, Discord, Slack, Signal, or web, with 40+ permission-hard… | 80 | 3060 | active |
| tensorflow/tflite-micro TensorFlow Lite for Microcontrollers (TFLM) is a C++ port of TensorFlow Lite for running ML models on microcontrollers, DSPs, and other mem… | 77 | 3059 | active |
| sonos/tract Tract is Sonos' tiny, self-contained neural-network inference engine written in Rust. It loads ONNX, TensorFlow/TFLite, and NNEF models, op… | 99 | 3045 | active |
| b4rtaz/distributed-llama A C++ application that connects multiple home devices into a cluster for distributed LLM inference using tensor parallelism. It supports Ll… | 79 | 3045 | active |
| swirlai/swirl-search SWIRL is an open-source federated AI search and RAG platform that queries enterprise apps and data sources live, re-ranks results with a hy… | 92 | 3043 | active |
| x-dr/chatgptProxyAPI A self-deployable reverse proxy for the OpenAI API built on Cloudflare Workers and Cloudflare Pages, letting users replace api.openai.com w… | 10 | 3029 | active |
| ianarawjo/ChainForge ChainForge is an open-source visual programming environment for battle-testing prompts to LLMs, built on ReactFlow and Flask. It lets users… | 66 | 3027 | active |
| betalgo/openai A community .NET library (Betalgo.Ranul.OpenAI) for accessing OpenAI's API, including ChatGPT, DALL-E, Whisper, and Azure OpenAI. It lets .… | 67 | 3021 | active |
| containers/ramalama RamaLama is a Python CLI tool that simplifies serving and running AI/LLM models locally by leveraging OCI containers (Podman/Docker). It au… | 86 | 3018 | active |
| J-Space Cognition Suite A model-agnostic inference-time control suite packaged as an installable AI agent Skill (SKILL.md with modules, references, and Python veri… | 69 | 3018 | active |
| TheBlewish/Automated-AI-Web-Researcher-Ollama A Python application that turns a locally-run Ollama LLM into an automated web research assistant. Given a single query, it generates focus… | 21 | 3012 | active |
| allweonedev/presentation-ai An open-source, AI-powered presentation generator (a Gamma.app alternative) built with Next.js that creates professional, fully editable sl… | 58 | 3006 | active |
| off-grid-ai/OGAM Off Grid AI (OGAM) is a cross-platform mobile and desktop application that runs AI entirely on-device: GGUF LLM chat with vision, Whisper s… | 78 | 3002 | active |
| thu-pacman/chitu Chitu (赤兔) is a high-performance inference framework for large language models, positioned as a production-grade LLM serving engine. It sup… | 84 | 2995 | active |
| Dicklesworthstone/llm_aided_ocr A Python tool that converts scanned PDFs to text via Tesseract OCR, then uses LLMs (local or API-based like OpenAI/Anthropic) to correct OC… | 71 | 2993 | active |
| dw-dengwei/daily-arXiv-ai-enhanced A tool that automatically crawls daily arXiv papers and summarizes them using LLMs like DeepSeek, publishing the results as a browsable sit… | 64 | 2991 | active |
| pytorch/TensorRT Torch-TensorRT is a compiler library that accelerates PyTorch model inference on NVIDIA GPUs using TensorRT. It supports just-in-time compi… | 94 | 2986 | active |
| MiniMax-AI/Mini-Agent A minimal yet professional Python demo project showcasing best practices for building a single LLM agent on the MiniMax M2.5 model via an A… | 47 | 2984 | active |
| OpenMOSS/MOSS MOSS is an open-source tool-augmented conversational large language model from Fudan University, released with base models, SFT models, plu… | 68 | 12230 | maintenance |
| eli64s/readme-ai ReadmeAI is a Python CLI tool that automatically generates README.md files for any codebase using large language models. It supports multip… | 66 | 2979 | active |
| smol-ai/developer smol developer is a 'junior developer' AI agent that scaffolds an entire codebase from a product spec, and can be embedded as a library to … | 29 | 12186 | maintenance |
| togethercomputer/MoA Together AI's Mixture-of-Agents (MoA) is a Python implementation of a layered LLM architecture where multiple open-source models answer pro… | 24 | 2970 | active |
| li-plus/chatglm.cpp A pure C++ implementation of ChatGLM-6B, ChatGLM2-6B, ChatGLM3, and GLM-4(V) based on ggml, enabling memory-efficient real-time chatting on… | 20 | 2962 | active |
| pytorch/ao TorchAO is a PyTorch-native library for model optimization through quantization and sparsity. It supports quantizing weights, gradients, op… | 89 | 2957 | active |
| luminal-ai/luminal Luminal is a high-performance general-purpose ML inference compiler written in Rust that lowers models to a minimal 15-op dataflow IR and c… | 78 | 2956 | active |
| duolahypercho/codex-router A local model router that lets external LLM providers (Anthropic, Kimi, DeepSeek, xAI, GitHub Copilot, and others) be used inside the Codex… | 80 | 2953 | active |
| supabase-community/database-build An in-browser Postgres sandbox where each database runs locally via PGlite (WASM Postgres) and is paired with an LLM assistant for generati… | 62 | 2953 | active |
| leigest519/ScreenCoder ScreenCoder is a UI-to-code generation system that converts screenshots or design mockups into clean, editable HTML/CSS using a modular mul… | 62 | 2951 | active |
| ErlichLiu/DeepClaude DeepClaude is a self-hostable service that chains DeepSeek R1's reasoning output with Claude Sonnet or Gemini models to produce a combined … | 52 | 2936 | active |
| turboderp/exllama ExLlama is a standalone Python/C++/CUDA implementation of the Llama architecture optimized for running 4-bit GPTQ quantized weights on mode… | 29 | 2936 | active |