function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| karakeep-app/karakeep Karakeep (formerly Hoarder) is a self-hostable 'bookmark everything' app for saving links, notes, images, and PDFs with AI-based automatic … | 92 | 28617 | active |
| langchain-ai/deepagents Deep Agents is an open-source, batteries-included agent harness from LangChain built on LangGraph, providing an opinionated agent with file… | 81 | 28559 | active |
| firecrawl/open-lovable Open Lovable is an AI-powered web application that lets users chat with an LLM to clone and recreate any website as a modern React app. It … | 42 | 28327 | active |
| google-research/timesfm TimesFM is a pretrained decoder-only foundation model from Google Research for time-series forecasting, offering zero-shot forecasts with u… | 81 | 28273 | active |
| srbhr/Resume-Matcher Resume Matcher is an open-source, self-hosted AI application that tailors resumes and cover letters to specific job descriptions using LLMs… | 86 | 28254 | active |
| MLX MLX is an array computation framework for machine learning on Apple silicon, developed by Apple ML research. It offers NumPy-like Python AP… | 94 | 28172 | active |
| p-e-w/heretic Heretic is a command-line tool that automatically removes censorship (safety alignment) from transformer-based language models using direct… | 76 | 28157 | active |
| yamadashy/repomix Repomix is a CLI tool (and web service) that packs an entire repository into a single AI-friendly file in XML, Markdown, or plain text form… | 85 | 28071 | active |
| OtterMind/Chat2DB Chat2DB is a free, cross-platform, local-first database client and SQL workspace supporting 40+ databases including MySQL, PostgreSQL, Orac… | 95 | 28038 | active |
| charmbracelet/crush Crush is a terminal-based agentic coding assistant from Charm that wires your codebase, tools, and workflows into the LLM of your choice. I… | 82 | 27713 | active |
| can1357/oh-my-pi oh-my-pi (omp) is a terminal-based AI coding agent with IDE integrations wired in, built as a fork of Pi with a Rust core and TypeScript/Bu… | 78 | 27695 | active |
| virattt/dexter Dexter is an autonomous AI agent built in TypeScript that performs deep financial research by planning tasks, executing them with real-time… | 81 | 27546 | active |
| Anil-matcha/Open-Generative-AI A self-hosted, MIT-licensed open-source web studio for AI image and video generation, offering access to 500+ models (Flux, Midjourney, Kli… | 92 | 27200 | active |
| Kilo-Org/kilocode Kilo Code is an open-source (MIT) AI coding agent available as a VS Code extension, JetBrains plugin, and CLI, with cloud agents and a mode… | 83 | 27033 | active |
| Fosowl/agenticSeek AgenticSeek is a fully local, privacy-focused AI assistant that serves as an open-source alternative to Manus AI. It runs reasoning models … | 64 | 27027 | active |
| decolua/9router 9Router is a self-hosted AI gateway/router that connects AI coding tools like Claude Code, Cursor, Codex, and Cline to 60+ LLM providers th… | 75 | 26418 | active |
| ahujasid/blender-mcp An MCP server and Blender addon that lets any LLM (Claude, ChatGPT, Gemini, Cursor, or local models via Ollama) control Blender 3D through … | 64 | 26346 | active |
| JustVugg/colibri Colibrì is a pure-C, zero-dependency inference engine that runs frontier Mixture-of-Experts models (744B to 2.8T parameters) on consumer ha… | 80 | 26251 | active |
| OpenBMB/MiniCPM-V MiniCPM-V and MiniCPM-o are a series of small multimodal large language models for efficient image, video, and audio understanding, deploya… | 61 | 26240 | active |
| black-forest-labs/flux Official minimal inference code from Black Forest Labs for running FLUX.1 open-weight image generation and editing models. It supports text… | 38 | 25912 | active |
| virgiliojr94/book-to-skill A Python CLI tool that converts technical books and document collections (PDF, EPUB, DOCX, MD, HTML, etc.) into agent skills for AI coding … | 80 | 25880 | active |
| mozilla-ai/llamafile llamafile is a Mozilla project that packages llama.cpp (and whisper.cpp for whisperfile) with Cosmopolitan Libc into single-file executable… | 94 | 25694 | active |
| toon-format/toon TOON (Token-Oriented Object Notation) is a compact, human-readable encoding of the JSON data model designed to reduce token usage in LLM pr… | 83 | 25261 | active |
| microsoft/JARVIS JARVIS (HuggingGPT) is a system that uses an LLM as a controller to plan tasks and orchestrate numerous expert ML models from the HuggingFa… | 45 | 25208 | active |
| SYSTRAN/faster-whisper A fast reimplementation of OpenAI's Whisper speech-to-text model built on the CTranslate2 inference engine, offering up to 4x speedup and l… | 57 | 25104 | active |
| haotian-liu/LLaVA LLaVA (Large Language and Vision Assistant) is an open-source multimodal large language model framework implementing visual instruction tun… | 20 | 25000 | active |
| Max-Eee/NeoPass NeoPass is a Chrome extension that bypasses proctoring restrictions on exam platforms like the Iamneo portal, Examly, NeoExamShield, and NP… | 80 | 24574 | active |
| baidu/Unlimited-OCR Baidu's Unlimited-OCR is an open vision-language OCR model for one-shot long-horizon document parsing, extending DeepSeek-OCR. It provides … | 56 | 24569 | active |
| google-ai-edge/gallery Google AI Edge Gallery is a mobile and desktop application for running open-source large language models like Gemma entirely on-device, off… | 86 | 24565 | active |
| deepseek-ai/DeepSeek-Coder DeepSeek Coder is a series of open code language models (1B to 33B) trained from scratch on 2T tokens of code and natural language, with a … | 50 | 24217 | active |
| titanwings/distilly Distilly is a Python tool that distills a person's experience, judgment, voice, and working style from messages, documents, interviews, and… | 63 | 24016 | active |
| deepseek-ai/DeepSeek-OCR DeepSeek-OCR is an open vision-language model from DeepSeek AI that researches 'contexts optical compression' - encoding long text contexts… | 45 | 23855 | active |
| vanna-ai/vanna Vanna is an open-source Python framework for building user-aware AI agents that translate natural-language questions into SQL queries, char… | 10 | 23818 | active |
| sinaptik-ai/pandas-ai PandasAI is a Python library that lets users query data in SQL, CSV, or parquet format using natural language, powered by LLMs and RAG. It … | 57 | 23772 | active |
| Tencent/ncnn ncnn is a high-performance neural network inference framework written in C++ and optimized for mobile, embedded, and desktop deployment. It… | 86 | 23753 | stable |
| mlc-ai/mlc-llm MLC LLM is a machine learning compiler and high-performance deployment engine for large language models, built on TVM-based ML compilation.… | 64 | 23095 | active |
| HKUDS/RAG-Anything RAG-Anything is an all-in-one Python framework for multimodal Retrieval-Augmented Generation, built on LightRAG. It treats text, images, ta… | 80 | 23081 | active |
| wandb/openui OpenUI is an open-source web application from Weights & Biases that generates UI components from natural language descriptions, rendering t… | 70 | 22516 | active |
| NVIDIA/NemoClaw NVIDIA NemoClaw is an open-source reference stack for running AI coding agents such as OpenClaw, Hermes, and LangChain Deep Agents Code sec… | 59 | 22287 | active |
| PromtEngineer/localGPT LocalGPT is a fully private, on-premise document intelligence platform that lets users chat with, summarize, and query their own documents … | 74 | 22205 | active |
| antirez/ds4 DwarfStar (ds4) is a self-contained native LLM inference engine written in C, optimized for DeepSeek V4 Flash and PRO and GLM 5.2, with sup… | 58 | 21807 | active |
| guidance-ai/guidance Guidance is a Python library providing an efficient programming paradigm for steering large language models, letting you interleave control… | 79 | 21714 | active |
| microsoft/onnxruntime ONNX Runtime is a cross-platform, high-performance machine-learning accelerator for running inference and training on ONNX models. It suppo… | 99 | 21654 | stable |
| HKUDS/AI-Trader AI-Trader is an agent-native trading platform where autonomous AI agents register, trade, and compete with live mark-to-market scoring and … | 56 | 21561 | active |
| alibaba/open-code-review OpenCodeReview is an AI-powered code review CLI tool that combines deterministic analysis pipelines with an LLM Agent to produce precise li… | 76 | 21468 | active |
| dyad-sh/dyad Dyad is a free, local, open-source desktop AI app builder that turns natural-language prompts into working full-stack web apps, positioned … | 82 | 21322 | active |
| huggingface/candle Candle is a minimalist machine learning framework for Rust focused on performance and ease of use, with CPU and CUDA GPU support. It ships … | 73 | 20955 | active |
| jundot/omlx oMLX is a native macOS menu bar application and LLM inference server built on Apple's MLX framework, featuring continuous batching and two-… | 78 | 20760 | active |
| tashfeenahmed/freellmapi FreeLLMAPI is an open-source, self-hosted LLM gateway that aggregates free tiers from 34 providers (600+ free model endpoints) behind a sin… | 81 | 20682 | active |
| openai/gpt-oss OpenAI's gpt-oss repository hosts reference implementations and tooling for two open-weight language models, gpt-oss-120b and gpt-oss-20b, … | 71 | 20347 | stable |
| karpathy/llama2.c A minimal, dependency-free implementation of the Llama 2 LLM architecture that trains in PyTorch and runs inference in a single ~700-line p… | 28 | 20028 | active |
| Alibaba-NLP/DeepResearch Tongyi DeepResearch is an open-source agentic large language model (30.5B total parameters, 3.3B activated per token) plus training and inf… | 52 | 19878 | active |
| QwenLM/Qwen3-VL Qwen3-VL is a series of open-weight multimodal vision-language models from Alibaba's Qwen team, available in Dense and MoE architectures wi… | 52 | 19847 | active |
| DB-GPT DB-GPT is an open-source agentic AI data assistant that connects to databases, spreadsheets, and knowledge bases, autonomously writes SQL a… | 93 | 19800 | active |
| nari-labs/dia Dia is a 1.6B-parameter open-weight text-to-speech model from Nari Labs that generates ultra-realistic multi-speaker dialogue in a single p… | 43 | 19378 | active |
| kvcache-ai/ktransformers KTransformers is a Python/C++ framework for CPU-GPU heterogeneous inference and fine-tuning of large language models, with a kt-kernel back… | 90 | 19303 | active |
| openai/evals OpenAI Evals is a Python framework for evaluating large language models and LLM-based systems, with an open-source registry of benchmark ev… | 66 | 19262 | active |
| lss233/kirara-ai Kirara AI is a self-hosted, DIY multimodal AI chatbot framework that connects large language models (DeepSeek, ChatGPT, Claude, Gemini, Oll… | 46 | 18972 | active |
| PrimeIntellect-ai/prime-agent Prime Agent is an open-source coding and research agent built around Recursive Language Models (RLM) and a Continual Harness that stores pr… | 81 | 18642 | active |
| 1jehuang/jcode jcode is an open-source AI coding agent for the terminal, written in Rust, that lets developers run many agent sessions in parallel with ve… | 78 | 18635 | active |
| WebLLM WebLLM is a high-performance in-browser LLM inference engine that runs language models entirely client-side using WebGPU hardware accelerat… | 85 | 18603 | active |
| emcie-co/parlant Parlant is an open-source Python framework that acts as an interaction control harness for building reliable, customer-facing LLM agents. I… | 83 | 18269 | active |
| xming521/WeClone WeClone is an end-to-end Python framework for creating a personal AI digital twin by fine-tuning large language models on your exported cha… | 80 | 18171 | active |
| avante-corp/avante.nvim A Neovim plugin that emulates the Cursor AI IDE experience inside the editor. It provides AI-driven code suggestions, chat-based code assis… | 90 | 18137 | active |
| langchain-ai/langchainjs LangChain.js is a TypeScript framework for building LLM-powered applications, providing standard interfaces for agents, models, embeddings,… | 93 | 18135 | active |
| apple/ml-stable-diffusion Apple's official tooling for running Stable Diffusion on Apple Silicon using Core ML. It includes a Python package for converting PyTorch m… | 36 | 17968 | active |
| confident-ai/deepeval DeepEval is an open-source Python framework for evaluating LLM applications with pytest-style unit tests and 50+ research-backed metrics in… | 95 | 17882 | active |
| AsyncFuncAI/deepwiki-open DeepWiki-Open (now Grok-Wiki) is an open-source, AI-powered tool that automatically generates interactive wikis, documentation, diagrams, a… | 63 | 17787 | active |
| deepseek-ai/Janus Janus-Series is DeepSeek's family of unified multimodal models (Janus, Janus-Pro, JanusFlow) that combine multimodal understanding and imag… | 24 | 17757 | active |
| camel-ai/camel CAMEL is an open-source Python framework for building multi-agent AI systems, providing agents, role-based societies, memory, RAG pipelines… | 84 | 17646 | active |
| Canner/WrenAI WrenAI is an open-source generative BI (GenBI) engine that turns natural-language questions into governed SQL, charts, and dashboards acros… | 87 | 17401 | active |
| microsoft/data-formulator Data Formulator is an AI-powered interactive data analysis and visualization application from Microsoft that lets users connect data source… | 86 | 17011 | active |
| stackblitz/bolt.new Bolt.new is an open-source AI-powered web development agent that lets users prompt, run, edit, and deploy full-stack web applications direc… | 22 | 16523 | active |
| ai-shifu/ChatALL ChatALL is an Electron desktop application that sends prompts to multiple AI chatbots (ChatGPT, Claude, Bing Chat, ChatGLM, ERNIE, and more… | 67 | 16483 | active |
| HKUDS/DeepCode DeepCode is an open agentic coding framework from HKUDS that orchestrates multi-agent LLM pipelines to autonomously generate production-gra… | 85 | 16434 | active |
| steven-tey/novel Novel is an open-source, headless Notion-style WYSIWYG rich text editor built on Tiptap and ProseMirror, with AI-powered autocompletion via… | 33 | 16432 | active |
| huggingface/transformers.js Transformers.js is a JavaScript library that lets you run Hugging Face Transformers pretrained models directly in the browser (or Node.js) … | 91 | 16270 | active |
| bradautomates/claude-video A Claude Code plugin and Agent Skill that lets AI coding agents watch videos. It downloads videos via yt-dlp, extracts scene-aware frames w… | 74 | 16260 | active |
| andrewyng/aisuite aisuite is a lightweight Python library providing a unified, OpenAI-style Chat Completions API across many LLM providers (OpenAI, Anthropic… | 83 | 16174 | active |
| raga-ai-hub/RagaAI-Catalyst RagaAI Catalyst is a Python SDK for observability, monitoring, and evaluation of LLM and agentic AI applications, with tracing of agents, L… | 55 | 16150 | active |
| vercel-labs/json-render json-render is a Generative UI framework that lets AI generate dynamic, personalized interfaces from natural language prompts, constrained … | 77 | 16039 | active |
| alibaba/MNN MNN is a lightweight, high-performance deep learning inference engine developed by Alibaba, supporting LLMs, vision, audio, and multimodal … | 93 | 15973 | active |
| ahmedkhaleel2004/gitdiagram GitDiagram is a web application that turns any public or private GitHub repository into an interactive architecture diagram in seconds, wit… | 65 | 15915 | active |
| xbtlin/ai-berkshire A collection of AI research skills for Claude Code and Codex that systematizes the value investing methodologies of Buffett, Munger, Duan Y… | 63 | 15907 | active |
| WEIFENG2333/VideoCaptioner VideoCaptioner is an open-source AI-powered video subtitling tool that handles the full pipeline: speech recognition, LLM-based subtitle se… | 82 | 15764 | active |
| GLips/Figma-Context-MCP Framelink Figma Context MCP is a Model Context Protocol server that fetches and simplifies Figma design data (layout, styling, components) … | 82 | 15720 | active |
| dottxt-ai/outlines Outlines is a Python library for structured generation that guarantees LLM outputs conform to a desired structure (JSON schemas, regex, gra… | 96 | 15695 | active |
| plandex-ai/plandex Plandex is an open-source, terminal-based AI coding agent that plans and executes large, multi-step coding tasks across many files in real-… | 49 | 15597 | active |
| Anionex/banana-slides Banana Slides is an AI-native presentation generator built on the nano banana pro image model that turns ideas, outlines, documents, or tem… | 68 | 15501 | active |
| modelscope/ms-swift ms-swift is a ModelScope community framework for fine-tuning, inference, evaluation, quantization, and deployment of 600+ text LLMs and 400… | 91 | 15371 | active |
| ggml-org/ggml ggml is a plain C/C++ tensor library for machine learning with no dependencies, designed to be simple, portable, and efficient. It supports… | 99 | 15236 | stable |
| GeeeekExplorer/nano-vllm A lightweight vLLM-style LLM inference engine implemented from scratch in about 1,200 lines of Python. It offers fast offline inference wit… | 54 | 15164 | active |
| GreyDGL/PentestGPT PentestGPT is an AI-powered penetration testing agent framework that drives LLMs (Claude Code, Codex, or many providers in legacy mode) to … | 70 | 15091 | active |
| llmware-ai/llmware llmware is a Python framework for building enterprise RAG pipelines and LLM-based applications using small, specialized open-source models.… | 76 | 14850 | active |
| RWKV RWKV is a novel language model architecture that combines RNN efficiency (linear time, constant memory, no KV-cache) with transformer-level… | 67 | 14683 | active |
| HBAI-Ltd/Toonflow-app Toonflow is an open-source cross-platform desktop application that turns novels and scripts into animated short dramas using AI. It integra… | 78 | 14647 | active |
| NVIDIA/TensorRT-LLM NVIDIA's TensorRT-LLM is a Python library and runtime for defining and serving Large Language Models with state-of-the-art inference optimi… | 84 | 14483 | active |
| SakanaAI/AI-Scientist The AI Scientist is a system that uses large language models to autonomously perform end-to-end scientific research, from idea generation a… | 49 | 14449 | active |
| aiming-lab/AutoResearchClaw AutoResearchClaw is a Python framework for fully autonomous, self-evolving research that takes an idea from chat to a generated paper using… | 76 | 14249 | active |
| netease-youdao/QAnything QAnything is a local knowledge base question-answering system by NetEase Youdao that supports uploading and querying various document forma… | 22 | 14078 | active |