function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| FujiwaraChoki/MoneyPrinter MoneyPrinter is a Python application that automates the creation of YouTube Shorts from a user-provided video topic, using local Ollama mod… | 59 | 13891 | active |
| EleutherAI/lm-evaluation-harness A Python framework for few-shot evaluation of language models across hundreds of standard benchmarks (HellaSwag, MMLU, BIG-Bench, etc.). It… | 89 | 13803 | active |
| 567-labs/instructor Instructor is a Python library that patches LLM clients to return validated, type-safe structured data using Pydantic models, with automati… | 92 | 13782 | stable |
| apache/tvm Apache TVM is an open machine learning compiler framework that takes pre-trained models and compiles them into optimized, deployable module… | 90 | 13691 | active |
| nanobrowser/nanobrowser Nanobrowser is an open-source Chrome extension that provides AI-powered web automation through a multi-agent system (Planner, Navigator, et… | 73 | 13683 | active |
| Lightning-AI/litgpt LitGPT is a Python library providing from-scratch, hackable implementations of 20+ open-source large language models with recipes for pretr… | 93 | 13629 | active |
| Physical-Intelligence/openpi Open-source repository from Physical Intelligence containing vision-language-action (VLA) models for robotics, including π₀, π₀-FAST, and π… | 66 | 13494 | active |
| NVIDIA/TensorRT NVIDIA TensorRT is an SDK for high-performance deep learning inference on NVIDIA GPUs, comprising an inference compiler that converts train… | 94 | 13293 | stable |
| Omi Omi is an open-source AI wearable and companion app ecosystem (necklace pendant, smart glasses, desktop and mobile apps) that captures conv… | 88 | 13264 | active |
| microsoft/agent-framework Microsoft Agent Framework (MAF) is an open-source, multi-language framework for building, orchestrating, and deploying production-grade AI … | 82 | 13135 | active |
| lllyasviel/stable-diffusion-webui-forge A fork/platform built on top of AUTOMATIC1111's Stable Diffusion WebUI that optimizes resource management, speeds up inference, and adds ex… | 32 | 12977 | active |
| PaddlePaddle/PaddleNLP PaddleNLP is an easy-to-use NLP and large language model development kit built on the PaddlePaddle deep learning framework, with a large pr… | 63 | 12967 | active |
| LangChain4j LangChain4j is an open-source Java library for building LLM-powered applications on the JVM, providing a unified API over 20+ LLM providers… | 95 | 12959 | active |
| huggingface/speech-to-speech A low-latency, fully modular voice-agent pipeline (VAD -> STT -> LLM -> TTS) built from open-source models, exposed via the OpenAI Realtime… | 85 | 12882 | active |
| XiaomiMiMo/MiMo-Code MiMoCode is a terminal-native AI coding assistant that can read and write code, run commands, manage Git, and maintain persistent project m… | 80 | 12881 | active |
| deepseek-ai/FlashMLA FlashMLA is DeepSeek's library of optimized CUDA attention kernels implementing Multi-head Latent Attention (MLA), including dense and toke… | 62 | 12872 | active |
| Portkey-AI/gateway Portkey AI Gateway is an open-source, TypeScript-based AI gateway that routes requests to 1,600+ LLMs across many providers through a singl… | 76 | 12837 | active |
| cloudwego/eino Eino is a Go-based framework for building LLM and AI applications, inspired by LangChain and LlamaIndex but following Go conventions. It pr… | 84 | 12831 | active |
| NoFxAiOS/nofx NOFX is an open-source, self-hosted AI trading terminal where a language model acts as the trading strategy, running a continuous decide-an… | 61 | 12761 | active |
| The-PR-Agent/pr-agent PR-Agent is an open-source, AI-powered code review agent that automatically reviews pull requests, generates descriptions, suggests improve… | 96 | 12720 | active |
| The-Pocket/PocketFlow-Tutorial-Codebase-Knowledge An AI agent built with the Pocket Flow LLM framework that crawls GitHub repositories, analyzes codebases to identify core abstractions, and… | 58 | 12632 | active |
| bentoml/OpenLLM OpenLLM is a Python framework and CLI for self-hosting open-source large language models (Llama, DeepSeek, Qwen, Mistral, etc.) as OpenAI-c… | 76 | 12518 | active |
| ace-step/ACE-Step-1.5 ACE-Step 1.5 is an open-source music generation foundation model combining a language model planner with a Diffusion Transformer to create … | 79 | 12421 | active |
| simonw/llm A CLI tool and Python library for running prompts against large language models from OpenAI, Anthropic, Google, and many other providers, i… | 97 | 12412 | stable |
| getomni-ai/zerox Zerox is a library (Node.js and Python packages) that performs OCR and document extraction by converting files like PDFs, DOCX, and images … | 35 | 12266 | active |
| TheR1D/shell_gpt ShellGPT (sgpt) is a Python command-line productivity tool that uses LLMs like GPT-4/5 or local models via Ollama to generate shell command… | 77 | 12259 | active |
| lidge-jun/opencodex opencodex is a lightweight local proxy (installed via npm, running on a bundled Bun runtime) that translates OpenAI Codex's Responses API a… | 76 | 12224 | active |
| elie222/inbox-zero Inbox Zero is an open-source AI email assistant web application that organizes your inbox, pre-drafts replies in your voice, blocks cold em… | 96 | 12091 | active |
| bytedance/trae-agent Trae Agent is an LLM-based CLI agent from ByteDance for general-purpose software engineering tasks, executing workflows via tools like file… | 48 | 12058 | active |
| jd-opensource/joyagent-jdgenie JoyAgent-JDGenie is an open-source, end-to-end, production-grade general-purpose multi-agent product built in Java. It directly answers use… | 42 | 11879 | active |
| nidhinjs/prompt-master A Claude skill (plugin) that generates and refines precise prompts for a wide range of AI tools like ChatGPT, Cursor, Midjourney, and Stabl… | 59 | 11818 | active |
| BeehiveInnovations/pal-mcp-server PAL MCP (formerly Zen MCP) is a Model Context Protocol server that lets AI coding CLIs like Claude Code, Gemini CLI, and Codex CLI orchestr… | 49 | 11724 | active |
| tensorzero/tensorzero TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation behi… | 10 | 11719 | active |
| github/copilot.vim A Vim/Neovim plugin that integrates GitHub Copilot, an AI pair programmer that generates code suggestions from natural language prompts. Su… | 66 | 11684 | active |
| NVIDIA/cosmos NVIDIA Cosmos is an open platform of omnimodal world foundation models, datasets, and tools for building Physical AI systems such as robots… | 72 | 11641 | active |
| microsoft/autogen AutoGen is a Python framework from Microsoft for building multi-agent AI applications where LLM-powered agents can act autonomously or coll… | 68 | 60643 | maintenance |
| LMCache/LMCache LMCache is a KV cache management layer for LLM inference that stores, compresses, and reuses KV caches across requests, sessions, and servi… | 86 | 11460 | active |
| wangrongding/wechat-bot A Node.js-based multi-platform instant messaging AI agent built on Wechaty that connects WeChat, Telegram, WhatsApp, and Lark to LLM provid… | 67 | 11295 | active |
| Doriandarko/claude-engineer An interactive AI assistant framework built on Anthropic's Claude 3.5 Sonnet that can autonomously generate and manage its own tools during… | 24 | 11222 | active |
| Arize-ai/phoenix Arize Phoenix is an open-source AI observability and evaluation platform for tracing, debugging, and evaluating LLM applications. It suppor… | 95 | 11205 | active |
| openai/openai-node The official JavaScript/TypeScript SDK for the OpenAI REST API, auto-generated from OpenAI's OpenAPI specification. It provides typed acces… | 95 | 11145 | stable |
| github/copilot-cli GitHub Copilot CLI brings the Copilot coding agent to the terminal, letting developers build, debug, and refactor code through natural lang… | 93 | 11122 | active |
| SparkAudio/Spark-TTS Spark-TTS is an LLM-based text-to-speech system built on Qwen2.5 that generates speech directly from single-stream decoupled speech tokens.… | 27 | 11006 | active |
| NoeFabris/opencode-antigravity-auth An OpenCode plugin that authenticates against Google's Antigravity IDE via OAuth, letting users access Gemini and Claude models with their … | 10 | 11003 | active |
| kyutai-labs/moshi Moshi is a speech-text foundation model and full-duplex spoken dialogue framework from Kyutai, built around the Mimi streaming neural audio… | 62 | 10949 | active |
| altic-dev/FluidVoice FluidVoice is an open-source (GPLv3) macOS dictation app that transcribes speech to text entirely on-device using models like Parakeet, Whi… | 84 | 10946 | active |
| triton-inference-server/server NVIDIA Triton Inference Server is an open-source inference serving software that deploys AI models from multiple frameworks (TensorRT, PyTo… | 98 | 10939 | stable |
| yizhiyanhua-ai/fireworks-tech-graph A Python-based Agent Skill for Codex and Claude Code that converts natural language descriptions (English or Chinese) into production-quali… | 79 | 10914 | active |
| huggingface/chat-ui Chat UI is the open source SvelteKit web application powering HuggingChat, providing a browser-based chat interface for LLMs. It connects t… | 84 | 10911 | active |
| linyqh/NarratoAI NarratoAI is an open-source, self-hosted AI-powered video commentary and automated editing tool. It uses large language models to generate … | 88 | 10879 | active |
| CodebuffAI/freebuff Freebuff is a free AI coding agent available as a desktop app, CLI, web builder, cloud IDE, and chat interface, funded by text ads instead … | 74 | 10875 | active |
| elder-plinius/G0DM0D3 G0DM0D3 is an open-source, multi-model AI chat interface designed for red-teaming, cognition research, and unrestricted LLM interaction. It… | 56 | 10790 | active |
| WasmEdge/WasmEdge WasmEdge is a lightweight, high-performance, extensible WebAssembly runtime hosted by CNCF, designed for cloud native, edge, and decentrali… | 97 | 10787 | active |
| huggingface/ml-intern An open-source autonomous ML engineer agent from Hugging Face that reads papers, writes code, trains models, and ships ML work using the Hu… | 59 | 10753 | active |
| sashabaranov/go-openai An unofficial Go client library for the OpenAI API covering chat completions, the Responses API, embeddings, image generation, audio transc… | 98 | 10748 | active |
| AntonOsika/gpt-engineer gpt-engineer is a Python CLI platform for experimenting with AI code generation, letting users specify software in natural language and wat… | 10 | 55120 | maintenance |
| alibaba/spring-ai-alibaba Spring AI Alibaba is a production-ready agentic AI framework for Java developers, built on Spring AI and a DAG-based Graph runtime. It prov… | 76 | 10712 | active |
| chathub-dev/chathub ChatHub is an all-in-one AI chat client, originally a Chrome/Edge browser extension and now also a web app and mobile/desktop apps, that le… | 62 | 10649 | active |
| langchain-ai/open-swe Open SWE is an open-source framework from LangChain for building an organization's internal asynchronous coding agent, built on LangGraph a… | 63 | 10620 | active |
| abetlen/llama-cpp-python Python bindings for llama.cpp providing low-level ctypes access to the C API plus a high-level API for text completion with OpenAI-like int… | 92 | 10585 | active |
| skypilot-org/skypilot SkyPilot is an open-source AI compute platform that unifies fragmented infrastructure (Kubernetes, Slurm, VMs, 20+ clouds) into a single po… | 97 | 10529 | active |
| bigscience-workshop/petals Petals is a Python library that lets you run and fine-tune large language models (Llama 3.1, Mixtral, Falcon, BLOOM) on a BitTorrent-style … | 23 | 10521 | active |
| Acly/krita-ai-diffusion A Krita plugin providing a streamlined interface for AI image generation, inpainting, and outpainting within the Krita painting application… | 95 | 10515 | active |
| easydiffusion/easydiffusion Easy Diffusion is a 1-click installer and browser-based UI for running Stable Diffusion text-to-image generation locally on your PC. It bun… | 81 | 10455 | active |
| github/copilot-sdk A multi-platform SDK exposing the GitHub Copilot CLI agent runtime for programmatic use, with official SDKs for Python, TypeScript, Go, .NE… | 82 | 10438 | active |
| NVIDIA/personaplex PersonaPlex is a real-time, full-duplex speech-to-speech conversational model from NVIDIA that supports persona control via text role promp… | 47 | 10393 | active |
| a16z-infra/ai-town AI Town is a deployable starter kit for building a virtual town where AI characters live, chat, and socialize, inspired by the Generative A… | 73 | 10392 | active |
| nexmoe/VidBee VidBee is a free, open-source Electron desktop app for downloading video and audio from 1000+ sites (via yt-dlp) and importing local media.… | 84 | 10386 | active |
| RunanywhereAI/runanywhere-sdks RunAnywhere is a set of cross-platform SDKs (Swift, Kotlin, React Native, Flutter, TypeScript, C++) over a shared C++ core for running AI m… | 85 | 10282 | active |
| OpenBMB/MiniCPM MiniCPM is a family of small, state-of-the-art on-device language models from OpenBMB, with MiniCPM5-1B being a dense 1B Transformer for lo… | 75 | 10250 | active |
| xai-org/grok-1 xAI's open release of the Grok-1 open-weights model (314B-parameter Mixture-of-Experts LLM) with JAX example code for loading and running i… | 25 | 52189 | maintenance |
| JOYCEQL/magic-resume Magic Resume is a free online AI-powered resume editor built with TanStack Start, TypeScript, and Framer Motion. It offers real-time previe… | 88 | 10209 | active |
| OpenGVLab/InternVL InternVL is a family of open-source multimodal large language models (vision-language models) that combine vision transformers with LLMs to… | 37 | 10146 | active |
| deepseek-ai/DeepEP DeepEP is a high-performance GPU communication library for expert parallelism (EP) in MoE training and inference, providing high-throughput… | 57 | 10066 | active |
| m87-labs/moondream Moondream is an open-weight family of small, efficient vision language models (2B to 9B MoE) that perform image captioning, visual question… | 61 | 10014 | active |
| presenton/presenton Presenton is an open-source AI presentation generator and API that creates editable PPTX and PDF slide decks from prompts, documents, or cu… | 82 | 9861 | active |
| HKUDS/AutoAgent AutoAgent is a fully-automated, zero-code framework for creating and deploying LLM agents using natural language alone. It acts as an auton… | 41 | 9753 | active |
| Tiiny-AI/PowerInfer PowerInfer is a CPU/GPU hybrid LLM inference engine that exploits activation locality to run large language models fast on consumer-grade h… | 63 | 9752 | active |
| mshumer/gpt-prompt-engineer A Jupyter Notebook-based tool that automatically generates, tests, and ranks LLM prompts for a given task using GPT-4, GPT-3.5-Turbo, or Cl… | 50 | 9675 | active |
| tmc/langchaingo Langchaingo is the Go language port of LangChain, providing composable building blocks for writing LLM-powered applications in Go. It offer… | 53 | 9637 | active |
| yihong0618/bilingual_book_maker A Python CLI tool that uses AI translation services (ChatGPT, Claude, Gemini, DeepL, and others) to create bilingual versions of epub, txt,… | 76 | 9609 | active |
| xorbitsai/inference Xinference is an open-source model serving platform for deploying LLMs, embedding, speech, image, and multimodal models via a unified OpenA… | 91 | 9523 | active |
| oumi-ai/oumi Oumi is an open-source Python framework and platform for the end-to-end lifecycle of open-weight LLMs: data synthesis, fine-tuning (SFT, Lo… | 86 | 9373 | active |
| spring-projects/spring-ai Spring AI is an application framework for AI engineering that provides Spring-friendly, portable abstractions for integrating AI models int… | 95 | 9358 | active |
| mengxi-ream/read-frog Read Frog is an open-source AI-powered browser extension for language learning and immersive translation. It translates web pages, selected… | 82 | 9346 | active |
| cactus-compute/needle Needle 2 is a 45M-parameter open language model packaged as a single 14MB binary for tool calling, device use, and structured extraction on… | 59 | 9311 | active |
| Narcooo/inkos InkOS is an AI agent system for story creation and multilingual translation, supporting long-form novels, scripts, interactive games, and I… | 81 | 9294 | active |
| LTX-2 Official Python package from Lightricks providing inference pipelines and LoRA training for LTX-2/LTX-2.5, an open-weights DiT-based founda… | 83 | 9260 | active |
| fastrepl/anarlog Anarlog is an open-source, local-first AI meeting notepad and Granola alternative that records device audio without a bot joining the call,… | 84 | 9181 | active |
| droidrun/mobilerun Mobilerun is an open-source Python framework for controlling Android and iOS devices with LLM-powered agents using natural language command… | 86 | 9127 | active |
| modelscope/modelscope ModelScope is a Python library and ecosystem built on the 'Model-as-a-Service' concept, providing unified APIs to download, run inference o… | 98 | 9111 | active |
| Future-House/paper-qa PaperQA2 is a Python library and CLI for high-accuracy retrieval augmented generation (RAG) over PDFs, text, Office documents, and source c… | 93 | 9103 | active |
| Nutlope/aicommits Aicommits is a Node.js CLI that automatically generates git commit messages from your staged changes using AI. It supports multiple LLM pro… | 96 | 9094 | active |
| BoundaryML/baml BAML is a domain-specific programming language for building LLM agents, with a Rust-like type system, typed errors, built-in tests/evals, a… | 90 | 9078 | active |
| Model Context Protocol A collection of reference implementations of Model Context Protocol (MCP) servers maintained by the MCP steering group, demonstrating how t… | 84 | 9065 | active |
| open-jarvis/OpenJarvis OpenJarvis is a Python framework for building local-first personal AI agents that run on your own hardware, with cloud APIs as an optional … | 73 | 9045 | active |
| Thysrael/Horizon Horizon is an AI-powered news aggregation platform that fetches content from sources like Hacker News, GitHub, RSS, Reddit, and Telegram, s… | 59 | 9040 | active |
| EvoMap/evolver Evolver is a Node.js-based self-evolving engine for AI agents built on the GEP (Gene Evolution Protocol), letting agents turn experience in… | 78 | 9007 | active |
| LearningCircuit/local-deep-research A self-hosted AI research assistant that performs deep, iterative multi-source research using local or cloud LLMs (Ollama, llama.cpp, OpenA… | 83 | 8995 | active |
| MervinPraison/PraisonAI PraisonAI is a Python multi-agent framework for building and deploying autonomous, self-improving AI agents with built-in memory, RAG, and … | 88 | 8965 | active |