function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| MacPaw/OpenAI A community-maintained Swift package that wraps the OpenAI public API, supporting chat completions, responses, function calling, MCP tools,… | 94 | 2935 | active |
| signerless/llm-checker A Node.js CLI tool that scans your hardware (GPU, VRAM, CPU, memory) and recommends which LLM or small language models you can run locally,… | 84 | 2935 | active |
| Alpha-Dojo/DojoAgents DojoAgents is a full-market AI copilot framework for personal investment, built around a Loop-Driven Cognitive Portfolio Agent that reasons… | 58 | 2934 | active |
| hero8152/Infinite-Canvas An infinite canvas desktop application for orchestrating AI image, video, and LLM generation workflows. It integrates with ComfyUI, OpenAI-… | 57 | 2934 | active |
| Biohub/esm A Python library and model release providing ESMC protein language models, ESMFold2 structure prediction, and the ESM Atlas of 6.8 billion … | 74 | 2927 | active |
| InternLM/InternLM-XComposer InternLM-XComposer is a family of large vision-language models from the InternLM team, including XComposer2.5 for long-context multimodal u… | 38 | 2925 | active |
| JunChenMoCode/ChatGPT_JCM A Vue2 + ElementUI web management interface that aggregates OpenAI API endpoints (models, chat, images, audio, fine-tuning, files) into a g… | 22 | 2923 | active |
| michaelfeil/infinity Infinity is a high-throughput, low-latency serving engine that exposes text-embedding, reranking, CLIP, CLAP and ColPali models via an Open… | 63 | 2917 | active |
| zjunlp/EasyEdit EasyEdit is an easy-to-use knowledge editing framework for large language models, supporting methods that modify model parameters or steer … | 71 | 2908 | active |
| llm-as-a-verifier/llm-as-a-verifier A Python framework that uses LLM token logprobs to produce fine-grained verification scores for agent trajectories without additional train… | 58 | 2901 | active |
| OpenMind/OM1 OM1 is a modular AI runtime and hardware abstraction layer for building multimodal AI agents that run on physical robots and in simulators.… | 82 | 2897 | active |
| xiamuceer-j/MuMuAINovel MuMuAINovel is a self-hosted AI-powered novel writing assistant built with FastAPI and React. It helps authors generate outlines, character… | 82 | 2886 | active |
| ax-llm/ax Ax is a TypeScript-first LLM programming framework implementing the DSPy model, providing typed signatures, agents, flows, and optimizers l… | 93 | 2885 | active |
| MartialBE/one-hub One Hub is a self-hosted OpenAI-compatible API management and distribution system forked from songquanpeng/one-api, written in Go. It aggre… | 67 | 2882 | active |
| Sophomoresty/gemini-web2api A single-file Python server that exposes Google Gemini's web interface as an OpenAI-compatible API, supporting chat completions, streaming,… | 57 | 2880 | active |
| xiao-zaiyi/illa-helper A browser extension that helps users learn languages naturally while browsing the web, based on the comprehensible input theory. It provide… | 79 | 2876 | active |
| thepersonalaicompany/amurex Amurex is an open-source Chrome extension that acts as an AI meeting copilot for Google Meet and MS Teams. It provides real-time suggestion… | 34 | 2868 | active |
| robert-mcdermott/ai-knowledge-graph A Python CLI tool that uses an LLM to extract Subject-Predicate-Object triplets from unstructured text documents and renders them as an int… | 56 | 2865 | active |
| AminBlg/SimpleEnglish An agent skill (SKILL.md) that makes LLMs write documentation in ASD-STE100 Simplified Technical English, the controlled language used in a… | 67 | 2862 | active |
| memodb-io/memobase Memobase is a user profile-based long-term memory system for LLM applications, providing per-user profiles and event timelines that chatbot… | 50 | 2862 | active |
| leigest519/OpenGame OpenGame is an open-source agentic framework for end-to-end web game creation from a single prompt, built around a reusable 'Game Skill' sy… | 59 | 2859 | active |
| VRSEN/OpenSwarm OpenSwarm is an open-source multi-agent system built on Agency Swarm that turns a single terminal prompt into complete deliverables like sl… | 77 | 2854 | active |
| anysphere/priompt Priompt is a JSX-based prompt design library for building LLM prompts with React-like components. It uses priority levels to decide which p… | 50 | 2850 | active |
| rpamis/comet Comet is an npm-distributed CLI and Skill platform that gives AI coding agents a resumable, phase-guarded workflow for long-running tasks, … | 80 | 2849 | active |
| Y-Research-SBU/QuantHarness QuantHarness is a multi-agent LLM framework for high-frequency trading analysis, built on LangChain and LangGraph. Specialized agents compu… | 62 | 2839 | active |
| moltis-org/moltis Moltis is a secure, self-hosted personal AI agent server written in Rust that ships as a single binary with sandboxed execution, multi-prov… | 78 | 2837 | active |
| TheSmallHanCat/flow2api Flow2API is a self-hosted Python/FastAPI service that exposes an OpenAI- and Gemini-compatible API on top of Google Flow (VideoFX/ImageFX) … | 60 | 2834 | active |
| Doriandarko/o1-engineer o1-engineer is a Python command-line tool that uses OpenAI's API to help developers generate code, edit files, plan projects, and review co… | 22 | 2834 | active |
| spcl/graph-of-thoughts Official Python implementation of the Graph of Thoughts (GoT) framework from ETH Zurich, which models complex problem-solving as a Graph of… | 51 | 2833 | active |
| jjyaoao/HelloAgents HelloAgents is a Python multi-agent framework built on OpenAI-compatible APIs, offering tool response protocols, context engineering, sessi… | 69 | 2831 | active |
| eracle/OpenOutreach OpenOutreach is a self-hosted, open-source AI agent for B2B lead generation. You describe your product and target market, and it discovers … | 77 | 2824 | active |
| Mega4alik/ollm oLLM is a lightweight Python library for large-context LLM inference built on Hugging Face Transformers and PyTorch. It offloads weights an… | 60 | 2788 | active |
| Luce-Org/lucebox Lucebox is an open-source LLM inference engine written in C++ with custom CUDA/HIP kernels, focused on speculative decoding and speculative… | 59 | 2788 | active |
| zts212653/clowder-ai Clowder AI is a TypeScript framework for building teams of AI agents that collaborate across multiple models like Claude, GPT, and Gemini i… | 80 | 2779 | active |
| fuergaosi233/claude-code-proxy A Python proxy server that translates Claude API requests (/v1/messages) into OpenAI-compatible API calls, letting the Claude Code CLI work… | 50 | 2778 | active |
| OpenDCAI/Paper2Any Paper2Any is an open-source Python application that turns research papers, text, or topics into editable research figures, technical route … | 56 | 2776 | active |
| microsoft/promptflow Prompt flow is a suite of Python SDK, CLI, and VS Code extension tools for the end-to-end development of LLM-based AI applications, linking… | 75 | 11229 | maintenance |
| FasterDecoding/Medusa Medusa is a framework that accelerates LLM text generation by adding multiple decoding heads to an existing model, avoiding the need for a … | 18 | 2770 | active |
| ideogram-oss/ideogram4 Ideogram 4 is an open-weight text-to-image foundation model trained from scratch, with inference code and weights released in Python. It fe… | 54 | 2766 | active |
| omnara-ai/omnara Omnara is an open-source platform for running managed, durable AI agents, handling execution and state while you bring your own models, too… | 85 | 2765 | active |
| langchain-ai/social-media-agent A TypeScript AI agent built on LangGraph that takes a URL, scrapes its content, and generates Twitter and LinkedIn posts using LLMs. It inc… | 66 | 2756 | active |
| huggingface/swift-coreml-diffusers A native SwiftUI application demonstrating how to run Stable Diffusion text-to-image generation on-device using Apple's Core ML Stable Diff… | 54 | 2756 | active |
| filip-michalsky/SalesGPT SalesGPT is an open-source, context-aware AI sales agent built with LLMs that automates sales outreach across voice, email, SMS, and messag… | 20 | 2743 | active |
| folke/sidekick.nvim A Neovim plugin that integrates Copilot LSP's Next Edit Suggestions with an embedded terminal for AI CLI tools like Claude Code, Gemini CLI… | 66 | 2742 | active |
| PenglongHuang/chinese-novelist-skill A Claude Code skill that drives AI to write complete Chinese novels end-to-end. It uses layered interactive Q&A, cross-session preference m… | 60 | 2739 | active |
| codeany-ai/open-agent-sdk-typescript An open-source TypeScript Agent SDK that runs the full agent loop in-process without CLI or subprocess dependencies, as an alternative to c… | 57 | 2739 | active |
| GodsScion/Auto_job_applier_linkedIn A Python/Selenium desktop application that automates LinkedIn Easy Apply job applications, finding relevant jobs, filling application quest… | 72 | 2736 | active |
| ModelTC/LightX2V LightX2V is a lightweight, high-performance inference framework for image and video generation, supporting tasks like text-to-video, image-… | 64 | 2733 | active |
| Zero6992/chatGPT-discord-bot A self-hosted Discord bot that integrates ChatGPT-style AI chat into Discord servers, supporting multiple AI providers including a free pro… | 37 | 2731 | active |
| songguoxs/seedance-prompt-skill A Claude Code custom skill that turns Claude into an AI video prompt engineer for ByteDance's Seedance 2.0 video generation platform. It co… | 45 | 2724 | active |
| HuaYaoAI/FinGenius FinGenius is an open-source AI financial analysis platform for China's A-share stock market, built on a multi-agent architecture where spec… | 35 | 2724 | active |
| openlit/openlit OpenLIT is an open-source AI engineering platform providing OpenTelemetry-native LLM observability, evaluations, guardrails, prompt managem… | 89 | 2718 | active |
| Bytez-com/docs Bytez is a serverless model inference platform offering a unified API with a single key to run 175k+ open and closed-source AI models (LLMs… | 56 | 2718 | active |
| AmberSahdev/Open-Interface Open Interface is a cross-platform desktop application that lets users control their computer using natural language requests processed by … | 65 | 2715 | active |
| getumbrel/llama-gpt LlamaGPT is a self-hosted, offline ChatGPT-like chatbot powered by Llama 2 and Code Llama models via llama.cpp, with all data staying on th… | 28 | 10940 | maintenance |
| vllm-project/vllm-ascend vllm-ascend is a community-maintained hardware plugin that enables vLLM to run large language model inference on Huawei Ascend NPUs. It imp… | 83 | 2711 | active |
| intel/neural-compressor Intel Neural Compressor is an open-source Python library providing state-of-the-art low-bit quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4),… | 92 | 2704 | active |
| huggingface/text-generation-inference Text Generation Inference (TGI) is a Rust, Python and gRPC toolkit for deploying and serving large language models with high performance, p… | 10 | 10889 | maintenance |
| TavernAI/TavernAI-v1 TavernAI is an atmospheric frontend application for roleplay chat and story writing with AI language models, connecting to backends like Ko… | 61 | 2700 | active |
| xdit-project/xDiT xDiT is a scalable inference engine for Diffusion Transformers (DiTs) that enables parallel deployment across multiple GPUs and machines. I… | 77 | 2699 | active |
| facebookresearch/HyperAgents A research framework from Meta AI implementing self-referential, self-improving LLM agents that can optimize their own code for arbitrary c… | 57 | 2697 | active |
| nicepkg/aide Aide is a VSCode extension that brings AI-assisted coding features such as one-click code commenting, language/framework conversion, UI scr… | 23 | 2690 | active |
| Vali-98/ChatterUI ChatterUI is a native mobile frontend for running large language models on-device via llama.cpp or connecting to commercial and open-source… | 83 | 2689 | active |
| KimMeen/Time-LLM Time-LLM is the official PyTorch implementation of an ICLR 2024 paper that reprograms frozen large language models (Llama, GPT-2, BERT) for… | 47 | 2685 | active |
| ColeMurray/background-agents An open-source, self-hostable background coding agent system (Open-Inspect) inspired by Ramp's Inspect, where AI agents work on tasks in sa… | 60 | 2683 | active |
| Project-N-E-K-O/N.E.K.O Project N.E.K.O. is an open-source AI companion application — a proactive catgirl-style AI that lives on your desktop, initiates interactio… | 85 | 2681 | active |
| thewh1teagle/kokoro-onnx A Python library that runs the Kokoro text-to-speech model via ONNX Runtime, supporting CPU and GPU inference. It provides multi-language T… | 67 | 2680 | active |
| abi/secret-llama Secret Llama is a fully private LLM chatbot that runs entirely in the browser with no server or installation required, powered by the WebLL… | 24 | 2678 | active |
| openai/openai-dotnet The official .NET library for accessing the OpenAI REST API, generated from the OpenAPI specification in collaboration with Microsoft. It p… | 86 | 2674 | active |
| ZHZisZZ/dllm dLLM is a Python library that unifies training, inference, and evaluation of diffusion language models such as LLaDA and Dream. It builds o… | 59 | 2672 | active |
| om-ai-lab/OmAgent OmAgent is a Python library for building multimodal language agents, wrapping worker orchestration, task queues, and graph-based workflow o… | 31 | 2665 | active |
| Haleclipse/CodexDesktop-Rebuild A cross-platform Electron rebuild of the OpenAI Codex Desktop app, distributed for macOS, Windows, and Linux. It packages the Codex coding … | 79 | 2654 | active |
| icereed/paperless-gpt A self-hosted companion application for paperless-ngx that uses LLMs and vision models to auto-generate document titles, tags, and dates, a… | 87 | 2651 | active |
| Mobile-Artificial-Intelligence/maid Maid is a free, open-source Android chat application (built with React Native) for talking to large language models. It runs GGUF models fu… | 83 | 2651 | active |
| Const-me/Whisper A Windows port of whisper.cpp that runs OpenAI's Whisper speech recognition model on the GPU via DirectCompute (Direct3D 11 compute shaders… | 60 | 10649 | maintenance |
| ryfineZ/codex-session-patcher A Python tool that cleans AI refusal responses from Codex CLI, Claude Code, and OpenCode session files so conversations can be resumed, wit… | 79 | 2649 | active |
| mito-ds/mito Mito is a set of JupyterLab extensions that combine an AI chat copilot, smart debugging, and an interactive spreadsheet interface that gene… | 79 | 2645 | active |
| black-forest-labs/flux2 Official inference repository for Black Forest Labs' FLUX.2 family of open-weight image generation and editing models. It provides minimal … | 48 | 2642 | active |
| kingjulio8238/Memary Memary is an open-source memory layer for autonomous AI agents that emulates human memory using knowledge graphs. It is a Python library (p… | 22 | 2641 | active |
| ExplosiveCoderflome/AI-Novel-Writing-Assistant An AI-native open-source workspace for producing complete long-form novels, orchestrating planning, worldbuilding, character assets, chapte… | 87 | 2640 | active |
| huangserva/3DCellForge A React + Three.js web application for AI-powered interactive 3D model generation, inspection, and presentation. It turns uploaded referenc… | 51 | 2624 | active |
| trustgraph-ai/trustgraph TrustGraph is an open-source context interoperability layer that builds unified semantic context layers using hypergraphs, knowledge graphs… | 68 | 2616 | active |
| wassim249/fastapi-langgraph-agent-production-ready-template A production-ready FastAPI template for building AI agent backends with LangGraph integration. It provides stateful conversations, long-ter… | 64 | 2614 | active |
| open-gigaai/giga-brain-0 GigaBrain-0/0.7 is an open-source vision-language-action (VLA) model family for generalist embodied agents, powered by world models and a t… | 62 | 2611 | active |
| aardio/ImTip ImTip is a lightweight (under 1 MB) Windows desktop assistant that shows real-time input method and keyboard state indicators at the text c… | 74 | 2610 | active |
| run-llama/sec-insights SEC Insights is a full-stack reference application built by LlamaIndex that uses RAG to answer questions about SEC 10-K and 10-Q financial … | 33 | 2609 | active |
| NVIDIA/NeMo-Agent-Toolkit NVIDIA NeMo Agent Toolkit is an open-source Python library for connecting, orchestrating, and optimizing teams of AI agents across any agen… | 83 | 2603 | active |
| kairos-agi/kairos Kairos is the official open-source implementation of a 4B-parameter native cross-embodiment world model that unifies video understanding, f… | 57 | 2598 | active |
| iamsrikanthnani/pluely Pluely is a privacy-first desktop AI assistant that runs as an invisible always-on-top overlay, providing live meeting transcription, on-sc… | 79 | 2596 | active |
| dreamzero0/dreamzero DreamZero is NVIDIA's World Action Model (WAM) that jointly predicts future video and actions from a pretrained video diffusion backbone, e… | 50 | 2593 | active |
| 6drf21e/ChatTTS_colab A one-click deployment wrapper around ChatTTS providing a Gradio web UI for text-to-speech, runnable in Google Colab or via an offline Wind… | 53 | 2592 | active |
| AnswerDotAI/llms-txt A proposal and Python tooling for the /llms.txt standard, a markdown file placed at a website root to give language models and agents conci… | 78 | 2586 | active |
| CodeGeeX CodeGeeX is a family of open multilingual code generation large language models (13B and successors CodeGeeX2/CodeGeeX4) pre-trained on 20+… | 23 | 2585 | active |
| ahmedeltaher/Android-MVVM-Architecture-Android-Voice-AI-SDK A reusable Android library (Kotlin, MVVM) that provides a full voice-driven AI conversation pipeline: microphone capture with VAD, speech-t… | 70 | 2578 | active |
| zachlatta/freeflow FreeFlow is a free, open-source macOS dictation app that transcribes speech via Groq's fast API and pastes cleaned text into any focused te… | 77 | 2576 | active |
| sourcegraph/sourcegraph-public-snapshot Sourcegraph is a code intelligence platform providing universal code search, code navigation, batch changes, and the Cody AI coding assista… | 10 | 10295 | maintenance |
| griptape-ai/griptape Griptape is a modular Python framework for building generative AI applications, providing abstractions for agents, pipelines, workflows, RA… | 94 | 2573 | active |
| BAAI-Agents/Cradle Cradle is a Python framework for General Computer Control (GCC), enabling foundation agents to perform complex computer tasks using screens… | 25 | 2573 | active |
| Memento-Teams/Memento Memento is a Python framework for building LLM agents that continually improve from experience via memory-based case-based reasoning, witho… | 39 | 2568 | active |
| yazinsai/OpenOats OpenOats is a macOS meeting assistant that transcribes both sides of a call in real time using on-device speech recognition and surfaces re… | 77 | 2565 | active |