function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| TheoKanning/openai-java A Java client library for the OpenAI API, providing POJOs, a Retrofit-based client, and a convenience service class for GPT-3, ChatGPT, and… | 10 | 4729 | abandoned |
| truefoundry/cognita Cognita is an open-source RAG (Retrieval Augmented Generation) framework by TrueFoundry for building modular, production-ready RAG applicat… | 10 | 4418 | abandoned |
| rawandahmad698/PyChatGPT PyChatGPT is a Python client and CLI for ChatGPT that reverse-engineers the ChatGPT web interface, handling automatic login, access token r… | 74 | 4183 | abandoned |
| nraiden/openv0 openv0 is a generative UI component framework that uses AI to generate and iteratively refine UI components with live preview. It runs a mu… | 28 | 3951 | abandoned |
| karpathy/reader3 A lightweight, self-hosted EPUB reader web app that presents books one chapter at a time so users can easily copy text into an LLM and read… | 40 | 3842 | abandoned |
| ravenscroftj/turbopilot TurboPilot is a self-hosted, Copilot-like code completion server that runs large language models (Salesforce CodeGen, StableCode, WizardCod… | 10 | 3779 | abandoned |
| askrella/whatsapp-chatgpt A self-hosted WhatsApp bot that connects OpenAI's GPT and DALL-E 2 to WhatsApp, letting users chat with an AI assistant and generate images… | 73 | 3777 | abandoned |
| twinnydotdev/twinny Twinny is a free, MIT-licensed Visual Studio Code extension providing AI code completion (fill-in-the-middle) and code chat, working with l… | 10 | 3630 | abandoned |
| mukulpatnaik/researchgpt A FastAPI-based research assistant that lets users have a conversation with any PDF research paper by extracting text, generating embedding… | 10 | 3531 | abandoned |
| gencay/vscode-chatgpt An unofficial Visual Studio Code extension integrating OpenAI ChatGPT, providing sidebar conversations, streaming answers, and code inserti… | 22 | 3451 | abandoned |
| latitudegames/AIDungeon AI Dungeon 2 is an open-source, infinitely generated text adventure game powered by a fine-tuned GPT-2 language model. It can be played loc… | 10 | 3224 | abandoned |
| protectai/llm-guard LLM Guard is a Python library providing input and output scanners to secure LLM interactions, including sanitization, harmful language dete… | 10 | 3203 | abandoned |
| alpa-projects/alpa Alpa is a Python system for training and serving large-scale neural networks by automatically parallelizing single-device code across distr… | 10 | 3178 | abandoned |
| neuralmagic/deepsparse DeepSparse is a sparsity-aware deep learning inference runtime that delivers GPU-class performance on x86 CPUs for ONNX models, covering co… | 10 | 3158 | abandoned |
| mintlify/writer Mintlify Writer is an AI-powered editor plugin (VS Code extension and IntelliJ plugin) that generates docstrings and documentation from hig… | 10 | 3131 | abandoned |
| NVIDIA/ChatRTX ChatRTX is a Windows demo application for building personalized RAG chatbots on local RTX GPUs using TensorRT-LLM, NVIDIA NIM, and LlamaInd… | 10 | 3121 | abandoned |
| gragland/chatgpt-chrome-extension A Chrome extension that integrates ChatGPT into every text box on the internet, letting users invoke AI assistance via right-click without … | 32 | 2963 | abandoned |
| janhq/cortex.cpp Cortex.cpp is a C++ local AI platform that runs GGUF (llama.cpp) and ONNX models on your own machine, exposing an OpenAI-compatible HTTP AP… | 10 | 2752 | abandoned |
| npiv/chatblade Chatblade is a Python CLI tool for querying OpenAI's ChatGPT from the terminal, supporting piped input, saved prompt preambles, session rec… | 50 | 2590 | abandoned |
| Azure-Samples/graphrag-accelerator A one-click Azure deployment accelerator that hosts the GraphRAG knowledge-graph-powered RAG system as a scalable API service. It builds on… | 10 | 2409 | abandoned |
| aress31/burpgpt burpgpt is a Burp Suite extension that sends HTTP traffic to OpenAI GPT models for AI-driven passive vulnerability scanning and traffic ana… | 30 | 2351 | abandoned |
| casper-hansen/AutoAWQ AutoAWQ is a Python library implementing the Activation-aware Weight Quantization (AWQ) algorithm for 4-bit quantization of large language … | 10 | 2349 | abandoned |
| google-gemini/deprecated-generative-ai-python The deprecated Google AI Python SDK for the Gemini API, which allowed developers to integrate Google's generative AI models into Python app… | 10 | 2327 | abandoned |
| instill-ai/instill-core Instill Core is a full-stack AI infrastructure tool for data, model, and pipeline orchestration, aimed at building AI-first applications wi… | 76 | 2321 | abandoned |
| microsoft/kernel-memory Kernel Memory is a multi-modal AI service and .NET library for indexing datasets through hybrid data pipelines, enabling RAG, semantic sear… | 69 | 2235 | abandoned |
| codestoryai/aide Aide is an open-source, AI-native code editor built as a fork of VS Code, integrating agentic AI for chat-driven multi-file edits, proactiv… | 10 | 2194 | abandoned |
| run-llama/llama_deploy LlamaDeploy is a Python framework for deploying LlamaIndex agentic workflows to production, with support for multi-agent services. The proj… | 73 | 2068 | abandoned |
| intel/intel-extension-for-pytorch A Python package extending official PyTorch with Intel-specific optimizations for CPUs and GPUs, including quantization and LLM inference a… | 10 | 2012 | abandoned |
| microsoft/onnxjs ONNX.js is a JavaScript library for running pre-trained ONNX machine learning models directly in browsers and Node.js, using WebAssembly fo… | 10 | 1814 | abandoned |
| n4ze3m/dialoqbase Dialoqbase is an open-source self-hosted application for creating custom chatbots powered by a personal knowledge base, supporting many cha… | 10 | 1793 | abandoned |
| IntelLabs/fastRAG fastRAG is a Python research framework for building efficient, optimized retrieval-augmented generation (RAG) pipelines, built on Haystack … | 10 | 1785 | abandoned |
| AutoForgeAI/autoforge AutoForge is a long-running autonomous coding agent built on the Claude Agent SDK that builds complete applications across multiple session… | 57 | 1769 | abandoned |
| justLV/onju-voice A hackable AI home assistant platform that replaces the internals of a Google Nest Mini with a custom ESP32-S3 PCB, paired with a server th… | 63 | 1711 | abandoned |
| huggingface/swift-coreml-transformers A Swift library with Core ML implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT, including tokenizers and demo apps for on-device … | 10 | 1682 | abandoned |
| 0x676e67/ninja Ninja is a reverse-engineered ChatGPT proxy server written in Rust that proxies ChatGPT-API/OpenAI-API requests, handles account authentica… | 10 | 1677 | abandoned |
| terrifyzhao/bert-utils A Python library that simplifies Google's BERT to generate sentence vectors with one line of code and perform text classification and text … | 32 | 1668 | abandoned |
| Veloera/Veloera Veloera is a self-hosted AI API gateway written in Go, forked from New API, that proxies and manages access to multiple LLM providers with … | 55 | 1627 | abandoned |
| MeetKai/functionary Functionary is a chat language model and serving code that interprets and executes functions/plugins, deciding when to call tools in parall… | 69 | 1595 | abandoned |
| Binaryify/open-kimi-ppt-skill An unofficial Kimi Slides skill that let AI agents generate editable PPTD and PPTX presentations, bundled with a local browser-based editor… | 10 | 1595 | abandoned |
| ELS-RD/kernl Kernl is an open-source PyTorch model optimization library that accelerates transformer inference on GPUs with a single line of code, using… | 10 | 1584 | abandoned |
| caesarHQ/textSQL textSQL is an open-source project that converts natural language questions into SQL queries using GPT-3.5, letting users query databases wi… | 30 | 1582 | abandoned |
| Farama-Foundation/ChatArena ChatArena is a Python library providing multi-agent language game environments for large language models, built around a Markov Decision Pr… | 37 | 1558 | abandoned |
| SmartSchoolAI/ai-to-pptx Ai-to-pptx is an open-source web application (Vue frontend plus PHP backend) that uses large language models like DeepSeek to generate pres… | 10 | 1459 | abandoned |
| nod-ai/AMD-SHARK-Studio AMD-SHARK Studio is a web UI distribution for high-performance machine learning inference built on SHARK and IREE, primarily for running St… | 48 | 1451 | abandoned |
| jamesturk/scrapeghost scrapeghost is an experimental Python library that uses OpenAI's GPT models to scrape structured data from websites without writing page-sp… | 49 | 1442 | abandoned |
| idootop/migpt-next MiGPT-Next is a TypeScript library and Docker-runnable service that connects Xiaomi's XiaoAI smart speakers to OpenAI-compatible large lang… | 10 | 1442 | abandoned |
| acheong08/Bard A Python SDK and CLI that reverse engineers Google's Bard chatbot API, allowing programmatic access using session cookies. It offers both s… | 10 | 1393 | abandoned |
| prefect-archive/ControlFlow ControlFlow is a Python framework for building agentic AI workflows, letting developers define discrete observable tasks, assign specialize… | 10 | 1390 | abandoned |
| irgolic/AutoPR AutoPR is a GitHub Action that autonomously generated pull requests in response to issues, planning a fix, writing code, pushing a branch, … | 10 | 1371 | abandoned |
| terry3041/pyChatGPT An unofficial Python wrapper that lets scripts interact with ChatGPT by automating a browser session, bypassing Cloudflare anti-bot protect… | 10 | 1337 | abandoned |
| miurla/babyagi-ui BabyAGI UI is a Next.js web application that provides a ChatGPT-like interface for running and developing with the BabyAGI autonomous AI ag… | 10 | 1325 | abandoned |
| CommandCodeAI/BaseAI BaseAI is a TypeScript-first web AI framework for building serverless, composable AI agents with memory (RAG), tools, and pipes, developed … | 59 | 1275 | abandoned |
| ray-project/ray-llm RayLLM was a library for serving and deploying large language models on top of Ray, providing LLM APIs built on Ray Serve. It has been arch… | 10 | 1261 | abandoned |
| OpenRouterArchived/openrouter-runner A deprecated LLM inference engine built with vLLM by OpenRouter in its early days to serve niche fine-tuned models to early adopters. It is… | 10 | 1257 | abandoned |
| Tencent/FeatherCNN FeatherCNN is a high-performance, lightweight CNN inference library developed by Tencent, targeting ARM CPUs on mobile (iOS/Android), embed… | 23 | 1227 | abandoned |
| langchain-ai/langchain-extract A FastAPI-based web server that uses LLMs to extract structured information from text and files, with extractors defined via JSON Schema an… | 10 | 1193 | abandoned |
| kootenpv/neural_complete Neural Complete is a generative LSTM-based autocomplete system trained on Python Keras source code to help write neural network code. It co… | 32 | 1159 | abandoned |
| github/gh-copilot A GitHub CLI extension providing a chat-like AI assistant in the terminal that suggests shell commands and explains what commands do. It ha… | 10 | 1151 | abandoned |
| acheong08/ChatGPT-to-API A Go server that exposes an OpenAI-compatible chat completions API backed by ChatGPT's web interface, cycling through accounts and access t… | 10 | 1141 | abandoned |
| snowby666/poe-api-wrapper A Python API wrapper for Poe.com that provides unofficial access to various AI chatbots like GPT-4, Claude, Llama, Gemini, and Mistral thro… | 10 | 1098 | abandoned |
| google-gemini/deprecated-generative-ai-swift The deprecated Google AI Swift SDK for calling the Gemini API from Apple platform apps. It is explicitly obsolete and developers are direct… | 10 | 1075 | abandoned |
| pinecone-io/canopy Canopy is an open-source Retrieval Augmented Generation (RAG) framework and context engine built on top of the Pinecone vector database, pr… | 10 | 1031 | abandoned |
| graykode/gpt-2-Pytorch A compact PyTorch reimplementation of OpenAI's GPT-2 for text generation, loading Hugging Face's converted pre-trained weights. It provides… | 32 | 1013 | abandoned |
| OpenClaw OpenClaw is a self-hosted personal AI assistant that runs on your own devices and connects LLM models, tools, and messaging channels throug… | 78 | 387718 | active |
| microsoft/markitdown MarkItDown is a lightweight Python utility from Microsoft that converts many file formats (PDF, Office documents, images, audio, HTML, EPub… | 83 | 176487 | active |
| Dify Dify is an open-source LLM application development platform for building agentic workflows and RAG pipelines in a collaborative visual work… | 93 | 153595 | active |
| Claude Code Claude Code is an agentic coding assistant from Anthropic that runs in the terminal (and IDE, desktop, and web), reads your codebase, edits… | 83 | 143092 | active |
| ComfyUI ComfyUI is a modular, node-graph based GUI, API, and backend for running diffusion models and other generative AI models to create images, … | 94 | 130155 | stable |
| Browser Use Browser Use is a Python library that lets LLM-powered AI agents control a web browser like a human — opening pages, clicking, typing, and f… | 84 | 110969 | active |
| Supabase Supabase is an open-source Postgres development platform that bundles a hosted (or self-hosted) Postgres database with authentication, auto… | 98 | 108441 | stable |
| google-gemini/gemini-cli Gemini CLI is an open-source AI agent from Google that brings Gemini models directly into the terminal for coding, debugging, and workflow … | 82 | 106698 | active |
| nexu-io/open-design OpenDesign is an open-source, local-first desktop application that turns coding agents like Claude Code, Codex, Cursor, and DeepSeek Harnes… | 81 | 91793 | active |
| infiniflow/ragflow RAGFlow is an open-source Retrieval-Augmented Generation (RAG) engine that combines deep document understanding with agent orchestration to… | 93 | 89328 | active |
| odysseus-dev/odysseus Odysseus is a self-hosted, privacy-first AI workspace that combines chat, autonomous agents, tools/MCP integration, model serving, and deep… | 58 | 86318 | active |
| 666ghj/MiroFish MiroFish is a Python-based swarm intelligence prediction engine that builds high-fidelity parallel digital worlds populated by thousands of… | 70 | 71572 | active |
| santifer/career-ops career-ops is an open-source, local-first AI job search system that runs inside AI coding CLIs (Claude Code, Codex, OpenCode, etc.). It sca… | 81 | 68683 | active |
| Mintplex-Labs/anything-llm AnythingLLM is an all-in-one, local-first AI application for chatting with your documents, running AI agents, and building workflows entire… | 95 | 65257 | active |
| ZhuLinsen/daily_stock_analysis An LLM-powered stock analysis application that aggregates multi-source market data, news, and fundamentals for A-share, HK, US, Japanese, K… | 82 | 64010 | active |
| Pathway Pathway is a Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG applications. It provides a unified Py… | 97 | 62383 | active |
| sansan0/TrendRadar TrendRadar is a self-hostable AI-powered public opinion and trending-news monitor that aggregates hot topics across multiple platforms and … | 61 | 61855 | active |
| upstash/context7 Context7 is an MCP server and hosted platform by Upstash that pulls up-to-date, version-specific library documentation and code examples di… | 82 | 61268 | active |
| MemPalace/mempalace MemPalace is a local-first AI memory system that stores conversation history and project files as verbatim text and retrieves them via sema… | 81 | 58666 | active |
| usestrix/strix Strix is an open-source AI penetration testing tool that deploys autonomous agents to find, validate, and fix application vulnerabilities. … | 84 | 58564 | active |
| BerriAI/litellm LiteLLM is an open-source AI gateway and Python SDK that provides a unified OpenAI-format interface to 100+ LLM providers (OpenAI, Anthropi… | 91 | 57340 | active |
| microsoft/VibeVoice VibeVoice is Microsoft's open-source frontier voice AI framework combining a next-token diffusion text-to-speech model for expressive, long… | 60 | 53225 | active |
| calesthio/OpenMontage OpenMontage is an open-source agentic video production system that turns AI coding assistants (Claude Code, Codex, Cursor, Copilot) into fu… | 59 | 51272 | active |
| CLIProxyAPI A Go-based proxy server that exposes OpenAI/Gemini/Claude/Codex-compatible API endpoints backed by CLI AI assistant accounts (Claude Code, … | 81 | 48891 | active |
| mudler/LocalAI LocalAI is an open-source, self-hosted AI inference runtime that runs LLMs, vision, speech, image, and video models behind OpenAI/Anthropic… | 93 | 48696 | active |
| oobabooga/textgen TextGen is an open-source desktop application for running large language models locally with full privacy and no telemetry. It provides a c… | 88 | 47585 | active |
| JeecgBoot JeecgBoot is an enterprise-grade AI low-code platform for Java that combines low-code and zero-code development with AI Skills that generat… | 95 | 47483 | active |
| HKUDS/nanobot nanobot is an ultra-lightweight, self-hosted personal AI agent framework written in Python. It provides a WebUI, terminal, and chat-app int… | 81 | 47427 | active |
| Kong Gateway Kong Gateway is a cloud-native, high-performance API gateway that proxies, routes, load-balances, and secures API, LLM, and MCP traffic, ex… | 91 | 44045 | stable |
| ray-project/ray Ray is a unified open-source framework for scaling AI and Python applications, consisting of a core distributed runtime (tasks, actors, obj… | 99 | 43614 | stable |
| danny-avila/LibreChat LibreChat is an open-source, self-hosted web application providing a unified ChatGPT-style interface for multiple AI providers (OpenAI, Ant… | 84 | 42491 | active |
| Agno Agno is a Python SDK and runtime (AgentOS) for building, running, and managing full agent platforms, including agents, teams, and workflows… | 95 | 41938 | active |
| hpcaitech/ColossalAI Colossal-AI is an integrated large-scale deep learning training system built on PyTorch that provides efficient parallelization techniques … | 74 | 41441 | active |
| Hmbown/CodeWhale Codewhale is an open-source, terminal-native coding agent written in Rust that reads your repository, edits files, runs commands, and works… | 78 | 40866 | active |
| 2noise/ChatTTS ChatTTS is a generative text-to-speech model optimized for conversational dialogue scenarios such as LLM assistants, supporting English and… | 69 | 39797 | active |
| datalab-to/marker Marker is a Python library and CLI tool that converts PDFs, images, and office documents (DOCX, PPTX, XLSX, EPUB, HTML) into markdown, JSON… | 92 | 39295 | active |
| yudao (ruoyi-vue-pro) Yudao (ruoyi-vue-pro) is an open-source rapid development platform and admin management system built on Spring Boot, MyBatis Plus, and Vue/… | 97 | 38991 | active |