function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| kellyvv/PhoneClaw PhoneClaw is a mobile-native local AI agent framework that turns phones into on-device agent runtimes, running Gemma models via LiteRT and … | 77 | 1220 | active |
| AllAboutAI-YT/easy-local-rag A Python application providing 100% local retrieval-augmented generation (RAG) using Ollama for LLM inference and embeddings. It supports i… | 15 | 1220 | active |
| uzairansaruzi/hermex Hermex is a native SwiftUI iPhone app for controlling a self-hosted Hermes (hermes-webui) AI agent from your phone. It provides streaming c… | 79 | 1219 | active |
| MaoXiaoYuZ/Long-Novel-GPT Long-Novel-GPT is a self-hosted web application that uses LLMs and RAG to generate and revise long-form novels through a top-down outline-c… | 48 | 1218 | active |
| vincanger/coverlettergpt A full-stack web application that generates tailored cover letters from a job description and the user's resume using OpenAI's GPT models. … | 44 | 1218 | active |
| xenodium/chatgpt-shell An Emacs package providing a multi-LLM shell and chat interface supporting ChatGPT, Claude, DeepSeek, Gemini, Kagi, Ollama, and Perplexity,… | 73 | 1216 | active |
| tonyqinatcmu/SlideBot-AI SlideBot AI is an AI-powered presentation generator that turns a topic, outline, or uploaded materials (documents, spreadsheets, meeting re… | 44 | 1214 | active |
| modal-labs/quillman QuiLLMan is a voice chat application built on Kyutai's Moshi speech-to-speech language model, deployed serverlessly on Modal with a FastAPI… | 68 | 1213 | active |
| sums001/Windows-Copilot-API A Python library and local server that reverse engineers Microsoft Copilot's free web chat into an OpenAI-compatible REST API, giving acces… | 53 | 1213 | active |
| wanxingai/LightAgent LightAgent is an ultra-lightweight open-source Python framework for building LLM-powered agents with tools, persistent memory, guardrails, … | 87 | 1211 | active |
| OpenTSLM/OpenTSLM OpenTSLM is a family of Time-Series Language Models that integrate time series as a native modality into pretrained LLMs (Llama, Gemma), en… | 61 | 1211 | active |
| TIGER-AI-Lab/OpenResearcher OpenResearcher is a fully open-source pipeline for synthesizing long-horizon deep research trajectories using LLM agents with retrieval and… | 54 | 1211 | active |
| Shopify/roast Roast is a Ruby-based domain-specific language for building structured AI workflows from composable building blocks called cogs. It orchest… | 82 | 1210 | active |
| Michaelliv/pi-dynamic-workflows A Pi extension that adds a `workflow` tool enabling the model to write JavaScript scripts that fan out work across isolated subagents and s… | 70 | 1209 | active |
| openai/codex-action A GitHub Action that runs OpenAI's Codex CLI inside GitHub Actions workflows, installing the CLI and routing API calls through a secure pro… | 62 | 1209 | active |
| Zefan-Cai/R-KV R-KV is a training-free, redundancy-aware KV cache compression method for reasoning LLMs, discarding repetitive tokens on-the-fly during de… | 60 | 1209 | active |
| ICT-FinD-Lab/alphagen AlphaGen is a Python research library that automatically generates formulaic alpha (predictive) stock factors using reinforcement learning,… | 70 | 1204 | active |
| JuneYaooo/gpt-image2-ppt-skills A Claude Code / OpenClaw skill that generates polished 16:9 presentations using OpenAI's gpt-image-2, rendering each slide as a complete vi… | 59 | 1203 | active |
| dulikaifazr/Cougar-CLI Cougar CLI is an AI programming agent that runs in the terminal, letting developers interact with LLMs to write and manage code from the co… | 45 | 1203 | active |
| hkjarral/AVA-AI-Voice-Agent-for-Asterisk An open-source AI voice agent that integrates with Asterisk/FreePBX phone systems via Audiosocket/RTP, built in Python with a modular pipel… | 85 | 1202 | active |
| NVIDIA/kvpress kvpress is a Python library from NVIDIA that implements multiple KV cache compression methods and benchmarks for long-context LLM inference… | 86 | 1201 | active |
| Mangi-11/Eta Eta is a third-party system-level AI agent for Android that uses Root and LSPosed to bypass app sandboxing, giving connected LLMs (BYOK: Ch… | 81 | 1201 | active |
| nishuzumi/gemini-teacher A Python CLI application that acts as an English speaking practice assistant powered by Google Gemini. It listens to your speech via microp… | 68 | 1200 | active |
| basetenlabs/truss Truss is a Python CLI and packaging framework for deploying and serving AI/ML models in production, primarily on the Baseten platform. It h… | 96 | 1199 | active |
| kizuna-ai-lab/sokuji Sokuji is a cross-platform real-time two-way speech translation app for bilingual meetings, available as a desktop application (Windows, ma… | 83 | 1199 | active |
| Morgensonne/EditDeck EditDeck is an end-to-end pipeline that turns plain-language requirements into structured slide outlines, rendered slide images, standard P… | 50 | 1199 | active |
| Artelnics/opennn OpenNN is an open-source C++ library for building, training, and deploying neural networks for advanced analytics. It is dependency-free, o… | 97 | 1198 | active |
| openvinotoolkit/nncf NNCF is Intel's Neural Network Compression Framework, a Python library providing post-training and training-time compression algorithms (qu… | 94 | 1197 | active |
| CreminiAI/skillpack SkillPack is an open-source CLI and runtime for packaging AI skills (prompts, scripts, tools) into distributable zip bundles that run as lo… | 58 | 1197 | active |
| brainlid/langchain An Elixir framework for building LLM-powered applications, providing abstractions and integrations for chat models like OpenAI, Anthropic C… | 96 | 1196 | active |
| sozercan/kubectl-ai A kubectl plugin written in Go that generates and applies Kubernetes manifests from natural-language prompts using OpenAI GPT or OpenAI API… | 22 | 1196 | active |
| qualcomm/ai-hub-models Qualcomm AI Hub Models is a curated collection of 300+ state-of-the-art machine learning models (vision, audio, speech, generative AI) pre-… | 89 | 1195 | active |
| cvs-health/uqlm UQLM is a Python library for detecting LLM hallucinations using uncertainty quantification techniques. It provides five categories of score… | 86 | 1195 | active |
| modu-ai/moai-adk MoAI-ADK is an agentic development harness for Claude Code that enforces SPEC-driven plan/run/sync workflows, TRUST 5 quality gates, and mu… | 80 | 1195 | active |
| mckaywrigley/ai-code-translator A web application that uses AI (LLMs) to translate source code from one programming language to another. It is a Next.js/TypeScript app tha… | 30 | 4169 | maintenance |
| heyhuynhgiabuu/proxypal ProxyPal is a native desktop app (Tauri-based) that wraps CLIProxyAPI to expose AI subscriptions like Claude, ChatGPT, Gemini, and GitHub C… | 78 | 1193 | active |
| Lakr233/FlowDown FlowDown is a native AI chat client for iPhone, iPad, and Mac built with Swift and UIKit. It connects to OpenAI-compatible APIs, self-hoste… | 84 | 1192 | active |
| jncraton/languagemodels A Python library providing simple building blocks for running large language models locally with as little as 512MB of RAM. It offers instr… | 60 | 1192 | active |
| microsoft/RecAI RecAI is a Microsoft research project exploring ways to integrate large language models into recommender systems (LLM4Rec). It includes a c… | 56 | 1192 | active |
| ngxson/wllama wllama is a WebAssembly binding for llama.cpp that enables running LLM inference entirely in the browser, with WebGPU support, multimodal i… | 92 | 1191 | active |
| symfony/ai Symfony AI is a set of PHP components and Symfony bundles that integrate AI capabilities into PHP applications, including a unified interfa… | 87 | 1191 | active |
| SimonSchubert/Kai Kai 9000 is an open-source, cross-platform AI assistant built with Kotlin Multiplatform that runs on Android, iOS, Windows, Mac, Linux, and… | 83 | 1191 | active |
| sashiko-dev/sashiko Sashiko is an agentic code review system for the Linux kernel that ingests patches from mailing lists, GitHub PRs, GitLab MRs, or local git… | 60 | 1191 | active |
| zai-org/VisualGLM-6B VisualGLM-6B is an open-source multimodal conversational language model supporting images, Chinese, and English, built on ChatGLM-6B with a… | 30 | 4154 | maintenance |
| linux-do/override A Go-based local proxy service that overrides GitHub Copilot's API endpoints in VSCode and JetBrains IDEs, redirecting code completion and … | 15 | 1187 | active |
| huggingface/llm.nvim llm.nvim is a Neovim plugin that brings LLM-powered features like ghost-text code completion to the editor, using llm-ls as its backend. It… | 67 | 1186 | active |
| clacky-ai/openclacky OpenClacky is an open-source (MIT) AI agent platform written in Ruby that runs locally, offering a CLI and Web UI for completing tasks via … | 79 | 1184 | active |
| polyuiislab/infiAgent infiAgent (MLA) is a Python agent framework for building domain-specific AI agents that run for days on complex tasks without context degra… | 72 | 1184 | active |
| madox2/vim-ai A Vim/Neovim plugin that adds AI capabilities such as code generation, text editing, and interactive ChatGPT conversations directly in the … | 62 | 1184 | active |
| anthropics/anthropic-sdk-go The official Go SDK for accessing Anthropic's Claude language model APIs. It provides typed clients for messages, streaming, and other Clau… | 87 | 1183 | active |
| googleapis/go-genai The official Google Gen AI SDK for Go, providing a client interface to Google's generative models such as Gemini via the Gemini Developer A… | 85 | 1183 | active |
| SamuelZ12/longcut LongCut is a Next.js web application that turns long-form YouTube videos into a structured learning workspace with AI-generated highlight r… | 56 | 1183 | active |
| GAIR-NLP/ASI-Arch A multi-agent framework that lets an LLM autonomously conduct end-to-end research on neural network architecture discovery, specifically li… | 43 | 1183 | active |
| orpatashnik/StyleCLIP Official implementation of StyleCLIP, a method for text-driven manipulation of StyleGAN-generated imagery using CLIP. It provides three app… | 32 | 4121 | maintenance |
| goodroot/hyprwhspr hyprwhspr is a native Linux system-wide speech-to-text dictation application supporting local models (Whisper, Parakeet, Cohere) with optio… | 85 | 1182 | active |
| shawwn/llama-dl A shell script that provides high-speed curl-based downloads of Meta's LLaMA model weights (7B, 13B, 30B, 65B), fetching all weights in und… | 30 | 4114 | maintenance |
| Facico/Chinese-Vicuna Chinese-Vicuna is a low-resource LLaMA+LoRA solution for building Chinese instruction-following language models, structured after Alpaca. I… | 38 | 4113 | maintenance |
| BiFangKNT/mtga MTGA is a local proxy application that lets IDEs with fixed model providers (such as Trae) use arbitrary LLM backends. It exposes an OpenAI… | 76 | 1179 | active |
| fmaclen/hollama Hollama is a minimal LLM chat application that runs entirely in the browser, connecting to Ollama and OpenAI servers with all data stored l… | 53 | 1179 | active |
| DAMO-NLP-SG/VideoLLaMA3 VideoLLaMA 3 is a frontier multimodal foundation model for image and video understanding, released with checkpoints, inference code, and de… | 37 | 1179 | active |
| langchain-ai/openevals OpenEvals is a library of readymade evaluators for testing LLM applications, including LLM-as-judge evaluators and reference-based/referenc… | 86 | 1178 | active |
| rudrankriyam/Foundation-Models-Framework-Lab A native iOS and macOS workbench app for learning, testing, and evaluating Apple's Foundation Models framework. It provides editable recipe… | 82 | 1177 | active |
| WuKongOpenSource/Wukong-AICRM Wukong AICRM is an open-source AI-powered CRM positioned as a Salesforce alternative, built in Java. It combines customer management, task … | 63 | 1176 | active |
| Soul-AILab/SoulX-LiveAct SoulX-LiveAct is the official inference code for a real-time human animation framework that generates lifelike, audio/multimodal-controlled… | 54 | 1176 | active |
| kleneway/pastemax PasteMax is a cross-platform desktop application for browsing a repository's files, selecting relevant ones, and copying their contents for… | 62 | 1174 | active |
| Opencode-DCP/opencode-dynamic-context-pruning A plugin for OpenCode that dynamically prunes and compresses conversation context to reduce token usage when working with LLMs. It replaces… | 78 | 4085 | maintenance |
| baichuan-inc/Baichuan2 Baichuan 2 is a family of open large language models (7B and 13B, Base and Chat variants with 4-bit quantized versions) trained by Baichuan… | 28 | 4084 | maintenance |
| MediaBrain-SJTU/MING MING (明医) is a Chinese medical consultation large language model fine-tuned on medical instruction data, with variants built on bloomz-7b a… | 40 | 1173 | active |
| nari-labs/dia2 Dia2 is a streaming dialogue text-to-speech model from Nari Labs that can generate conversational audio in realtime without needing the ful… | 41 | 1172 | active |
| Azure/GPT-RAG GPT-RAG is an Azure solution accelerator providing architecture templates and deployment assets for enterprise-scale Retrieval-Augmented Ge… | 90 | 1170 | active |
| Zen4-bit/Proxima Proxima is a local-first Electron application that runs a multi-AI Model Context Protocol (MCP) server, routing ChatGPT, Claude, Gemini, an… | 75 | 1169 | active |
| whyiyhw/chatgpt-wechat A self-hosted Go application that lets users safely use LLM assistants (ChatGPT, Gemini, DeepSeek, Dify workflows) inside WeChat by relayin… | 59 | 1169 | active |
| nv-tlabs/LLaMA-Mesh LLaMA-Mesh is a fine-tuned large language model from NVIDIA Research that generates and understands 3D meshes by representing vertex coordi… | 28 | 1166 | active |
| lmstudio-ai/mlx-engine mlx-engine is the Apple MLX-based LLM inference engine that powers LM Studio on Mac, built on mlx-lm with support for vision models via mlx… | 67 | 1165 | active |
| RhythmicWave/NovelForge NovelForge is an AI-assisted long-form novel writing application built in Python, using a card-based creation model with JSON Schema-driven… | 85 | 1157 | active |
| techjarves/OpenClaude-Portable A portable AI coding agent that runs Claude Code-style functionality from a USB drive or any folder with no installation, bundling a self-c… | 50 | 1157 | active |
| nico-martin/gemma4-browser-extension A Chrome extension that runs an on-device AI agent powered by Transformers.js and Gemma 4 via WebGPU, entirely locally with no data sent to… | 60 | 1156 | active |
| wuyoscar/Internal-Safety-Collapse ISC-Bench/TVD is a research framework for studying 'Internal Safety Collapse' in frontier LLMs, where agents placed in adversarial codespac… | 59 | 1155 | active |
| character-ai/prompt-poet Prompt Poet is a Python library for designing LLM prompts using YAML and Jinja2 templates in a low-code approach. It renders dynamic, struc… | 53 | 1155 | active |
| ricklamers/shell-ai Shell-AI (shai) is a Python CLI utility that converts natural language descriptions into shell command suggestions using LLMs via LangChain… | 47 | 1154 | active |
| liujuntao123/smart-mermaid Smart Mermaid is an AI-powered web application that converts natural language text into Mermaid diagram code and renders it as visual chart… | 41 | 1154 | active |
| ALwrity/ALwrity ALwrity is an open-source, AI-first digital marketing platform built in Python that combines content strategy planning, multimodal AI conte… | 79 | 1153 | active |
| AI-QL/tuui TUUI is an open-source desktop MCP client and local AI playground built with Vue3 and Electron, serving as a unified interface for Model Co… | 74 | 1153 | active |
| apple/ml-clara CLaRa is Apple's open-source end-to-end Retrieval-Augmented Generation model that compresses documents into continuous latent representatio… | 43 | 1153 | active |
| Libr-AI/OpenFactVerification Loki is an open-source Python tool that automates fact verification by decomposing texts into claims, retrieving evidence via search, and u… | 15 | 1153 | active |
| shyamsn97/mario-gpt MarioGPT is a Python library and finetuned GPT2 model that generates playable Super Mario Bros levels from text prompts. It accompanies the… | 21 | 1152 | active |
| i-am-bee/agentstack Agent Stack is open-source infrastructure (hosted by the Linux Foundation) for turning AI agents into running HTTP services, built on the A… | 80 | 1149 | active |
| cloudflare/ai A monorepo of TypeScript packages and examples for building AI-powered applications on Cloudflare. It provides Vercel AI SDK and TanStack A… | 80 | 1148 | active |
| HammerGPT/Hyper-Alpha-Arena A self-hosted AI-powered crypto trading platform for Hyperliquid and Binance Futures perpetual contracts. It combines LLM-driven trading ag… | 54 | 1148 | active |
| agent-topia/evolving_personality JPAF is a Python framework that gives LLM agents structured, evolving MBTI personalities based on Carl Jung's psychological types. It uses … | 48 | 1145 | active |
| Codium-ai/AlphaCodium Official implementation of the AlphaCodium paper, a test-based, multi-stage, iterative flow for LLM code generation on competitive programm… | 26 | 3965 | maintenance |
| terobox/ChatGPT-API-Faucet The frontend code for ChatGPT API Faucet, a public-welfare website where AI developers can claim $1.00 of free API tokens every 24 hours fo… | 35 | 1144 | active |
| ovg-project/kvcached kvcached is a Python library that brings OS-style virtual memory abstraction to KV cache management for LLM serving and training on shared … | 74 | 1143 | active |
| chrishayuk/larql LARQL decompiles transformer models into a queryable 'vindex' (vector index) format and provides LQL, a SQL-like query language, to browse,… | 82 | 1142 | active |
| KoboldAI/KoboldAI-Client KoboldAI Client is a browser-based front-end for AI-assisted writing and text adventure gameplay that connects to multiple local and remote… | 23 | 3944 | maintenance |
| laravel/ai The Laravel AI SDK is a PHP package offering a unified, expressive API for interacting with AI providers such as OpenAI, Anthropic, and Gem… | 84 | 1139 | active |
| DeepMyst/Mysti Mysti is a VS Code extension that orchestrates a team of AI coding agents from 12 providers (Claude Code, OpenAI Codex, Gemini, Copilot, Cl… | 71 | 1136 | active |
| IST-DASLab/marlin Marlin is a highly optimized FP16xINT4 matrix multiplication CUDA kernel for LLM inference that achieves near-ideal 4x speedups at batch si… | 26 | 1136 | active |
| Gentleman-Programming/gentleman-guardian-angel Gentleman Guardian Angel (gga) is a pure-Bash CLI tool that runs AI-powered code reviews on staged files via a git pre-commit hook, using a… | 77 | 1135 | active |
| RUC-NLPIR/DeepAgent DeepAgent is a general reasoning agent framework that dynamically searches for and uses tools from large toolsets (e.g., 16,000+ RapidAPIs)… | 52 | 1134 | active |