function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| yan5xu/ququ QuQu is an open-source, free desktop voice dictation app built with Electron, designed as a privacy-first Wispr Flow alternative optimized … | 31 | 2271 | active |
| aixcoder-plugin/aiXcoder-7B Official repository for aiXcoder-7B, an open-weights 7B-parameter code large language model trained on 1.2T tokens for code completion, gen… | 29 | 2271 | active |
| yawiii/ComfyUI-Prompt-Assistant A ComfyUI plugin that provides an all-in-one prompt assistant, connecting to cloud LLM/VLM APIs (Zhipu, SiliconFlow, Gemini, Baidu) and loc… | 70 | 2269 | active |
| mainframecomputer/fullmoon-ios fullmoon is a free, open-source iOS/iPadOS/macOS/visionOS app for chatting with local large language models, built on Apple's MLX Swift fra… | 33 | 2267 | active |
| qhjqhj00/MemoRAG MemoRAG is a Python RAG framework that uses a super-long memory model to build a global understanding of an entire corpus (up to ~1M tokens… | 43 | 2265 | active |
| numman-ali/cc-mirror CC-MIRROR is an npm-distributed CLI that creates isolated, provider-native variants of Claude Code wired to alternative LLM providers such … | 72 | 2259 | active |
| K-Dense-AI/claude-scientific-writer An AI-powered deep research and scientific writing tool that generates publication-ready papers, reports, posters, grant proposals, and lit… | 82 | 2254 | active |
| aattaran/deepclaude A shell/PowerScript wrapper that launches Claude Code's autonomous coding agent loop backed by cheaper Anthropic-compatible LLM providers l… | 56 | 2254 | active |
| PrismML-Eng/Bonsai-demo A demo repository for running PrismML's Bonsai family of 1-bit and ternary-quantized language models locally via llama.cpp and MLX. It prov… | 59 | 2252 | active |
| rushindrasinha/youtube-shorts-pipeline Verticals v3 (repo youtube-shorts-pipeline) is a Python CLI that automates producing and publishing YouTube Shorts: it researches a topic, … | 61 | 2251 | active |
| jwadow/kiro-gateway A self-hosted proxy API gateway for the Kiro API (Amazon Q Developer / AWS CodeWhisperer) that exposes Claude and other models through Open… | 65 | 2241 | active |
| GalTransl/GalTransl GalTransl is an automated translation tool for visual novels (galgames) that leverages large language models like GPT-4, Claude, Deepseek, … | 84 | 2239 | active |
| datapizza-labs/datapizza-ai Datapizza AI is a Python framework for building production-ready generative AI applications with agents, LLM clients, and RAG pipelines. It… | 68 | 2239 | active |
| Doriandarko/deepseek-engineer DeepSeek Engineer is a Python command-line coding assistant that connects to the DeepSeek API and uses function calling to read, create, an… | 32 | 2238 | active |
| apconw/Aix-DB Aix-DB is an AI-powered data analysis system (ChatBI) built on LangChain/LangGraph with an MCP Skills multi-agent architecture, converting … | 72 | 2234 | active |
| microsoft/LLaVA-Med LLaVA-Med is a large language-and-vision assistant fine-tuned for the biomedicine domain, built on the LLaVA multimodal architecture. It su… | 40 | 2231 | active |
| baturyilmaz/wordpecker-app WordPecker is a personalized language-learning web application that combines Duolingo-style lessons with user-curated vocabulary lists. It … | 36 | 2229 | active |
| antoinezambelli/forge Forge is a Python framework providing a reliability layer for self-hosted LLM tool-calling, with guardrails like rescue parsing, retry nudg… | 82 | 2227 | active |
| e2b-dev/open-computer-use An open-source AI agent that controls a secure cloud Linux desktop (via E2B Desktop Sandbox) using keyboard, mouse, and shell commands, pow… | 63 | 2224 | active |
| floneum/kalosm Kalosm is a Rust ecosystem of crates providing simple interfaces for running pre-trained language, audio, and image models locally or remot… | 64 | 2223 | active |
| ai-christianson/RA.Aid RA.Aid is a standalone autonomous coding agent built on LangGraph that researches, plans, and implements multi-step software development ta… | 52 | 2223 | active |
| tnm/zclaw zclaw is a personal AI assistant firmware written in C that runs entirely on ESP32 microcontrollers within a strict 888 KiB all-in firmware… | 66 | 2221 | active |
| zakirkun/deep-eye Deep Eye is an AI-driven penetration testing CLI that orchestrates multiple LLM providers (OpenAI, Claude, Gemini, OLLAMA, Groq, and others… | 68 | 2219 | active |
| OpenSPG/openspg OpenSPG is a knowledge graph engine built on the SPG (Semantic-enhanced Programmable Graph) framework, developed by Ant Group with OpenKG. … | 38 | 2217 | active |
| getkimchi/kimchi Kimchi is a terminal-based AI coding agent CLI that autonomously executes multi-step development tasks like refactoring, writing tests, and… | 77 | 2214 | active |
| lessweb/deepcode-cli Deep Code CLI is an open-source terminal-based AI coding assistant optimized for DeepSeek-V4 models, supporting deep thinking mode, reasoni… | 81 | 2211 | active |
| langchain-ai/executive-ai-assistant Executive AI Assistant (EAIA) is an open-source AI agent built with LangGraph that performs the job of an executive assistant, primarily by… | 10 | 2208 | active |
| AnotiaWang/deep-research-web-ui A web UI for the dzhng/deep-research project that performs iterative, AI-driven research by combining search engines, web scraping, and LLM… | 66 | 2205 | active |
| NX-AI/xlstm Official PyTorch implementation of xLSTM, an extended Long Short-Term Memory recurrent architecture with exponential gating and matrix memo… | 59 | 2198 | active |
| TransWithAI/Faster-Whisper-TransWithAI-ChickenRice A high-performance audio/video transcription and translation application built on Faster Whisper, optimized for Japanese-to-Chinese transla… | 77 | 2196 | active |
| LigphiDonk/academic-figure-generator A self-hosted AI-powered platform that generates high-quality academic paper figures: users upload a paper (PDF/DOCX/TXT), Claude analyzes … | 62 | 2190 | active |
| pretzelai/pretzelai Pretzel is an open-source fork of Jupyter Notebooks that adds AI code generation, inline tab completion, sidebar chat, and error fixing whi… | 16 | 2188 | active |
| MarkPDFdown/markpdfdown MarkPDFDown is a Python CLI tool that converts PDF documents and images into clean Markdown using multimodal large language models via Lite… | 60 | 2184 | active |
| numman-ali/opencode-openai-codex-auth An OpenCode plugin that authenticates via OpenAI's official ChatGPT OAuth flow, letting users with ChatGPT Plus/Pro subscriptions access GP… | 55 | 2183 | active |
| TimmyOVO/deepseek-ocr.rs A Rust implementation of the DeepSeek-OCR inference stack with multiple OCR/VLM backends (DeepSeek-OCR, PaddleOCR-VL, DotsOCR), DSQ quantiz… | 60 | 2182 | active |
| 1sdv/TripStar TripStar is an AI-powered travel planning application built on the HelloAgents multi-agent framework, using LLMs to generate personalized t… | 59 | 2181 | active |
| Jazee6/cloudflare-ai-web A self-hosted web application for chatting with multiple AI models via Cloudflare Workers AI and AI Gateway (including Gemini). Built with … | 81 | 2174 | active |
| intel/intel-extension-for-transformers Intel's toolkit for accelerating transformer-based GenAI/LLM workloads on Intel platforms, offering state-of-the-art compression (e.g., INT… | 10 | 2174 | active |
| MCPJam/inspector MCPJam is an open-source testing and evaluation platform for MCP server developers, providing an inspector playground, OAuth/EMA debugger, … | 82 | 2172 | active |
| mozilla-ai/any-llm any-llm is a Python library from Mozilla.ai that provides a single, unified interface for calling many LLM providers (OpenAI, Anthropic, Mi… | 80 | 2171 | active |
| MoonshotAI/MoBA MoBA (Mixture of Block Attention) is a PyTorch implementation of a trainable block-sparse attention mechanism for long-context large langua… | 27 | 2169 | active |
| withcatai/node-llama-cpp node-llama-cpp is a Node.js library providing bindings to llama.cpp for running LLMs locally, with pre-built binaries and automatic GPU sup… | 93 | 2162 | active |
| warden-protocol/wardenprotocol Warden Protocol is a blockchain protocol monorepo providing a network layer for AI agents, including an agentic wallet, a peer-to-peer perm… | 92 | 2161 | active |
| op7418/Youtube-clipper-skill A Claude Code skill that downloads YouTube videos and uses AI to generate semantic chapters, clip segments, translate subtitles into biling… | 44 | 2159 | active |
| Nutlope/notesGPT NotesGPT is an open-source AI-powered voice note-taking web app that records voice notes, transcribes them with Whisper, and generates summ… | 68 | 2153 | active |
| boson-ai/higgs-audio Higgs Audio is a text-audio foundation model project from Boson AI providing code and weights for conversational text-to-speech with zero-s… | 57 | 8329 | maintenance |
| ggml-org/llama.vim A Vim/Neovim plugin that provides LLM-assisted code and text completion via a local llama.cpp server. It offers fill-in-the-middle (FIM) au… | 78 | 2149 | active |
| rupeshs/fastsdcpu FastSD CPU is a Python application that runs Stable Diffusion image generation quickly on CPUs and Intel AI PCs using Latent Consistency Mo… | 78 | 2143 | active |
| btahir/open-deep-research An open-source web application that replicates Gemini's Deep Research by searching the web, extracting page content, and generating AI-writ… | 10 | 2141 | active |
| Agency Orchestrator Agency Orchestrator is a multi-agent YAML workflow engine and desktop client that lets a single sentence trigger multiple AI expert roles t… | 81 | 2134 | active |
| PKU-YuanGroup/LLaVA-CoT LLaVA-CoT is an 11B vision language model with training, inference, and dataset-generation code for spontaneous step-by-step multimodal rea… | 47 | 2132 | active |
| tairov/llama2.mojo A single-file implementation of Llama 2 transformer inference written in pure Mojo, leveraging SIMD and vectorization for fast CPU inferenc… | 57 | 2125 | active |
| google-gemini/genai-processors GenAI Processors is a lightweight Python library for building modular, asynchronous, and composable AI pipelines around Gemini and other ge… | 76 | 2120 | active |
| fuxicodex/Fuxi FuXi is a terminal-based AI coding agent built in Go that reads code, edits files, runs commands, and drives tools through a rich TUI using… | 79 | 2117 | active |
| ModelEngine-Group/fit-framework FIT is an enterprise-grade AI development framework for the Java ecosystem, combining a polyglot function engine (Java/Python/C++), a strea… | 66 | 2117 | active |
| zilliztech/GPTCache GPTCache is a Python library for building a semantic cache that stores and retrieves LLM responses based on embedding similarity, reducing … | 35 | 8172 | maintenance |
| fastmachinelearning/hls4ml hls4ml is a Python package that converts machine learning models from Keras, PyTorch, and ONNX into high-level synthesis (C++) code for FPG… | 82 | 2114 | active |
| deepspeedai/DeepSpeed-MII DeepSpeed-MII is a Python library for high-throughput, low-latency large language model inference, built on DeepSpeed. It provides blocked … | 43 | 2111 | active |
| ozgrozer/ai-renamer A Node.js CLI tool that uses local AI models (via Ollama or LM Studio) or OpenAI to intelligently rename files based on their contents, inc… | 26 | 2111 | active |
| LC1332/Chat-Haruhi-Suzumiya An open-source role-playing chatbot project that revives anime characters like Haruhi Suzumiya using large language models, mimicking their… | 29 | 2107 | active |
| OpenAutoCoder/Agentless Agentless is a Python framework that automatically solves software development problems (bug fixing) using LLMs without an agent loop. It f… | 14 | 2104 | active |
| QwenLM/Qwen2-Audio Official repository for Qwen2-Audio, a 7B-parameter large audio-language model from Alibaba Cloud that accepts audio inputs and responds to… | 31 | 2099 | active |
| anthropics/anthropic-sdk-typescript The official TypeScript/JavaScript SDK for accessing Anthropic's Claude language model APIs from server-side applications. It provides a ty… | 94 | 2096 | active |
| tsingyuai/scientify Scientify is an OpenClaw plugin that automates end-to-end scientific research workflows, continuously ingesting new papers, evolving hypoth… | 67 | 2094 | active |
| ex3ndr/llama-coder Llama Coder is a self-hosted GitHub Copilot replacement implemented as a VS Code extension. It uses Ollama running CodeLlama or Stable Code… | 27 | 2089 | active |
| bigcode-project/starcoder2 StarCoder2 is a family of open code generation language models (3B, 7B, 15B) trained on 600+ programming languages from The Stack v2, with … | 26 | 2087 | active |
| vitoplantamura/OnnxStream A lightweight C++ inference library for ONNX models that streams weights to run large models in very little memory, accelerated by XNNPACK.… | 59 | 2086 | active |
| itigges22/ATLAS ATLAS is a local-first coding agent that wraps compact open models with planning, candidate generation, quality scoring, sandboxed testing,… | 76 | 2078 | active |
| MiniMax-AI/cli The official CLI for the MiniMax AI Platform, written in TypeScript, that generates text, images, video, speech, and music from the termina… | 81 | 2073 | active |
| stepfun-ai/Step-3.5-Flash Step 3.5 Flash is an open-source 196B-parameter sparse Mixture-of-Experts foundation model (11B active per token) from StepFun, focused on … | 49 | 2070 | active |
| kimjammer/Neuro A local recreation of the Neuro-Sama AI VTuber that runs open-source LLMs on consumer hardware, combining realtime speech-to-text, text-to-… | 28 | 2070 | active |
| jd-opensource/OxyGent OxyGent is an open-source Python framework that unifies tools, models, and agents into modular 'Oxy' components for building multi-agent sy… | 60 | 2067 | active |
| BetaStreetOmnis/xhs_ai_publisher A desktop application for AI-powered content creation and automated publishing on Xiaohongshu (Rednote), built with PyQt5, FastAPI, and Pla… | 70 | 2064 | active |
| chi111i/BypassAIGC A self-hosted AI academic writing assistant that polishes and rewrites papers using LLM APIs (e.g., Gemini via OpenAI-compatible endpoints)… | 68 | 2056 | active |
| espressif/esp-claw ESP-Claw is Espressif's 'Chat Coding' AI agent framework for IoT devices, reimplemented in C for ESP32-series chips. It lets users define d… | 66 | 2051 | active |
| Nutlope/llamatutor LlamaTutor is an open-source AI personal tutor web application powered by Meta's Llama 3.1 70B model via Together AI inference. It is built… | 65 | 2049 | active |
| microsoft/foundry-dev-tools Microsoft Foundry Toolkit (formerly AI Toolkit) is a Visual Studio Code extension for building, testing, and deploying AI agents and models… | 80 | 2047 | active |
| CyberStrikeus/CyberStrike CyberStrike is an open-source AI-powered offensive security harness that runs automated penetration testing from the terminal. It orchestra… | 81 | 2043 | active |
| 4thfever/cultivation-world-simulator An open-source AI-driven Cultivation (Xianxia) world simulator where every cultivator NPC is an independent LLM-powered agent that observes… | 83 | 2042 | active |
| 01-ai/Yi Yi is a family of open-source large language models trained from scratch by 01.AI, including base and chat models in multiple sizes, with b… | 27 | 7836 | maintenance |
| KEV0143/Telegram-MedicalBot A Telegram-based user support system for medical inquiries that manages user questions through a ticket system. It generates suggested resp… | 43 | 2039 | active |
| boldsoftware/meat meat is a Go CLI tool that uses an LLM to abridge a code diff into a 'reading diff', reducing agent-written changes to the conceptually imp… | 56 | 2030 | active |
| noamgat/lm-format-enforcer A Python library that enforces output formats like JSON Schema or regex on language models by filtering allowed tokens at each generation s… | 64 | 2029 | active |
| Ocyss/boss-helper A browser extension (also available as a userscript) that enhances the Boss Zhipin job platform by removing ads, improving the UI, enabling… | 91 | 2027 | active |
| 0xShug0/audio.cpp audio.cpp is a pure C++ inference engine for audio models built on ggml, supporting TTS, STT, VAD, voice conversion, music generation, and … | 80 | 2022 | active |
| memochou1993/gpt-ai-assistant GPT AI Assistant is an application that connects the OpenAI API with the LINE Messaging API, letting users chat with their own AI assistant… | 61 | 7739 | maintenance |
| IBM/mcp-cli A Python command-line client for interacting with Model Context Protocol (MCP) servers, integrating LLMs via CHUK Tool Processor and CHUK-L… | 85 | 2020 | active |
| Doorman11991/smallcode SmallCode is a terminal-native AI coding agent optimized for small local LLMs (8B-35B parameters) running on consumer hardware. It compensa… | 75 | 2017 | active |
| tatsu-lab/alpaca_eval AlpacaEval is an LLM-based automatic evaluation framework for instruction-following language models, producing win rates against a GPT-4 ba… | 36 | 2012 | active |
| xiangsx/gpt4free-ts A TypeScript replication of gpt4free that provides free access to GPT-4 and other LLM models by reverse-engineering various websites offeri… | 20 | 7681 | maintenance |
| showlab/Code2Video Code2Video is a research framework from Show Lab (NUS) that generates educational videos by having multi-agent LLM pipelines write and rend… | 62 | 2003 | active |
| zai-org/GLM-130B GLM-130B is an open bilingual (English and Chinese) 130-billion-parameter dense language model pre-trained with the General Language Model … | 32 | 7651 | maintenance |
| apple/coreai-models Apple's repository of model export recipes, Python primitives, and Swift runtime utilities for building on-device AI with the Core AI frame… | 67 | 1999 | active |
| mil-tokyo/webdnn WebDNN is a framework for running deep neural network inference directly in the web browser, accepting ONNX models without Python preproces… | 65 | 1999 | active |
| glebkudr/shotgun_code Shotgun is a desktop application that packages a local codebase into a structured context payload for LLMs, with AI-powered file selection … | 52 | 1999 | active |
| 64bit/async-openai async-openai is a Rust library providing typed, async clients for the OpenAI API, covering chat completions, responses, embeddings, assista… | 93 | 1998 | active |
| aws/aws-toolkit-vscode AWS Toolkit is a Visual Studio Code extension that connects the IDE to AWS resources, supporting credential management, Lambda debugging vi… | 95 | 1996 | active |
| Niek/chatgpt-web A single-page web interface for OpenAI-compatible chat APIs, built with Svelte, where users bring their own API key and chats are stored pr… | 75 | 1996 | active |
| shcherbak-ai/contextgem ContextGem is an open-source Python LLM framework for extracting structured data and insights from documents with minimal code. It provides… | 87 | 1993 | active |
| win4r/AISuperDomain Aila (AISuperDomain) is a cross-platform desktop application that lets users send one prompt to 10+ AI chat services (ChatGPT, Gemini, Clau… | 63 | 1993 | active |