function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| NVlabs/LongLive LongLive is an NVIDIA research framework providing parallel training and inference infrastructure for real-time long video generation, usin… | 60 | 2563 | active |
| kayba-ai/agentic-context-engine Agentic Context Engine (ACE) is a Python library that lets LLM-based agents learn from experience by maintaining and evolving an evolving c… | 75 | 2558 | active |
| 1rgs/nanocode A minimal Claude Code alternative implemented as a single ~250-line Python file with zero dependencies. It runs an agentic loop with file a… | 43 | 2556 | active |
| jolibrain/deepdetect DeepDetect is an open-source deep learning runtime, CLI, and REST server written in C++ for training and inference across images, text, tab… | 95 | 2551 | active |
| google-research/tabfm TabFM is a scikit-learn compatible tabular foundation model from Google Research that performs zero-shot classification and regression on t… | 67 | 2551 | active |
| spring-ai-alibaba/DataAgent An enterprise-grade AI data analyst agent built on Spring AI Alibaba Graph that converts natural language to SQL, runs Python deep analysis… | 83 | 2550 | active |
| fogsightai/fogsight Fogsight is an open-source AI agent and animation engine powered by large language models that turns abstract concepts or words into polish… | 51 | 2550 | active |
| YeQing17-2026/OmniAgent OmniAgent is an open-source Python agent framework that self-evolves across skills, context, memory, and its underlying model during intera… | 56 | 2549 | active |
| ulab-uiuc/LLMRouter LLMRouter is an open-source Python library that intelligently routes queries to the most suitable LLM based on task complexity, cost, and p… | 61 | 2547 | active |
| FB208/OpenBidKit_Yibiao OpenBidKit Yibiao (易标) is an open-source Electron desktop application that uses LLMs and RAG to generate bid/tender proposal documents from… | 81 | 2546 | active |
| X-PLUG/mPLUG-Owl mPLUG-Owl is a family of open-source multimodal large language models that combine visual encoders with LLMs (built on LLaMA) for image and… | 36 | 2539 | active |
| leookun/cursor-byok cursor-byok is an open-source local implementation of Cursor's backend that runs a gateway service on your machine, connecting the Cursor e… | 81 | 2537 | active |
| lamini-ai/lamini The official Python client and SDK for Lamini's hosted API for building generative AI applications. It lets developers call Lamini's LLM in… | 37 | 2534 | active |
| VITA-MLLM/VITA VITA is an open-source interactive omni multimodal large language model (VITA-1.5) that supports real-time vision and speech interaction, s… | 29 | 2534 | active |
| aingdesk/AingDesk AingDesk is an open-source, MIT-licensed AI assistant application for running local AI models (e.g., via Ollama) and third-party model APIs… | 60 | 2533 | active |
| bestruirui/octopus Octopus is a self-hosted LLM API aggregation gateway that unifies multiple provider channels behind a single endpoint, converting between O… | 79 | 2527 | active |
| huggingface/lighteval Lighteval is Hugging Face's all-in-one toolkit for evaluating large language models across multiple inference backends (Transformers, vLLM,… | 77 | 2527 | active |
| google-coral/coralnpu Coral NPU is an open-source neural processing unit (NPU) hardware IP core from Google Research, built on the 32-bit RISC-V ISA with matrix,… | 73 | 2526 | active |
| microsoft/foundry-local Foundry Local is Microsoft's end-to-end local AI runtime and SDK suite (C#, JavaScript, Python, Rust) for running optimized models entirely… | 85 | 2525 | active |
| cixingguangming55555/wechat-bot A free toolset (funtool) providing proxy executables that let the Claude Code CLI use free NVIDIA-hosted large models such as Kimi K2.5, GL… | 62 | 2523 | active |
| microsoft/ResearchStudio ResearchStudio is a Microsoft collection of AI agent skills that cover the entire research lifecycle, from an under-specified research dire… | 59 | 2523 | active |
| alecm20/story-flicks Story-Flicks is a Python + FastAPI application with a React frontend that generates high-definition story short videos from a user-provided… | 26 | 2523 | active |
| Tencent-Hunyuan/HY-Motion-1.0 HY-Motion 1.0 is a series of text-to-3D human motion generation models from Tencent Hunyuan, based on Diffusion Transformer and Flow Matchi… | 58 | 2519 | active |
| SafeAILab/EAGLE EAGLE is the official implementation of the EAGLE family of speculative decoding algorithms (EAGLE-1/2/3) for accelerating LLM text generat… | 57 | 2510 | active |
| itayinbarr/little-coder little-coder is a terminal coding agent built on the pi agent loop, tuned for small local language models like Qwen via Ollama or llama.cpp… | 81 | 2504 | active |
| huggingface/huggingface.js A collection of TypeScript JavaScript libraries for interacting with the Hugging Face ecosystem, including Hub repository management, unifi… | 94 | 2501 | active |
| InternLM/HuixiangDou HuixiangDou is an LLM-based professional knowledge assistant designed for group chat scenarios, using a three-stage pipeline of preprocess,… | 46 | 2499 | active |
| HarleyCoops/Math-To-Manim-lfs-archive A Python project that turns math and physics questions into reasoned visual explanations rendered as Manim animations and study notes. It u… | 76 | 2495 | active |
| huggingface/tau Tau is a small, readable terminal coding agent written in Python, inspired by Pi, that can read files, edit code, run commands, and maintai… | 80 | 2492 | active |
| janhq/ichigo Ichigo is a Python speech package for developers offering local realtime voice AI capabilities, including a compact 22M-parameter speech to… | 50 | 2492 | active |
| Live-GalGame/LiveGalGame LiveGalGame is a playful app that overlays a visual-novel (galgame) interface onto real-life conversations, providing real-time speech-to-t… | 58 | 2490 | active |
| nikmcfly/MiroFish-Offline MiroFish-Offline is a fully local, English-language fork of MiroFish, a multi-agent swarm intelligence engine that simulates public opinion… | 48 | 2490 | active |
| darrenburns/elia Elia is a keyboard-centric terminal user interface (TUI) application for chatting with large language models like ChatGPT, Claude, Gemini, … | 31 | 2482 | active |
| microsoft/PIKE-RAG PIKE-RAG is a Microsoft framework for building Retrieval-Augmented Generation systems that extract, understand, and apply specialized domai… | 31 | 2480 | active |
| perplexityai/modelcontextprotocol The official Model Context Protocol (MCP) server for the Perplexity API Platform, exposing real-time web search, reasoning, and research to… | 64 | 2478 | active |
| pykeio/ort ort is a safe, fast Rust binding for Microsoft's ONNX Runtime, providing hardware-accelerated machine learning inference and training for O… | 89 | 2476 | active |
| 24mlight/A_Share_investment_Agent A Python-based multi-agent AI investment analysis system for China's A-share market that uses LLMs (Gemini/OpenAI-compatible) with bull/bea… | 59 | 2474 | active |
| harry0703/AudioNotes AudioNotes is a locally-run web application that transcribes audio and video files into text and organizes them into structured Markdown no… | 67 | 2472 | active |
| kaqijiang/Auto-GPT-ZH A Chinese-language community fork and translation of AutoGPT, the autonomous AI agent platform, kept in sync with the upstream stable branc… | 37 | 2466 | active |
| xusenlinzy/api-for-open-llm A Python server that exposes open-source large language models (LLaMA, ChatGLM, Qwen, Baichuan, Falcon, etc.) behind an OpenAI-compatible A… | 20 | 2460 | active |
| mirage-project/mirage Mirage Persistent Kernel (MPK) is a compiler and runtime that transforms multi-GPU LLM inference into a single fused megakernel, reducing i… | 83 | 2452 | active |
| KsanaDock/Microverse Microverse is a god-simulation sandbox game built on Godot 4 that implements a multi-agent AI social simulation. AI characters have indepen… | 51 | 2450 | active |
| UnicomAI/wanwu Yuanjing Wanwu is China Unicom's enterprise-grade, multi-tenant AI agent development platform built in Go. It provides all-in-one tooling f… | 82 | 2448 | active |
| AtomicBot-ai/atomic-agent Atomic Agent is a local-first AI agent that runs entirely on your machine, using quantized models via llama.cpp to browse the web, edit fil… | 77 | 2446 | active |
| antirez/h3.c A native C inference engine for the MiniMax H3 model on Apple Silicon, using Metal for GPU acceleration. It generates video (and audio) fro… | 56 | 2443 | active |
| Zafer-Liu/Data-Analysis-Agent An LLM-powered conversational data analysis agent that lets users connect data sources (Excel/CSV files, databases, Feishu tables) and ask … | 81 | 2442 | active |
| FullAgent/fulling Fulling is an AI-powered full-stack engineer agent platform that provides dedicated AI workspaces combining skills, files, memory, and runt… | 69 | 2439 | active |
| inference-labs-inc/subnet-2 Subnet 2 on the Bittensor network implements a Proof-of-Inference system that uses zero-knowledge machine learning (zk-ML) to cryptographic… | 86 | 2437 | active |
| google/XNNPACK XNNPACK is a highly optimized C library of floating-point neural network inference operators for ARM, x86, WebAssembly, and RISC-V platform… | 77 | 2435 | active |
| hitsz-ids/synthetic-data-generator SDG (Synthetic Data Generator) is a Python framework for generating high-quality synthetic structured tabular data using deep learning, GAN… | 74 | 2434 | active |
| microsoft/chat-copilot Chat Copilot is a sample application from Microsoft for building your own LLM-powered chat copilot, built on Semantic Kernel. It consists o… | 10 | 2434 | active |
| volcengine/ai-app-lab AI App Lab from Volcano Engine (Volcengine) provides Arkitect, a high-code Python SDK for building LLM applications, plus Demohouse, a coll… | 83 | 2433 | active |
| erew123/alltalk_tts AllTalk TTS is a text-to-speech application built on the Coqui TTS engine, usable standalone or as an extension for Text-generation-webui, … | 44 | 2429 | active |
| Yuyz0112/claude-code-reverse A tool that reverse engineers Claude Code by capturing and visualizing its LLM API interactions. It monkey-patches Claude Code's request co… | 37 | 2428 | active |
| roboflow/inference Roboflow Inference is a Python library and self-hostable inference server for deploying computer vision models on any computer or edge devi… | 91 | 2427 | active |
| ggozad/oterm oterm is a terminal-based chat client for LLMs, supporting Ollama, OpenAI, Anthropic, and any pydantic-ai-supported provider. It offers a r… | 92 | 2426 | active |
| theJayTea/WritingTools A system-wide AI writing assistant for Windows, Linux, and macOS that proofreads, rewrites, and summarizes text in any application via a ho… | 83 | 2426 | active |
| zurawiki/gptcommit gptcommit is a git prepare-commit-msg hook written in Rust that uses large language models like GPT to automatically generate clear, descri… | 57 | 2420 | active |
| prism-php/prism Prism is a Laravel package providing a unified, fluent interface for working with Large Language Models across providers like OpenAI, Anthr… | 68 | 2419 | active |
| jackmpcollins/magentic A Python library for seamlessly integrating LLMs into Python code via @prompt and @chatprompt decorators that return typed, structured outp… | 74 | 2415 | active |
| Jittor/JittorLLMs JittorLLMs is a large language model inference library built on the Jittor deep learning framework, supporting models like ChatGLM, LLaMA/L… | 34 | 2413 | active |
| lmnr-ai/index Index is an open-source browser agent that autonomously performs complex tasks on the web by turning any website into an accessible API. It… | 10 | 2411 | active |
| yetone/voice-input-src A macOS menu-bar voice input app (Swift, macOS 14+) that lets users hold the Fn key to record and release to inject streaming-transcribed t… | 50 | 2409 | active |
| BloopAI/bloop bloop is a fast code search engine and desktop app that lets developers search their codebase with natural language, regex, and symbol quer… | 10 | 9499 | maintenance |
| oritera/Cairn Cairn is a general-purpose AI state-space search engine built on a blackboard architecture with a fact-intent graph, where LLM agent worker… | 72 | 2395 | active |
| leilei926524-tech/anti-distill A tool that sanitizes AI Skill files employees are asked to write, producing a professional-looking but hollowed-out version for submission… | 55 | 2388 | active |
| Nano-Collective/nanocoder Nanocoder is an open-source, terminal-based AI coding agent built by a community collective. It runs agentic coding workflows against any m… | 81 | 2384 | active |
| microsoft/Olive Olive is Microsoft's AI model optimization toolkit for the ONNX Runtime, automating finetuning, conversion, quantization, and compression o… | 91 | 2382 | active |
| BigDawnGhost/wenyi Wenyi is a Python CLI tool that translates EPUB books into Chinese using LLMs, with whole-book context analysis, real-time glossary extract… | 80 | 2381 | active |
| gkamradt/needle-in-a-haystack A benchmarking tool (niah / needlehaystack) that pressure-tests LLM long-context retrieval by sweeping context length x needle depth cells … | 72 | 2375 | active |
| FMInference/FlexLLMGen FlexLLMGen is a high-throughput generation engine for running large language models on a single GPU with limited memory. It uses IO-efficie… | 10 | 9352 | maintenance |
| spacedriveapp/spacebot Spacebot is an opinionated AI agent harness written in Rust for teams, communities, and multi-user environments, deployable as a single sel… | 76 | 2371 | active |
| zai-org/GLM-V GLM-V is the open-source repository for Zhipu AI's GLM-4.6V, GLM-4.5V, and GLM-4.1V-Thinking vision-language models, which perform versatil… | 60 | 2370 | active |
| DeepInsight-AI/DeepBI DeepBI is an AI-native business intelligence platform that uses large language models to let users explore, query, visualize, and share dat… | 24 | 2370 | active |
| showlab/Paper2Video Paper2Video is a Python pipeline that automatically generates academic presentation videos from scientific papers, taking a paper PDF, a sp… | 48 | 2368 | active |
| orhanerday/open-ai A community-maintained PHP SDK for accessing the OpenAI API, supporting GPT-3/GPT-4 chat completions, DALL-E image generation, and ChatGPT-… | 28 | 2366 | active |
| neavo/LinguaGacha LinguaGacha is an AI-powered desktop text translator that batch-translates novels, game scripts, subtitles, and ebooks using LLM APIs (Open… | 83 | 2364 | active |
| drl990114/MarkFlowy MarkFlowy is a lightweight, cross-platform Markdown and text editor built with Tauri, React, and Prosemirror, offering WYSIWYG and source-c… | 99 | 2362 | active |
| Natively-AI-assistant/natively-cluely-ai-assistant Natively is a free, source-available desktop AI meeting assistant and interview copilot that provides real-time transcription, AI-generated… | 82 | 2354 | active |
| NarratorAI-Studio/narrator-ai-cli-skill A machine-readable SKILL.md skill package that teaches AI agents (Claude Code, OpenClaw, Windsurf, Cursor, etc.) how to drive the narrator-… | 59 | 2354 | active |
| vava-nessa/free-coding-models A CLI tool that tracks a large catalog of free coding LLM models across 20+ providers in real time, benchmarking their speed and stability.… | 78 | 2345 | active |
| agentuniverse-ai/agentUniverse agentUniverse is a Python-based multi-agent framework built on large language models, providing extensible components for building individu… | 70 | 2343 | active |
| maitrix-org/llm-reasoners LLM Reasoners is a Python library that provides a standardized modular framework for advanced LLM reasoning, implementing algorithms like M… | 31 | 2343 | active |
| MaxMiksa/Auto-Company Auto Company is a fully autonomous AI company simulation that runs 24/7 on your own PC, orchestrating 14 autonomous AI agents modeled after… | 66 | 2339 | active |
| jmiao24/Paper2Agent Paper2Agent is a multi-agent AI system that automatically converts research paper codebases into interactive AI agents with minimal human i… | 47 | 2338 | active |
| hydropix/TranslateBooksWithLLMs A desktop application that translates full-length books, subtitles, and documents using local or cloud LLMs (Ollama, OpenAI-compatible APIs… | 82 | 2334 | active |
| fruitbars/simple-one-api A single-binary Go gateway that unifies access to multiple LLM providers (Qianfan, iFlytek Spark, Tencent Hunyuan, MiniMax, DeepSeek, etc.)… | 91 | 2329 | active |
| refuel-ai/autolabel Autolabel is a Python library for labeling, cleaning, and enriching text datasets using large language models like GPT-4, Claude, and open-… | 25 | 2329 | active |
| copilot-emacs/copilot.el An unofficial Emacs plugin for GitHub Copilot that provides inline ghost-text completions, a chat interface, and Next Edit Suggestions. It … | 93 | 2328 | active |
| PhialsBasement/Chain-of-Recursive-Thoughts CoRT (Chain of Recursive Thoughts) is a tool that makes LLMs recursively refine their answers by generating multiple alternative responses,… | 28 | 2328 | active |
| PKU-YuanGroup/MoE-LLaVA MoE-LLaVA is an open-source Mixture-of-Experts based sparse large vision-language model, released with the MoE-Tuning training strategy fro… | 31 | 2322 | active |
| severian42/GraphRAG-Local-UI A local-first GraphRAG application adapted from Microsoft's GraphRAG, providing a FastAPI backend plus Gradio-based UIs for indexing, promp… | 14 | 2321 | active |
| XiaomiMiMo/MiMo Xiaomi's MiMo is a 7B-parameter reasoning language model trained from pretraining through posttraining with reinforcement learning, release… | 30 | 2299 | active |
| togethercomputer/OpenChatKit OpenChatKit is an open-source toolkit from Together, LAION, and Ontocord.ai for building and fine-tuning chat language models, including in… | 30 | 8984 | maintenance |
| HKUDS/FastCode FastCode is a token-efficient framework for repository-scale code understanding and reasoning, using structural scouting over a semantic ma… | 58 | 2292 | active |
| romgX/openrelay OpenRelay is a desktop application that aggregates AI model quotas from local tools (Claude Desktop, Kiro, Windsurf, Copilot, etc.) and dir… | 76 | 2286 | active |
| sqliteai/warp WARP is a dependency-free, embeddable C inference engine that runs trillion-parameter mixture-of-experts models like Kimi K3 by streaming a… | 80 | 2284 | active |
| Karben233/bili-hardcore A Rust CLI tool that automatically answers Bilibili's 'hardcore member' daily quiz questions using LLM APIs. It also ships as an agent skil… | 85 | 2278 | active |
| InternLM/lagent Lagent is a lightweight Python framework for building LLM-based agents, designed with a PyTorch-like philosophy where agents are composed l… | 84 | 2278 | active |
| jamubc/gemini-mcp-tool A Model Context Protocol (MCP) server that lets AI assistants like Claude Code invoke the Google Gemini CLI (or its successor, the Antigrav… | 79 | 2278 | active |