function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| MayDay-wpf/snow-cli Snow CLI is a terminal-based AI coding agent written in TypeScript that can read, write, and refactor code directly from the command line. … | 80 | 1062 | active |
| brenpoly/be-more-agent An offline-first conversational AI agent that turns a Raspberry Pi into a fully local voice assistant. It combines OpenWakeWord, Whisper.cp… | 64 | 1060 | active |
| pepperoni21/ollama-rs A Rust client library for interacting with the Ollama API, supporting completion generation, streaming, chat, model management, embeddings,… | 88 | 1059 | active |
| ufy2024/AuC AuC (Agents-ufy-Core) is a Python single-agent framework built on asyncio with a pluggable ReAct reasoning loop, LLM adapters, tiered tool … | 56 | 1058 | active |
| bigcode-project/bigcode-evaluation-harness A framework for evaluating autoregressive code generation language models on benchmarks like HumanEval, MBPP, MultiPL-E, and HumanEvalPack.… | 38 | 1058 | active |
| pytorch/torchchat torchchat is a compact PyTorch-native codebase for running large language models locally via Python, a C/C++ runner on desktop/server, or o… | 10 | 3612 | maintenance |
| Fullive-AI/Anima Anima is an open-source Agent OS that gives home hardware perceptive, decision-making, and learning AI capabilities by connecting devices t… | 58 | 1056 | active |
| XYZ-AI-Lab/axrl AxisRL is an agentic reinforcement learning post-training framework for large language models, built on SGLang for high-throughput rollout … | 55 | 1056 | active |
| Dicklesworthstone/swiss_army_llama A FastAPI-based REST service that exposes local LLM capabilities including text embeddings, completions, semantic similarity, and semantic … | 32 | 1056 | active |
| zai-org/GLM-TTS GLM-TTS is a Python-based text-to-speech synthesis system built on large language models, using a two-stage LLM plus Flow model architectur… | 50 | 1055 | active |
| BAAI-DCAI/Bunny Bunny is a family of lightweight multimodal vision-language models that combine plug-and-play vision encoders (EVA-CLIP, SigLIP) with langu… | 26 | 1053 | active |
| orailnoor/cross-platform-llm-client PrivateLM is a cross-platform AI chat client built with Flutter that unifies local on-device LLM inference (GGUF models with Vulkan GPU acc… | 72 | 1051 | active |
| InternRobotics/PointLLM PointLLM is a multimodal large language model that understands colored 3D point clouds of objects, built on a point cloud encoder fused wit… | 65 | 1051 | active |
| OpenLAIR/dr-claw Dr. Claw is an open-source, model-agnostic AI research assistant that provides a desktop and mobile UI for Claude Code, Cursor CLI, Codex, … | 82 | 1050 | active |
| shyamsaktawat/OpenAlpha_Evolve OpenAlpha_Evolve is an open-source Python framework inspired by DeepMind's AlphaEvolve that uses LLMs to iteratively generate, test, and ev… | 30 | 1049 | active |
| CRui5in/paper-ppt-agent A self-hosted web application that uses a multi-agent AI pipeline (Strategist, Executor, Critic) to convert academic papers from PDF or LaT… | 79 | 1048 | active |
| Kieirra/murmure Murmure is a privacy-first, open-source desktop speech-to-text application that transcribes voice entirely on-device using NVIDIA's Parakee… | 85 | 1047 | active |
| onnx/onnx-mlir ONNX-MLIR is a compiler infrastructure built on LLVM/MLIR that lowers ONNX neural network graphs into executable code, MLIR files, LLVM byt… | 86 | 1046 | active |
| Xilinx/finn FINN is an open-source dataflow compiler from AMD/Xilinx that generates highly efficient FPGA accelerators for quantized neural network (QN… | 68 | 1046 | active |
| Minima-AI-Inc/minima Minima is an open-source, on-premises conversational RAG system deployed via Docker containers. It supports fully local operation with Olla… | 41 | 1046 | active |
| langchain-ai/langsmith-sdk Official Python and JavaScript/TypeScript client SDKs for the LangSmith platform, enabling tracing, debugging, evaluation, and monitoring o… | 93 | 1044 | active |
| vstorm-co/pydantic-deepagents Pydantic Deep Agents is an open-source, self-hosted terminal AI assistant (a Claude Code alternative) plus the Python framework behind it, … | 78 | 1044 | active |
| xiaochong/hi-kid HiKid is a free, open-source Electron desktop app that lets children in non-English-speaking countries practice English speaking and listen… | 54 | 1043 | active |
| scrya-com/rotorquant RotorQuant is a KV cache compression method for LLM inference that replaces full d×d orthogonal rotations with block-diagonal Clifford roto… | 50 | 1043 | active |
| yoheinakajima/instagraph InstaGraph is a Flask web application that converts text input or URLs into knowledge graphs using OpenAI's GPT-3.5. It renders color-coded… | 28 | 3553 | maintenance |
| HannesStark/boltzgen BoltzGen is an open-source all-atom generative diffusion model for designing protein and peptide binders against arbitrary biomolecular tar… | 71 | 1042 | active |
| octos-org/octos Octos is a self-hosted AI agent runtime that runs a personal AI assistant on your own machine, connecting to providers like Anthropic, Open… | 80 | 1041 | active |
| wrtnlabs/agentica Agentica is a TypeScript agentic AI framework specialized in LLM function calling, enhanced by compiler-driven validation via typia. Agents… | 71 | 1041 | active |
| Nutlope/blinkshot BlinkShot is an open-source web application that generates AI images in real time as you type, powered by the Flux Schnell model via Togeth… | 67 | 1040 | active |
| metauto-ai/GPTSwarm GPTSwarm is a Python library for building LLM-based agents as computational graphs, with modules for agent graphs, memory, LLM backends, an… | 55 | 1040 | active |
| tonbistudio/turboquant-pytorch A from-scratch PyTorch implementation of Google's TurboQuant (ICLR 2026) algorithm for compressing LLM key-value caches, including an impro… | 50 | 1040 | active |
| darrencxl0301/StageRAG StageRAG is a Python framework/blueprint for building production-ready RAG applications with switchable 3-step (Speed) and 4-step (Precisio… | 38 | 1040 | active |
| smallcloudai/refact Refact is an open-source, self-hosted AI coding agent written in Rust that plans and executes engineering tasks end-to-end from your IDE, w… | 10 | 3540 | maintenance |
| PatterAI/Patter Patter is an open-source, MIT-licensed SDK (Python and TypeScript) that connects AI agents to real phone calls, handling telephony, speech-… | 78 | 1039 | active |
| vercel-labs/tersa Tersa is an open-source visual AI playground built as a Next.js web application where users drag, drop, and connect nodes on a canvas to bu… | 64 | 1039 | active |
| ricklamers/gpt-code-ui An open-source, self-hosted implementation of OpenAI's ChatGPT Code Interpreter. Users chat with GPT-3.5/GPT-4 models that generate and exe… | 20 | 3536 | maintenance |
| wxywb/history_rag A Chinese history question-answering application built with RAG (retrieval-augmented generation) over a corpus of Chinese historical texts,… | 26 | 1038 | active |
| villesau/ai-codereviewer A GitHub Action that uses OpenAI's GPT-4 API to automatically review pull requests and post intelligent feedback comments. It retrieves PR … | 21 | 1037 | active |
| juanjuandog/FinSight-AI FinSight AI is an open-source equity research workspace for A-share companies that turns market data, filings, and financial metrics into s… | 59 | 1034 | active |
| thu-ml/SpargeAttn SpargeAttn is the official implementation of a training-free sparse and quantized attention method that accelerates inference for language,… | 51 | 1033 | active |
| centerforaisafety/HarmBench HarmBench is a standardized, open-source evaluation framework for automated red teaming of large language models, comparing attack methods … | 26 | 1033 | active |
| samuel-vitorino/lm.rs A minimal Rust implementation for running LLM inference locally on CPU without ML libraries, supporting Gemma 2, Llama 3.2, and Phi 3.5 (in… | 23 | 1033 | active |
| NVIDIA-NeMo/Skills Nemo-Skills is a collection of Python pipelines for improving the skills of large language models, covering synthetic data generation, mode… | 70 | 1031 | active |
| m-damien/VisualStoryWriting A web application that automatically visualizes a story's chronology, characters, and their movements, letting writers edit the story by ma… | 32 | 1031 | active |
| adrida/tracer TRACER is a Python library that learns a routing policy from LLM classification traces, fitting a lightweight traditional ML surrogate to h… | 78 | 1030 | active |
| minimaxir/simpleaichat A Python library providing a minimal, low-complexity interface for building applications on top of chat LLM APIs like ChatGPT and GPT-4. It… | 20 | 3500 | maintenance |
| Felix3322/PotPlayer_ChatGPT_Translate A PotPlayer plugin that integrates OpenAI-compatible AI APIs (including ChatGPT and Ollama) to translate video subtitles in real time. It u… | 84 | 1029 | active |
| kuleshov-group/bd3lms BD3-LMs is a research implementation of Block Discrete Denoising Diffusion Language Models that interpolate between autoregressive and diff… | 34 | 1029 | active |
| kuprel/min-dalle min(DALL·E) is a fast, minimal PyTorch port of DALL·E Mini/Mega stripped down for text-to-image inference, with only numpy, requests, pillo… | 31 | 3494 | maintenance |
| Goekdeniz-Guelmez/Local-NotebookLM A local, open-source alternative to Google's NotebookLM that converts PDF documents into audio content like podcasts, summaries, and interv… | 74 | 1027 | active |
| mybigday/llama.rn A React Native binding of llama.cpp that enables on-device LLM inference on iOS and Android. It supports GPU/NPU acceleration (Metal, Hexag… | 94 | 1026 | active |
| walter-grace/mac-code A free, local coding agent for Apple Silicon Macs that runs large quantized LLMs (e.g., Qwen 35B) via llama.cpp, using techniques like flas… | 49 | 1026 | active |
| jucasoliveira/terminalGPT TerminalGPT is a Node.js CLI tool that brings ChatGPT-like LLM chat conversations directly into your terminal. It supports multiple LLM pro… | 37 | 1024 | active |
| andyzoujm/representation-engineering Official library for Representation Engineering (RepE), a top-down approach to AI transparency that monitors and manipulates population-lev… | 18 | 1024 | active |
| rayenfeng/riko_project Project Riko is an anime-themed conversational voice assistant that combines OpenAI's GPT for dialogue, GPT-SoVITS for voice synthesis, and… | 31 | 1023 | active |
| hrithikkoduri/WebRover WebRover is an autonomous AI web agent that interprets user input, navigates websites via browser automation, and performs tasks or deep re… | 14 | 1022 | active |
| Tencent/WeSmartFlow WeSmartFlow is an agent-native adaptive learning framework that models the full learning process with ReAct-based tutoring agents, graph me… | 59 | 1020 | active |
| XGenerationLab/XiYan-SQL XiYan-SQL is a multi-generator ensemble framework for converting natural language questions into SQL queries, achieving SOTA results on ben… | 59 | 1019 | active |
| hinterdupfinger/obsidian-ollama An Obsidian plugin that integrates locally running Ollama models into your notes. It provides preconfigured prompts (summarize, explain, re… | 18 | 1018 | active |
| microsoft/Tutel Tutel is Microsoft's optimized Mixture-of-Experts (MoE) library for efficient training and inference of large language models, featuring dy… | 79 | 1016 | active |
| srcbookdev/srcbook Srcbook is a local-first TypeScript notebook application that runs as a CLI with a web interface, supporting executable TypeScript cells, m… | 67 | 3444 | maintenance |
| MeshAnything MeshAnything is an autoregressive transformer model that generates artist-created 3D meshes (up to 1600 faces in V2) aligned with a given s… | 31 | 1015 | active |
| addyosmani/bg-remove A React + Vite web application that removes image backgrounds entirely in the browser using Transformers.js with the RMBG-1.4 model (and op… | 22 | 1014 | active |
| deepseek-ai/DeepSeek-Math DeepSeekMath is a 7B open language model specialized in mathematical reasoning, initialized from DeepSeek-Coder and trained on 500B math-re… | 26 | 3436 | maintenance |
| jianjieyiban/JJYB_AI_VideoAutoCut JJYB_AI 智剪 is a local-first desktop AI video creation workbench that combines material analysis, smart shot segmentation, commentary script… | 70 | 1013 | active |
| Alpha-VLLM/Lumina-Image-2.0 Lumina-Image 2.0 is an open-source 2.6B-parameter text-to-image generation framework built on a unified Next-DiT architecture with a unifie… | 58 | 1013 | active |
| SunOner/sunone_aimbot An AI-powered aimbot for first-person shooter games that uses YOLO object detection models (YOLOv8/v10/v12) with TensorRT/ONNX acceleration… | 65 | 1010 | active |
| 274056675/springboot-openai-chatgpt A full-stack AI chatbot application built on Spring Boot/Spring Cloud that integrates GPT-3.5, GPT-4, Baidu ERNIE Bot, Stable Diffusion, an… | 36 | 1010 | active |
| stitionai/devika Devika is an open-source agentic AI software engineer that takes high-level instructions, plans steps, researches via web browsing, and wri… | 44 | 19561 | experimental |
| kaito-project/kaito KAITO is a Kubernetes operator suite that automates LLM inference, fine-tuning, and RAG engine deployment using simplified CRD APIs. It aut… | 95 | 1009 | active |
| braintrustdata/autoevals Autoevals is a Python and TypeScript library for automatically evaluating AI model outputs using LLM-as-a-judge, heuristic, and statistical… | 85 | 1009 | active |
| HumeAI/tada TADA is an open-source speech-language model from Hume AI that generates expressive, high-fidelity speech via text-acoustic dual alignment,… | 51 | 1009 | active |
| xqdoo00o/chatgpt-web A pure JavaScript ChatGPT web client that ships as a single HTML file and talks directly to the OpenAI API (plus DeepSeek, Gemini, and Clau… | 33 | 1009 | active |
| voquill/voquill Voquill is an open-source, cross-platform AI voice dictation app that lets users dictate into any desktop application, with AI-powered tran… | 77 | 1008 | active |
| ZTE-AICloud/Co-Sight Co-Sight is an open-source Python framework from ZTE for building Manus-like autonomous AI agent systems that generate high-quality researc… | 44 | 1008 | active |
| synthetic-lab/octofriend OctoFriend is a zero-telemetry, terminal-based AI coding assistant that works with any OpenAI- or Anthropic-compatible LLM API, including l… | 65 | 1007 | active |
| MoonshotAI/checkpoint-engine Checkpoint-engine is a lightweight Python middleware for updating model weights in-place across LLM inference engines, a critical step in r… | 80 | 1005 | active |
| aws-samples/bedrock-access-gateway A proxy service that exposes Amazon Bedrock foundation models through OpenAI-compatible RESTful APIs. It lets existing applications built o… | 64 | 1005 | active |
| minimaxir/gpt-2-simple A Python package that simplifies fine-tuning OpenAI's GPT-2 text-generation model (124M/355M) on custom text and generating text from the r… | 23 | 3400 | maintenance |
| MichaelYuhe/ai-group-tabs A Chrome extension that uses AI (OpenAI API) to automatically organize and group browser tabs into categories. Users can customize categori… | 10 | 1004 | active |
| ZJUI-AI4H/Hulu-Med Hulu-Med is a family of open-source transparent generalist medical vision-language models ranging from 4B to 235B parameters, covering text… | 62 | 1001 | active |
| JIA-Lab-research/MGM Official PyTorch implementation of Mini-Gemini, a multimodal vision-language model framework built on LLaVA that supports dense and MoE LLM… | 25 | 3327 | maintenance |
| pytorch/glow Glow is a machine learning compiler and execution engine for neural network hardware accelerators, developed under the PyTorch project. It … | 10 | 3320 | maintenance |
| deep-diver/LLM-As-Chatbot A Python application that serves open-source instruction-tuned LLMs as chatbot services via a Gradio web UI or a Discord bot. It integrates… | 30 | 3319 | maintenance |
| X-D-Lab/LangChain-ChatGLM-Webui A Gradio-based web UI that combines LangChain with ChatGLM-6B and other open-source LLMs to answer questions over a local knowledge base. U… | 30 | 3312 | maintenance |
| bytedance/lightseq LightSeq is a high-performance CUDA-based library for training and inference of sequence models like Transformer, BERT, GPT, and BART, with… | 10 | 3295 | maintenance |
| missuo/FreeGPT35 A self-hosted service that exposes the login-free ChatGPT Web's GPT-3.5-Turbo as an OpenAI-compatible /v1/chat/completions API. It can be d… | 10 | 3228 | maintenance |
| SilasMarvin/lsp-ai LSP-AI is an open-source language server written in Rust that acts as a backend for AI-powered editor features like LLM chat and code compl… | 28 | 3201 | maintenance |
| eureka-research/Eureka Eureka is a research library from NVIDIA that uses coding LLMs like GPT-4 to automatically generate reward functions for reinforcement lear… | 27 | 3189 | maintenance |
| huawei-noah/Pretrained-Language-Model A collection of pretrained language models and optimization techniques from Huawei Noah's Ark Lab, including PanGu-α (200B-parameter Chines… | 32 | 3165 | maintenance |
| DAMO-NLP-SG/Video-LLaMA Video-LLaMA is an instruction-tuned audio-visual language model that extends LLaMA with video and audio understanding via cross-modal pretr… | 29 | 3139 | maintenance |
| linyiLYi/bilibot A local chatbot fine-tuned from Bilibili user comments, built on Qwen1.5-32B-Chat using Apple's MLX LoRA fine-tuning. It supports text chat… | 24 | 3139 | maintenance |
| Kyle-Ye/XcodeLLMEligible A shell-based utility that overrides macOS eligibility checks to enable Xcode LLM, Apple Intelligence, and iPhone Mirroring on unsupported … | 10 | 3121 | maintenance |
| qwopqwop200/GPTQ-for-LLaMa A Python library that applies GPTQ 4-bit weight quantization to LLaMA large language models, drastically reducing memory usage and checkpoi… | 30 | 3073 | maintenance |
| reloadware/reloadium Reloadium is a Python library and PyCharm plugin providing advanced hot reloading, profiling, and AI-assisted error fixing. It lets develop… | 32 | 2988 | maintenance |
| mpoon/gpt-repository-loader A Python command-line tool that converts a Git repository's files and structure into a single text format suitable for feeding to LLMs. It … | 30 | 2973 | maintenance |
| johnma2006/mamba-minimal A single-file, readable PyTorch implementation of the Mamba selective state space model architecture. It produces numerically equivalent ou… | 26 | 2965 | maintenance |
| FreedomIntelligence/LLMZoo LLM Zoo is a project providing data, models, and evaluation benchmarks for large language models, including the multilingual Phoenix and Ch… | 30 | 2935 | maintenance |
| Tencent/PhoenixGo PhoenixGo is a Go (weiqi/baduk) AI engine from Tencent that implements the AlphaGo Zero approach of learning the game without human knowled… | 23 | 2917 | maintenance |
| muellerberndt/mini-agi MiniAGI is a simple general-purpose autonomous AI agent built on the OpenAI API, compatible with GPT-3.5-Turbo and GPT-4. It combines a rob… | 30 | 2899 | maintenance |