function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| iflytek/astron-agent Astron Agent is an enterprise-grade, commercial-friendly agentic workflow platform for building, orchestrating, and deploying multi-agent A… | 83 | 8911 | active |
| shaxiu/XianyuAutoAgent An AI-powered customer service bot for the Xianyu (Goofish) secondhand marketplace that provides 24/7 automated message handling. It uses L… | 59 | 8836 | active |
| bentoml/BentoML BentoML is a Python framework for building online serving systems for AI apps and model inference, turning model inference scripts into RES… | 94 | 8808 | stable |
| Anxcye/anx-reader Anx Reader is a cross-platform e-book reader app built with Flutter that supports EPUB, MOBI, AZW3, FB2, TXT, and PDF formats. It integrate… | 78 | 8750 | active |
| freecodexyz/free-code A community fork of Anthropic's Claude Code terminal AI coding agent, rebuilt from an exposed source snapshot with telemetry removed, secur… | 57 | 8722 | active |
| reorproject/reor Reor is a local-first, AI-powered desktop note-taking app that automatically links related notes, answers questions over your notes via RAG… | 10 | 8568 | active |
| OpenBMB/XAgent XAgent is an open-source, LLM-driven autonomous agent framework for automatically solving complex tasks. It features a dispatcher and plann… | 61 | 8544 | active |
| HKUDS/ClawWork ClawWork is an open-source benchmark and framework that turns AI assistants into 'AI coworkers' which must earn income by completing real p… | 47 | 8522 | active |
| google-deepmind/alphafold3 Google DeepMind's official implementation of the AlphaFold 3 inference pipeline for predicting biomolecular structures and interactions. It… | 83 | 8495 | active |
| OptimalScale/LMFlow LMFlow is an extensible Python toolkit for finetuning and inference of large foundation models such as LLaMA, GPT-2, and Galactica. It prov… | 64 | 8484 | active |
| bitsandbytes-foundation/bitsandbytes bitsandbytes is a Python library providing k-bit quantization primitives for PyTorch, enabling 8-bit (LLM.int8()) and 4-bit (QLoRA) quantiz… | 99 | 8439 | active |
| alvinunreal/oh-my-opencode-slim A lightweight agent orchestration plugin for OpenCode that delegates tasks to a team of seven specialized sub-agents (Orchestrator, Explore… | 78 | 8437 | active |
| Nutlope/logocreator An open-source AI logo generator web app that creates brand-ready logos using FLUX models on Together AI, with logo editing, style presets,… | 64 | 8435 | active |
| ogx-ai/ogx OGX (formerly Llama Stack) is an open-source, OpenAI-compatible agentic API server and Python library that bundles inference, vector stores… | 86 | 8416 | active |
| 0xPlaygrounds/rig Rig is a Rust library for building modular, scalable LLM-powered applications and agents, offering a unified API across 20+ model providers… | 86 | 8415 | active |
| FlashML-org/FreeToken FreeToken is an edge-native Mixture-of-Experts (MoE) LLM serving engine that runs frontier-scale open-weight models on consumer hardware by… | 68 | 8370 | active |
| zai-org/ChatGLM-6B ChatGLM-6B is an open-source bilingual (Chinese-English) 6.2B-parameter dialogue language model based on the GLM architecture, optimized fo… | 30 | 40983 | maintenance |
| qualcomm/GenieX GenieX is an on-device generative AI inference runtime for Qualcomm Snapdragon devices, running frontier LLMs and VLMs on the Hexagon NPU, … | 85 | 8326 | active |
| intitni/CopilotForXcode An Xcode Source Editor Extension that brings GitHub Copilot, Codeium, and ChatGPT-powered AI assistance to Xcode on macOS. It provides inli… | 78 | 8225 | active |
| duixcom/Duix-Mobile Duix Mobile is an open-source SDK for building real-time interactive AI avatars (digital humans) that run on-device on Android, iOS, tablet… | 69 | 8198 | active |
| k8sgpt-ai/k8sgpt k8sgpt is a CLI tool that scans Kubernetes clusters, diagnoses issues, and triages them in plain English using AI backends like OpenAI, Azu… | 97 | 8120 | active |
| lm-sys/FastChat FastChat is an open platform for training, serving, and evaluating large language model based chatbots, originally releasing the Vicuna mod… | 57 | 39525 | maintenance |
| a-ghorbani/pocketpal-ai PocketPal AI is an open-source mobile app that runs GGUF language models entirely on-device for private, offline chat. It supports model do… | 85 | 8067 | active |
| elder-plinius/OBLITERATUS OBLITERATUS is a Python toolkit for abliteration — identifying and removing refusal behaviors from large language models by locating and in… | 71 | 8056 | active |
| withastro/flue Flue is an open-source TypeScript framework for building autonomous AI agents using a React-like hooks API ('use agent' directive, useModel… | 58 | 8039 | active |
| InternLM/lmdeploy LMDeploy is a toolkit for compressing, quantizing, deploying, and serving large language models, built around its high-performance TurboMin… | 95 | 8024 | active |
| anthropics/claude-agent-sdk-python The official Python SDK for building agents on top of Claude Code, providing async query APIs, bidirectional conversations, custom tools vi… | 81 | 7981 | active |
| Upsonic/Upsonic Upsonic is a Python framework for building autonomous and traditional AI agents, with support for tools, memory, knowledge bases, RAG, mult… | 77 | 7948 | active |
| NVIDIA/Isaac-GR00T NVIDIA Isaac GR00T N1.7 is an open vision-language-action (VLA) foundation model for generalized humanoid robot skills, taking language and… | 76 | 7926 | stable |
| RayVentura/ShortGPT ShortGPT is a Python framework that automates AI-driven video content creation, combining LLM script generation, voiceover synthesis, asset… | 33 | 7884 | active |
| ai-dynamo/dynamo NVIDIA Dynamo is an open-source, datacenter-scale distributed inference serving framework that orchestrates inference engines like vLLM, SG… | 87 | 7872 | active |
| AI4Finance-Foundation/FinRobot FinRobot is an open-source AI agent platform for financial applications built on large language models, unifying LLMs, reinforcement learni… | 81 | 7866 | active |
| PriorLabs/TabPFN TabPFN is a pre-trained tabular foundation model (transformer) that performs classification, regression, forecasting, and anomaly detection… | 99 | 7855 | active |
| OpenCoworkAI/open-codesign Open CoDesign is an open-source, local-first desktop AI design tool that turns prompts into polished artifacts like UI prototypes, slide de… | 75 | 7842 | active |
| OpenDCAI/DataFlow DataFlow is a Python framework for preparing, cleaning, and synthesizing data for LLM training and RAG using composable LLM-based operators… | 80 | 7764 | active |
| 2FastLabs/agent-squad Agent Squad is an open-source framework for orchestrating multiple AI agents, routing each user query to the most suitable specialized agen… | 87 | 7748 | active |
| deepseek-ai/DeepGEMM DeepGEMM is a high-performance CUDA BLAS kernel library for NVIDIA tensor cores, providing FP8, FP4, and BF16 GEMMs plus fused MoE and othe… | 84 | 7738 | active |
| sweepai/sweep Sweep is an AI coding assistant plugin for JetBrains IDEs (IntelliJ, PyCharm, GoLand, etc.) offering an AI agent, fast next-edit autocomple… | 38 | 7706 | active |
| osaurus-ai/osaurus Osaurus is a native Swift macOS application that acts as a local-first AI harness, running open models on Apple Silicon (via MLX and Apple … | 80 | 7691 | active |
| meta-llama/llama-models Meta's official repository of Llama large language models with Python utilities for working with them, including model cards, licenses, and… | 55 | 7685 | active |
| mattpocock/sandcastle A TypeScript library for orchestrating AI coding agents (like Claude Code) inside isolated sandboxes via a single sandcastle.run() call. It… | 74 | 7674 | active |
| EricLBuehler/mistral.rs mistral.rs is a fast and flexible LLM inference engine written in Rust, supporting many model families with quantization (ISQ/UQFF, GGUF), … | 92 | 7628 | active |
| mufeedvh/code2prompt Code2Prompt is a Rust-based CLI tool that converts an entire codebase into a single structured LLM prompt, including a source tree, file co… | 66 | 7619 | active |
| lllyasviel/Omost Omost is a Python application that converts LLM coding capability into image composition by having pretrained LLMs (based on Llama3 and Phi… | 24 | 7610 | active |
| pickle-com/glass Glass by Pickle is an open-source Electron desktop app that acts as a 'digital mind extension', seeing your screen and listening in real ti… | 42 | 7589 | active |
| GoogleCloudPlatform/kubectl-ai kubectl-ai is an AI-powered Kubernetes assistant that translates natural-language user intent into precise kubectl operations. It is a Go C… | 77 | 7558 | active |
| chenyme/grok2api A self-hosted multi-account API gateway for Grok Build, Grok Web, and Grok Console, written in Go with a React frontend. It exposes Grok ca… | 84 | 7538 | active |
| di-sukharev/opencommit OpenCommit is a CLI tool that uses LLMs (OpenAI, Claude, Ollama, and other providers) to automatically generate meaningful git commit messa… | 66 | 7525 | active |
| tailcallhq/forgecode ForgeCode is an open-source, Rust-based CLI coding agent (AI pair programmer) that integrates LLMs from 300+ providers directly into the te… | 83 | 7513 | active |
| refly-ai/refly Refly is an open-source platform for building, versioning, and running reusable AI agent skills through a natural-language 'vibe workflow' … | 80 | 7495 | active |
| mnfst/manifest Manifest is an open-source LLM gateway and router that lets AI agents and apps reach all their model providers through one OpenAI- and Anth… | 94 | 7481 | active |
| apple/ml-fastvlm Official implementation of FastVLM, a vision language model with an efficient hybrid vision encoder (FastViTHD) that reduces token count an… | 28 | 7411 | active |
| microsoft/graphrag GraphRAG is a Python library and CLI pipeline from Microsoft Research that extracts a knowledge graph from unstructured text, builds commun… | 91 | 35699 | maintenance |
| farzaa/clicky Clicky is an open-source macOS app that acts as an AI teacher living next to your cursor — it can see your screen, talk with you via voice,… | 50 | 7394 | active |
| ArvinLovegood/go-stock A Wails + NaiveUI desktop application that combines stock market data (A-shares, Hong Kong, US markets) with LLM-powered analysis, includin… | 84 | 7367 | active |
| zai-org/GLM-OCR GLM-OCR is an open-source 0.9B-parameter multimodal OCR model built on the GLM-V encoder-decoder architecture for complex document understa… | 65 | 7366 | active |
| open-compass/opencompass OpenCompass is a Python-based evaluation platform and toolkit for benchmarking large language models across 100+ datasets, supporting model… | 96 | 7365 | active |
| MuiseDestiny/zotero-gpt A Zotero plugin that integrates OpenAI GPT models into the Zotero reference manager, letting users ask questions about PDFs, summarize pape… | 82 | 7361 | active |
| guardrails-ai/guardrails Guardrails is a Python framework for building reliable LLM applications by running Input/Output Guards that detect and mitigate risks, and … | 97 | 7321 | active |
| vladmandic/sdnext SD.Next is an open-source, self-hosted WebUI server application for AI generative image and video creation built on Stable Diffusion and Di… | 76 | 7320 | active |
| AAswordman/Operit Operit AI is an open-source AI agent and chat application for Android built with Kotlin and Jetpack Compose. It connects cloud or local LLM… | 86 | 7294 | active |
| teamchong/pxpipe pxpipe is a local TypeScript proxy that reduces Claude Code input token usage by rewriting bulky text context (system prompts, tool docs, h… | 80 | 7280 | active |
| PaddlePaddle/Paddle-Lite Paddle Lite is a high-performance, lightweight deep learning inference engine from Baidu's PaddlePaddle ecosystem, designed for mobile, emb… | 58 | 7273 | active |
| algorithmicsuperintelligence/openevolve OpenEvolve is an open-source Python implementation of DeepMind's AlphaEvolve, an evolutionary coding agent that uses LLM ensembles to auton… | 81 | 7271 | active |
| InternLM/InternLM Official repository for the InternLM series of open-source large language models (InternLM, InternLM2, InternLM2.5, InternLM3), including p… | 41 | 7269 | active |
| mit-han-lab/streaming-llm StreamingLLM is a research framework from MIT Han Lab implementing the Attention Sinks method (ICLR 2024) for efficient streaming language … | 27 | 7268 | stable |
| MineDojo/Voyager Voyager is an open-source LLM-powered embodied agent that plays Minecraft through GPT-4, using an automatic curriculum, an ever-growing ski… | 29 | 7160 | active |
| Nutlope/llamacoder Llama Coder is an open-source web application that generates small apps from a single prompt, similar to Claude Artifacts, powered by Llama… | 67 | 7120 | active |
| ymcui/Chinese-LLaMA-Alpaca-2 A project releasing Chinese-adapted LLaMA-2 base models and Alpaca-2 instruction-tuned chat models with an extended Chinese vocabulary, inc… | 54 | 7120 | stable |
| chenfei-wu/TaskMatrix TaskMatrix (Visual ChatGPT) is a Python framework that connects ChatGPT with a suite of visual foundation models like Stable Diffusion, Gro… | 30 | 34003 | maintenance |
| kyegomez/swarms Swarms is an enterprise-grade Python framework for orchestrating multi-agent systems powered by large language models, offering prebuilt se… | 65 | 7087 | active |
| zai-org/GLM-4 Official repository for the GLM-4 series of open-weight, multilingual (primarily Chinese/English) multimodal chat language models from Z.ai… | 67 | 7070 | active |
| zai-org/GLM-5 GLM-5 is a family of open-weight large language models from Z.ai (Zhipu AI), including the flagship GLM-5.2 with 1M-token context and the m… | 60 | 7057 | active |
| deepseek-ai/DeepSpec DeepSpec is a full-stack Python codebase from DeepSeek for training and evaluating draft models used in speculative decoding of large langu… | 54 | 7041 | active |
| google/gemma.cpp A lightweight, standalone C++ inference engine for Google's Gemma foundation models (Gemma 2/3, PaliGemma 2), with a small ~2K LoC core and… | 72 | 7031 | active |
| Strands Agents Strands Agents is an open-source SDK (Python and TypeScript) for building production AI agents with a model-driven approach, supporting any… | 81 | 7022 | active |
| zhouxiaoka/autoclip AutoClip is a self-hosted AI-powered video clipping system that downloads videos from YouTube and Bilibili, uses LLM-based content analysis… | 66 | 7019 | active |
| NVIDIA-NeMo/Guardrails NVIDIA NeMo Guardrails is an open-source Python toolkit for adding programmable guardrails to LLM-based conversational applications. It sit… | 93 | 7017 | active |
| interviewstreet/hiring-agent An open-source AI agent pipeline from HackerRank that extracts structured data from resume PDFs, enriches it with GitHub signals, and produ… | 60 | 7007 | active |
| deepseek-ai/DeepSeek-Coder-V2 DeepSeek-Coder-V2 is an open-weight Mixture-of-Experts code language model family released by DeepSeek AI, with weights available on Huggin… | 47 | 7007 | active |
| TheTom/turboquant_plus TurboQuant+ is a Python reference implementation of the TurboQuant KV cache compression method (ICLR 2026), using PolarQuant codebooks and … | 56 | 7006 | active |
| dwzhu-pku/PaperBanana PaperBanana is a reference-driven multi-agent framework that automatically generates publication-ready academic illustrations such as metho… | 55 | 7000 | active |
| mckaywrigley/chatbot-ui Chatbot UI is an open-source, self-hostable web chat application for conversing with any AI model, with chat history and multi-model suppor… | 30 | 33340 | maintenance |
| albertan017/LLM4Decompile LLM4Decompile is an open-source series of large language models (1.3B to 33B) trained to decompile binary code back into readable, executab… | 55 | 6986 | active |
| JerryZLiu/Dayflow Dayflow is an open-source, local-first macOS app that automatically turns your screen activity into a timeline of what you accomplished, us… | 79 | 6948 | active |
| mishushakov/llm-scraper A TypeScript library that turns any webpage into structured data using LLMs, built on Playwright and the Vercel AI SDK. It supports multipl… | 67 | 6915 | active |
| yjh051108/dsh-routing-suite A bundled kit for the DSH (DeepSeek harness) agent runtime combining a runtime injector plugin with task-aware reasoning-mode router preset… | 68 | 6853 | active |
| leejet/stable-diffusion.cpp A pure C/C++ inference engine for diffusion models (Stable Diffusion, FLUX, Wan, Qwen Image, Z-Image, and more) built on ggml in the style … | 91 | 6846 | active |
| open-multi-agent/open-multi-agent Open Multi-Agent (OMA) is an MIT-licensed TypeScript framework for multi-agent AI orchestration that runs inside Node.js backends. A coordi… | 81 | 6831 | active |
| olimorris/codecompanion.nvim CodeCompanion.nvim is a Neovim plugin that integrates LLMs (Anthropic, OpenAI, Gemini, Copilot, Ollama, and more) and CLI coding agents (Cl… | 89 | 6824 | active |
| postgresml/postgresml PostgresML is a PostgreSQL extension that brings machine learning and AI inference directly into the database, with GPU acceleration and pg… | 36 | 6821 | active |
| ml-explore/mlx-lm A Python package for running and fine-tuning large language models on Apple silicon using the MLX framework. It integrates with the Hugging… | 80 | 6801 | active |
| stagewise-io/stagewise Stagewise is an open-source agentic IDE with a built-in coding agent that can see live app context, run git workflows, and orchestrate mult… | 81 | 6789 | active |
| lingfengQAQ/webnovel-writer A Claude Code plugin for long-form serialized webnovel writing, built in Python. It provides slash-command skills for initializing story se… | 74 | 6788 | active |
| SWE-agent/mini-swe-agent A radically minimal AI software engineering agent (~100 lines of core Python) that solves GitHub issues and assists in the command line, sc… | 79 | 6779 | active |
| purocean/yn Yank Note is a highly extensible desktop Markdown editor built on Electron and the Monaco editor, with features like version control, docum… | 96 | 6744 | active |
| superagent-ai/superagent Superagent is an open-source SDK and platform for securing AI applications and agents, offering runtime guardrails that block prompt inject… | 78 | 6719 | active |
| HaujetZhao/CapsWriter-Offline CapsWriter-Offline is a fully offline voice input tool for Windows that transcribes speech to text when you hold CapsLock or mouse side but… | 92 | 6691 | active |
| dataease/SQLBot SQLBot is an open-source conversational data analysis (ChatBI) system that converts natural language questions into SQL queries using large… | 86 | 6688 | active |
| opensquilla/opensquilla OpenSquilla is a token-efficient, microkernel AI agent that routes each conversation turn to the cheapest capable LLM via a local model rou… | 81 | 6687 | active |