function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| danveloper/flash-moe An application for running large language models locally on small laptops, written in Objective-C. It enables on-device inference of big mo… | 48 | 4086 | active |
| hao-ai-lab/FastVideo FastVideo is a unified Python framework for post-training and real-time inference of video diffusion models, covering data preprocessing, f… | 80 | 4076 | active |
| TransformerOptimus/SuperAGI SuperAGI is a dev-first open source framework for building, managing, and running autonomous AI agents powered by large language models. It… | 21 | 17660 | maintenance |
| FedML-AI/FedML FedML (TensorOpera) is a unified Python library for large-scale distributed training, model serving, and federated learning across GPU clou… | 45 | 4062 | active |
| anymorph-ai/Claudable Claudable is an open-source, Next.js-based web app builder that uses local CLI coding agents like Claude Code, Codex, Gemini CLI, Qwen Code… | 44 | 4045 | active |
| defog-ai/sqlcoder SQLCoder is a family of fine-tuned open-weight LLMs that convert natural language questions into SQL queries, outperforming GPT-4 on text-t… | 28 | 4044 | active |
| knownsec/aipyapp AiPy (Python-Use) is an AI desktop assistant from Knownsec that implements a 'Code is Agent' paradigm: it gives an LLM a full Python execut… | 62 | 4017 | active |
| nyldn/claude-octopus A Claude Code plugin that orchestrates multiple AI model providers (Codex, Copilot, Gemini, Ollama, and others) against the same task, usin… | 78 | 4010 | active |
| microsoft/PromptWizard PromptWizard is a discrete prompt optimization framework from Microsoft that uses a self-evolving mechanism where an LLM generates, critiqu… | 45 | 4007 | active |
| zml/zml ZML is a production LLM inference stack written in Zig, built on MLIR and OpenXLA, that compiles models to run at peak performance across N… | 72 | 4003 | active |
| jackMort/ChatGPT.nvim A Neovim plugin written in Lua that integrates OpenAI's ChatGPT API directly into the editor. It provides interactive Q&A, persona-based co… | 50 | 3999 | active |
| ucbepic/docetl DocETL is a Python library and CLI for building LLM-powered data processing and ETL pipelines over structured and unstructured data using d… | 86 | 3995 | active |
| hanshuaikang/AI-Media2Doc A self-hostable web application that uses AI large language models to convert video and audio into various document styles such as Xiaohong… | 57 | 3994 | active |
| OpenMinis/OpenMinis OpenMinis is a free, open-source mobile AI agent app for iOS and Android that connects leading LLM providers (Claude, GPT, Gemini, etc.) vi… | 69 | 3985 | active |
| lemony-ai/cascadeflow Cascadeflow is an open-source agent runtime intelligence layer (Python and TypeScript SDKs) that sits inside the AI agent loop to observe, … | 75 | 3980 | active |
| QwenLM/Qwen3-Omni Qwen3-Omni is a natively end-to-end omni-modal large language model from Alibaba Cloud's Qwen team that understands text, images, audio, an… | 52 | 3980 | active |
| gusye1234/nano-graphrag nano-graphrag is a lightweight, ~1100-line Python implementation of Microsoft's GraphRAG, designed to be small, fast, and easy to read or h… | 54 | 3974 | active |
| UditAkhourii/adhd ADHD is a TypeScript skill for coding agents built on the Claude and Codex Agent SDKs that implements parallel divergent ideation via tree-… | 65 | 3953 | active |
| city96/ComfyUI-GGUF A ComfyUI custom node pack that adds GGUF quantization support for loading diffusion models like FLUX and Stable Diffusion 3.5 in low-bitra… | 51 | 3953 | active |
| ML-GSAI/LLaDA Official PyTorch implementation of LLaDA, a family of large language diffusion models (8B base/instruct, MoE, and iLLaDA variants) with pre… | 62 | 3943 | active |
| googleapis/python-genai The official Google Gen AI Python SDK for integrating Google's generative models (Gemini Developer API and Gemini Enterprise Agent Platform… | 84 | 3942 | active |
| OpenHealthForAll/open-health OpenHealth is a self-hostable AI health assistant that consolidates personal health data (blood tests, checkups, wearables, symptoms) into … | 50 | 3942 | active |
| Nunchaku Nunchaku is a high-performance inference engine for 4-bit quantized diffusion models (and LLMs) based on the SVDQuant technique from an ICL… | 65 | 3937 | active |
| Lightning-AI/LitServe LitServe is a Python framework for building custom AI inference servers with full control over batching, routing, streaming, and scaling lo… | 92 | 3930 | active |
| Paper2Poster/Paper2Poster Paper2Poster is an open-source multi-agent system (PosterAgent) that converts scientific paper PDFs into editable PowerPoint posters using … | 58 | 3923 | active |
| badrisnarayanan/antigravity-claude-proxy A Node.js proxy server that exposes Antigravity's Cloud Code Claude and Gemini models behind an Anthropic-compatible Messages API. It trans… | 74 | 3919 | active |
| evalstate/fast-agent fast-agent is a Python CLI-first framework for building, running, and evaluating LLM agents with first-class MCP, ACP, and A2A protocol sup… | 83 | 3902 | active |
| iree-org/iree IREE is an MLIR-based end-to-end machine learning compiler and runtime that lowers models from frameworks like PyTorch, TensorFlow, JAX, an… | 87 | 3901 | active |
| DaKheera47/job-ops A self-hosted job search assistant that aggregates searches across 10+ job boards, AI-scores role fit, tailors CVs per posting, and tracks … | 83 | 3884 | active |
| huggingface/smollm Hugging Face's repository for the SmolLM and SmolVLM families of compact, fully open language and vision-language models, including trainin… | 59 | 3884 | active |
| LazyAGI/LazyLLM LazyLLM is a low-code Python framework for building multi-agent LLM applications, covering prototype assembly, data feedback, and iterative… | 91 | 3875 | active |
| NVlabs/VILA VILA is a family of open-source vision language models (VLMs) optimized for efficient video and multi-image understanding, spanning edge, d… | 57 | 3857 | active |
| anthropics/anthropic-sdk-python The official Python SDK for accessing the Anthropic Claude API. It provides a typed client for creating messages, streaming responses, and … | 94 | 3855 | stable |
| circlemind-ai/fast-graphrag fast-graphrag is a Python library providing a streamlined, promptable GraphRAG framework for interpretable, high-precision retrieval workfl… | 44 | 3849 | active |
| openJiuwen-ai/jiuwenswarm JiuwenSwarm is an open-source multi-agent AI system built on openJiuwen that lets a Leader agent decompose complex tasks and coordinate tea… | 82 | 3846 | active |
| predibase/lorax LoRAX is a multi-LoRA inference server that serves thousands of fine-tuned LLM adapters on a single GPU by sharing a common base model. It … | 62 | 3826 | active |
| TransformerLensOrg/TransformerLens TransformerLens is a Python library for mechanistic interpretability of GPT-style transformer language models. It lets researchers load tho… | 99 | 3825 | active |
| HKUDS/Paper2Slides Paper2Slides is a Python application that converts research papers, reports, and documents into professional presentation slides and poster… | 53 | 3822 | active |
| tbxark/ChatGPT-Telegram-Workers A Telegram ChatGPT bot deployable on Cloudflare Workers, Vercel, or Docker with a single-file, dependency-free setup. It supports multiple … | 66 | 3811 | active |
| 1186258278/OpenClawChineseTranslation A Chinese-localized distribution of OpenClaw, an open-source personal AI assistant that connects Claude/ChatGPT/Gemini LLMs to messaging pl… | 76 | 3806 | active |
| meta-llama/codellama Official Meta repository with inference code for the Code Llama family of large language models for code, based on Llama 2. It provides min… | 10 | 16265 | maintenance |
| CommandCodeAI/command-code Command Code is a terminal-based AI coding agent that builds full-stack projects, fixes bugs, writes tests, and refactors code. Its disting… | 66 | 3793 | active |
| jihe520/MathModelAgent MathModelAgent is a multi-agent LLM application that automates mathematical modeling competitions end-to-end: it analyzes a problem, builds… | 87 | 3791 | active |
| snap-stanford/Biomni Biomni is a general-purpose biomedical AI agent from Stanford that autonomously executes research tasks across biomedical subfields. It com… | 71 | 3790 | active |
| SciSharp/LLamaSharp LLamaSharp is a cross-platform C#/.NET library that wraps llama.cpp to run LLaMA-family and other LLMs (including multimodal LLaVA) efficie… | 92 | 3787 | active |
| gofireflyio/aiac aiac is a command line tool and Go library that uses LLM providers such as OpenAI, Amazon Bedrock, and Ollama to generate Infrastructure-as… | 55 | 3786 | active |
| Polymarket/agents A Python developer framework and utility set for building AI agents that trade autonomously on Polymarket prediction markets. It integrates… | 10 | 3777 | active |
| PeterH0323/Streamer-Sales Streamer-Sales is a fine-tuned LLM-based AI sales livestreamer application that generates persuasive product commentary from product descri… | 27 | 3761 | active |
| shobrook/adrenaline Adrenaline is an AI-powered assistant that answers technical questions about programming concepts, GitHub repositories, documentation, and … | 32 | 3741 | active |
| 1rgs/claude-code-proxy A Python proxy server that translates Anthropic API requests so Anthropic clients like Claude Code can run against OpenAI, Gemini, or Anthr… | 60 | 3736 | active |
| ilysenko/codex-desktop-linux An unofficial community-maintained Linux distribution of OpenAI's ChatGPT desktop app (formerly Codex), repackaged from the official signed… | 60 | 3719 | active |
| llm-workflow-engine/llm-workflow-engine LLM Workflow Engine (LWE) is a power CLI and workflow manager for interacting with LLMs like ChatGPT/GPT-4 from the terminal. It supports m… | 84 | 3717 | active |
| rahulnyk/knowledge_graph A Python/Jupyter Notebook project that converts any text corpus into a knowledge graph by extracting concepts with an LLM and linking co-oc… | 71 | 3713 | active |
| PaddlePaddle/FastDeploy FastDeploy is a high-performance inference and deployment toolkit for large language models and vision-language models built on PaddlePaddl… | 88 | 3711 | active |
| zou-group/textgrad TextGrad is a Python framework that implements automatic 'differentiation' via text, using LLM-generated natural language feedback as gradi… | 29 | 3706 | active |
| thu-ml/SageAttention SageAttention is a family of quantized attention kernels (INT8/FP8/FP4) that accelerate transformer inference 2-5x over FlashAttention with… | 40 | 3684 | active |
| Stability-AI/StableLM StableLM is Stability AI's repository of open-weight decoder-only transformer language models, including the 3B-parameter StableLM-3B-4E1T … | 30 | 15684 | maintenance |
| memodb-io/Acontext Acontext is an open-source memory layer for AI agents that captures learnings from agent runs and stores them as human-readable Markdown sk… | 70 | 3676 | active |
| Yue-Yang/ChatGPT-Siri A set of Apple Shortcuts that connect Siri to the ChatGPT API, turning Siri into an AI chat assistant on iPhone, iPad, Mac, HomePod, and Ca… | 74 | 3653 | active |
| zai-org/ChatGLM2-6B ChatGLM2-6B is an open-source bilingual (Chinese-English) 6B-parameter conversational large language model built on the GLM architecture. I… | 29 | 15528 | maintenance |
| steipete/oracle Oracle is a TypeScript CLI and MCP server that bundles a prompt with selected project files and sends that context to frontier AI models (G… | 82 | 3648 | active |
| EverMind-AI/Raven Raven is an open-source, self-improving agent harness built on EverOS that combines terminal-first execution, local tracing, long-term memo… | 81 | 3640 | active |
| NExT-GPT/NExT-GPT NExT-GPT is an end-to-end any-to-any multimodal large language model that accepts and generates arbitrary combinations of text, image, vide… | 37 | 3638 | active |
| PublicAffairs/openai-gemini A serverless proxy that translates OpenAI API requests into Google Gemini API calls, letting any OpenAI-only tool use Gemini's free tier. I… | 59 | 3630 | active |
| thu-ml/TurboDiffusion TurboDiffusion is a Python framework that accelerates end-to-end video diffusion model generation by 100-200x using SageAttention, Sparse-L… | 59 | 3623 | active |
| mit-han-lab/llm-awq AWQ is a Python library implementing Activation-aware Weight Quantization for compressing large language models to low-bit (INT3/4) weights… | 44 | 3621 | active |
| sooryathejas/METATRON METATRON is a CLI-based AI penetration testing assistant that runs a local LLM (via Ollama) entirely offline on Linux, primarily Parrot OS.… | 48 | 3616 | active |
| microsoft/mcp Microsoft's official catalog and monorepo of MCP (Model Context Protocol) server implementations, including Azure MCP and Microsoft Fabric … | 86 | 3604 | active |
| kagent-dev/kagent kagent is a Kubernetes-native framework for building, deploying, and managing AI agents as first-class Kubernetes resources via CRDs. It su… | 83 | 3595 | active |
| whoiskatrin/chart-gpt Chart-GPT is a web application that generates charts from natural language text input using AI. Users describe what they want to visualize … | 42 | 3583 | active |
| PlexPt/chatgpt-java An unofficial Java SDK that wraps OpenAI's ChatGPT API for GPT-3.5, GPT-4, GPT-4o, and GPT-5 models, published on Maven Central with a flue… | 46 | 3579 | active |
| urchade/GLiNER GLiNER is a Python framework for training and deploying lightweight, zero-shot Named Entity Recognition models that can extract arbitrary e… | 87 | 3574 | active |
| CosmosShadow/gptpdf A small Python library that parses PDF files into Markdown using a vision-capable LLM such as GPT-4o. It uses PyMuPDF to detect non-text ar… | 31 | 3564 | active |
| RUC-NLPIR/FlashRAG FlashRAG is a modular Python toolkit for reproducing and developing Retrieval-Augmented Generation (RAG) research, bundling 36 pre-processe… | 68 | 3557 | active |
| raullenchai/Rapid-MLX Rapid-MLX is a local LLM inference engine for Apple Silicon Macs built on MLX, exposing a drop-in OpenAI/Anthropic-compatible API. It offer… | 78 | 3551 | active |
| AIDotNet/OpenDeepWiki OpenDeepWiki is an AI-driven platform that turns Git repositories, ZIP archives, and local directories into searchable, structured document… | 86 | 3550 | active |
| CookSleep/gpt_image_playground A web-based playground for generating and editing images via the OpenAI gpt-image-2 API, built with React, TypeScript, and Tailwind CSS. It… | 77 | 3549 | active |
| microsoft/hummingbird Hummingbird is a Microsoft library that compiles trained traditional ML models (decision trees, random forests, XGBoost, LightGBM) into ten… | 37 | 3544 | active |
| deta/surf Deta Surf is an open-source, local-first AI notebook for macOS, Windows, and Linux that lets users organize files, PDFs, websites, and YouT… | 77 | 3542 | active |
| rashadphz/farfalle Farfalle is an open-source, self-hosted AI-powered search engine that works like Perplexity, answering questions with cited web search resu… | 24 | 3541 | active |
| SkyworkAI/DeepResearchAgent DeepResearchAgent is a Python framework for hierarchical multi-agent systems that combines a top-level planning agent with specialized lowe… | 68 | 3532 | active |
| fnnx-ai/scikit-llm Scikit-LLM is a Python library that integrates large language models like GPT into scikit-learn workflows as estimators. It enables tasks s… | 72 | 3529 | active |
| EverMind-AI/MSA MSA (Memory Sparse Attention) is a Python framework for end-to-end trainable sparse latent-memory attention that scales LLM context to 100M… | 53 | 3515 | active |
| google-deepmind/alphafold Open-source implementation of the AlphaFold 2 inference pipeline for predicting protein structures from amino acid sequences, including Alp… | 58 | 14811 | maintenance |
| liu-ziting/what-to-eat An AI-powered recipe generation web platform built with Vue 3 and TypeScript that creates recipes across China's eight major cuisines plus … | 45 | 3500 | active |
| PKU-YuanGroup/Video-LLaVA Video-LLaVA is a large vision-language model that aligns image and video representations into a unified visual space before projection into… | 27 | 3500 | active |
| liujuntao123/smart-excalidraw-next A web-based AI-powered diagramming application built on Excalidraw that generates professional charts and diagrams from natural language de… | 45 | 3494 | active |
| NVIDIA/Model-Optimizer NVIDIA Model Optimizer (ModelOpt) is a Python library of state-of-the-art model optimization techniques including quantization, pruning, di… | 91 | 3488 | active |
| fauxpilot/fauxpilot FauxPilot is a self-hosted, open-source alternative to the GitHub Copilot code completion server. It serves SalesForce CodeGen models via N… | 32 | 14691 | maintenance |
| huggingface/optimum Optimum is a Hugging Face library that extends Transformers, Diffusers, timm, and Sentence Transformers with hardware-specific optimization… | 98 | 3469 | active |
| MiniMax-AI/MiniMax-01 Official repository for MiniMax-Text-01 and MiniMax-VL-01, open-weight large language and vision-language models built on a linear attentio… | 34 | 3466 | active |
| JusticeRage/Gepetto Gepetto is an IDA Pro plugin that queries large language models (OpenAI, Anthropic, Gemini, Ollama, and others) to explain decompiled funct… | 79 | 3461 | active |
| n3d1117/chatgpt-telegram-bot A self-hosted Telegram bot written in Python that integrates with OpenAI's official ChatGPT, DALL·E, and Whisper APIs to answer questions, … | 33 | 3460 | active |
| entropy-research/Devon Devon is an open-source AI pair programmer that acts as an autonomous software engineering agent, installable via pipx (backend) and npm (U… | 35 | 3453 | active |
| HanaokaYuzu/Gemini-API A reverse-engineered asynchronous Python client library for Google's Gemini web app (formerly Bard), published as gemini-webapi on PyPI. It… | 92 | 3437 | active |
| Kedreamix/Linly-Talker Linly-Talker is a Python-based digital avatar conversational system that combines LLMs (Linly, Qwen, Gemini), speech recognition (Whisper, … | 48 | 3436 | active |
| openai/openai-go The official Go client library for the OpenAI REST API, providing typed access to models, the Responses API, conversations, and other OpenA… | 86 | 3434 | active |
| Mirix-AI/MIRIX MIRIX is a self-hosted multi-agent personal assistant that captures on-screen activity and conversation to build structured long-term memor… | 72 | 3434 | active |
| Qwen3-ASR Qwen3-ASR is a family of open-source speech recognition models from Alibaba's Qwen team, supporting ASR and language identification across … | 55 | 3423 | active |
| Grt1228/chatgpt-java An unofficial Java SDK for the OpenAI API covering all official endpoints including chat completions (GPT-3.5/GPT-4), DALL-E image generati… | 21 | 3423 | active |