function: llm-inference

3362 products, primary matches first, then adoption-weighted; health v2 shown.

Product	Health v2	Stars	Maturity
haibbo/cf-openai-azure-proxy A Cloudflare Worker script that proxies OpenAI-compatible API requests to Azure OpenAI Service, letting standard OpenAI clients use Azure d…	21	1608	active
modem-works/dream-recorder Dream Recorder is an open-source DIY bedside device built on a Raspberry Pi 5 that records users speaking their dreams aloud and turns them…	41	1607	active
clusterzx/paperless-ai A self-hosted web application that automatically analyzes, tags, and classifies documents in Paperless-ngx using LLMs via OpenAI-compatible…	73	5910	maintenance
THUDM/WebGLM WebGLM is an efficient web-enhanced question answering system (KDD 2023) that combines a large language model with web search retrieval and…	35	1602	active
Roy3838/Observer Observer AI is a desktop application for building micro-agents that observe screen, camera, microphone, and audio inputs, process them with…	87	1601	active
gi-dellav/zerostack zerostack is a minimal coding agent written in Rust, optimized for a tiny memory footprint (~16 MB RAM) and fast performance. It provides a…	79	1600	active
Norsico/Video-Materials-AutoGEN-Workstation A self-hosted short-video production workstation that combines AI script generation (Gemini), batch TTS voiceover, AI image asset synthesis…	54	1599	active
m1guelpf/plz-cli plz is a Rust-based CLI tool that acts as 'Copilot for your terminal', generating bash shell scripts from natural language descriptions usi…	22	1596	active
enoch3712/ExtractThinker ExtractThinker is a Python document intelligence library that uses LLMs to extract and classify structured data from documents like PDFs, i…	45	1595	active
RamiAwar/dataline DataLine is an open-source, privacy-focused AI data analysis and visualization application that lets users chat with their data in natural …	58	1594	active
UbiquitousLearning/mllm MLLM is a fast, lightweight multimodal LLM inference engine written in C++ for mobile and edge devices, with backends for ARM CPU, Qualcomm…	73	1593	active
qnguyen3/chat-with-mlx A chat playground application for running LLMs locally on Apple Silicon Macs using Apple's MLX framework. It provides a chat UI with suppor…	25	1593	active
tphakala/birdnet-go BirdNET-Go is a self-hosted, 24/7 realtime soundscape analyser that classifies birds, bats, and other wildlife using local AI models like B…	94	1592	active
ponysb/91Writing A browser-based AI novel-writing application built with Vue 3 and Element Plus that integrates user-configured LLM APIs (GPT, Claude, Gemin…	40	1591	active
iBUHub/AIStudioToAPI A self-hosted proxy service that wraps the Google AI Studio Build web interface into OpenAI, Gemini, and Anthropic compatible APIs using br…	82	1589	active
D-Star-AI/dsRAG dsRAG is a high-performance retrieval engine for unstructured data, implemented as a Python library for building RAG pipelines. It improves…	48	1589	active
intel/auto-round AutoRound is Intel's advanced quantization toolkit for LLMs and vision-language models, using sign-gradient descent to achieve high accurac…	91	1588	active
MoonshotAI/Kimi-Linear Kimi Linear is a hybrid linear attention architecture (Kimi Delta Attention, based on Gated DeltaNet) released by Moonshot AI with 48B-para…	40	1588	active
Tencent-Hunyuan/HY-WorldPlay HY-WorldPlay (HY-World 1.5) is Tencent Hunyuan's open-source framework for interactive 3D world modeling, generating explorable 3D scenes f…	55	1586	active
pytorch/FBGEMM FBGEMM is a collection of highly optimized low-precision matrix multiplication and convolution kernels for server-side deep learning infere…	93	1584	active
AkariAsai/OpenScholar OpenScholar is a retrieval-augmented language model system from the Allen Institute for AI that answers scientific questions by searching t…	38	1584	active
Alisa0808/vox-director An agent skill that turns a single topic into a finished Vox-style paper-collage explainer or ad video, automating script, keyframes, motio…	56	1582	active
ShenSeanChen/waku-agent Waku Agent is a local-first AI agent harness in Python that exposes the full agent stack — harness, loop, memory, and eval/LLM-Ops — as sma…	78	1580	active
RWKV/rwkv.cpp A C++ port of the RWKV language model to the ggml tensor library, providing FP32, FP16, and quantized INT4/INT5/INT8 inference focused on C…	37	1579	active
lynote-ai/humanize-text An open-source Python toolkit for making AI-generated drafts read more naturally, offering four documented humanization approaches plus a p…	67	1574	active
patched-codes/patchwork Patchwork is an open-source agentic AI framework and CLI that automates development chores like PR reviews, bug fixing, security patching, …	67	1572	active
t59688/arboris-novel Arboris-Novel is a self-hosted AI writing companion web application for fiction authors, built in Python and deployable via Docker. It help…	55	1570	active
MLSysOps/MLE-agent MLE-Agent is an LLM-powered CLI agent that acts as a pairing assistant for machine learning engineers and researchers. It automates ML task…	67	1568	active
cyberark/FuzzyAI FuzzyAI is an automated LLM fuzzing tool from CyberArk that tests LLM APIs for jailbreak vulnerabilities. It ships as a Python CLI with a w…	51	1568	active
hyperfield/ai-file-sorter AI File Sorter is a cross-platform desktop application that uses local or remote LLMs to categorize files and suggest cleaner filenames for…	87	1564	active
BAI-LAB/MemoryOS MemoryOS is a Python library that provides an operating-system-inspired memory layer for personalized AI agents, using a hierarchical short…	61	1561	active
evo-design/evo Evo is a 7-billion-parameter DNA foundation model built on the StripedHyena architecture, trained on ~300 billion tokens of prokaryotic who…	71	1560	active
byjlw/video-analyzer A Python CLI tool that analyzes videos by extracting key frames, transcribing audio with Whisper, and describing content using vision LLMs …	60	1559	active
unitedbyai/droidclaw DroidClaw is an AI agent that controls Android phones via ADB, using a perceive-reason-act loop over accessibility trees and screenshots to…	59	1559	active
hexdocom/lemonai Lemon AI is a full-stack open-source general AI agent platform that runs locally, offering an alternative to Manus and Genspark AI. It comb…	56	1559	active
langchain-ai/langgraph-swarm-py A Python library for building swarm-style multi-agent systems on top of LangGraph, where specialized agents dynamically hand off control to…	70	1558	active
neovateai/neovate-code Neovate Code is an open-source AI coding agent that runs in the terminal to generate code, fix bugs, review code, and add tests. It support…	61	1557	active
afc163/fanyi Fanyi is a command-line Chinese-English translation tool that aggregates results from iciba, youdao, and an LLM (with a free built-in proxy…	95	1553	active
kijai/ComfyUI-CogVideoXWrapper A ComfyUI custom node wrapper for CogVideoX and related video generation models (including Fun variants, CogVideoX 1.5, and Go-with-the-Flo…	39	1549	active
waybarrios/vllm-mlx vllm-mlx is a vLLM-style LLM inference server for Apple Silicon Macs built on native MLX, exposing both OpenAI and Anthropic compatible API…	82	1547	active
Tencent/AngelSlim AngelSlim is a Python toolkit from Tencent for compressing large language models and related architectures (VLMs, diffusion, audio models) …	72	1547	active
baaivision/Emu3.5 Emu3.5 is BAAI's native multimodal foundation model that jointly predicts next states across vision and language, trained on 10T+ interleav…	43	1547	active
Memento-Teams/Memento-Skills Memento-Skills is a generalist, continually-learnable LLM agent framework where reusable skills stored as markdown files serve as persisten…	77	1546	active
baichuan-inc/Baichuan-7B Baichuan-7B is an open-source, commercially usable 7-billion-parameter pretrained language model built on the Transformer architecture, tra…	29	5649	maintenance
RunanywhereAI/RCLI RCLI is a single-binary CLI that runs open-source AI models locally on your machine, covering chat, vision, speech-to-text, text-to-speech,…	82	1542	active
KruxAI/ragbuilder RagBuilder is a Python toolkit that automatically builds an optimal, production-ready Retrieval-Augmented Generation (RAG) pipeline for you…	35	1541	active
hyperbrowserai/HyperAgent HyperAgent is a TypeScript library and CLI that adds LLM-powered natural language commands to Playwright for browser automation. It support…	52	1540	active
svnscha/mcp-windbg A Model Context Protocol (MCP) server that bridges AI models with Windows debuggers (CDB and KD) for crash dump analysis, user-mode remote …	82	1538	active
ATH-MaaS/Marco-o1 Marco-o1 is an open large reasoning model from Alibaba International Digital Commerce, designed for o1-like chain-of-thought reasoning acro…	61	1538	active
tldraw/make-real-starter A starter template for building 'Make Real' style apps that turn hand-drawn UI mockups on a tldraw infinite canvas into working HTML using …	63	1537	active
HKUDS/OpenOPC OpenOPC is a Python framework for building a personal AI-native company that recruits role-specific AI employees, assigns and hands off tas…	58	1537	active
xLLM-AI/xllm xLLM is a high-performance C++ inference engine for LLM, VLM, DiT and recommendation models, optimized for heterogeneous AI accelerators su…	78	1536	active
Haidra-Org/AI-Horde AI Horde is a crowdsourced distributed cluster where volunteers share GPU/CPU compute to generate AI images and text for others for free. T…	77	1536	active
hhhuang/CAG Cache-Augmented Generation (CAG) is a Python implementation of a retrieval-free alternative to RAG that preloads knowledge into an LLM's co…	32	1535	active
RayBytes/ChatMock ChatMock is a Python tool that exposes OpenAI- and Ollama-compatible API endpoints backed by a ChatGPT/Codex account login. It runs a local…	68	1534	active
K-Dense-AI/karpathy Karpathy is an open-source agentic Machine Learning Engineer that trains state-of-the-art ML models using the Claude Agent SDK and Google A…	60	1532	active
ibelick/zola Zola is an open-source, self-hostable chat interface for interacting with multiple AI models including OpenAI, Claude, Gemini, Mistral, and…	45	1531	active
amd/gaia GAIA is AMD's open-source Python framework for building AI agents that run entirely locally on AMD Ryzen AI hardware, using NPU and iGPU ac…	87	1530	active
PaperDebugger/paperdebugger PaperDebugger is an AI-powered academic writing assistant delivered as a Chrome extension that integrates directly into Overleaf for in-edi…	62	1527	active
David-Kunz/gen.nvim A Neovim plugin that generates text using LLMs (primarily via Ollama) with customizable prompts. It streams model responses into editor win…	36	1527	active
Mirascope/mirascope Mirascope is a Python (and TypeScript) library providing a unified, provider-agnostic interface for calling frontier LLMs like OpenAI, Anth…	86	1522	active
Coframe/coffee Coffee is an AI-powered tool for building and iterating on React UI components directly from your IDE. It uses LLMs to generate and refine …	67	1522	active
NousResearch/autonovel An autonomous AI agent pipeline that writes, revises, evaluates, typesets, illustrates, and narrates a complete novel from a seed concept. …	48	1521	active
xiaoY233/Chat2API Chat2API is an Electron-based desktop application that exposes OpenAI-compatible API endpoints backed by official AI provider web UIs such …	70	1520	active
CopilotKit/OpenGenerativeUI An open-source framework and showcase for building generative UI, where an AI agent (CopilotKit + LangChain Deep Agents) generates interact…	53	1520	active
microsoft/Mage Mage is a family of lightweight 4B-parameter multimodal models from Microsoft, including Mage-VL for image and video understanding and Mage…	57	1516	active
andrewnguonly/Lumos Lumos is a Chrome extension that acts as a RAG-powered LLM co-pilot for browsing the web, letting users summarize pages and ask questions a…	19	1516	active
openai/openai-java The official Java SDK for accessing the OpenAI REST API, written in Kotlin and published to Maven Central. It provides typed clients for th…	84	1515	active
ATH-MaaS/Ovis Ovis is an open-source Multimodal Large Language Model (MLLM) architecture that structurally aligns visual and textual embeddings, with rel…	65	1512	active
magnitudedev/magnitude Magnitude is an open-source CLI tool and agent that bundles local LLM inference with an agent harness, profiling your hardware to recommend…	80	1510	active
miuuyy/codex-chatgpt-web A local bridge application that exposes ChatGPT Web (including Pro tiers) as a native model inside the OpenAI Codex app, routing model turn…	80	1509	active
PrathamLearnsToCode/paper2code An agent skill for coding agents like Claude Code that converts an arXiv paper URL into a working Python implementation. It generates citat…	48	1509	active
imoneoi/openchat OpenChat is a library of open-source large language models fine-tuned with C-RLFT, an offline reinforcement learning strategy that learns f…	20	5490	maintenance
langchain-ai/open-canvas Open Canvas is an open-source web application for collaborating with LLM agents to write documents and code, inspired by OpenAI's Canvas. I…	10	5490	maintenance
Pickle-Pixel/ApplyPilot ApplyPilot is an open-source AI agent that autonomously applies to jobs on your behalf across any job site or application form. It runs a 6…	60	1508	active
langchain-ai/deepagentsjs Deep Agents (deepagentsjs) is a TypeScript library from LangChain for building controllable LLM-powered agents on top of LangGraph. It prov…	80	1507	active
amicalhq/amical Amical is an open-source, local-first AI dictation app for macOS and Windows that transcribes speech with Whisper and enhances it with LLMs…	84	1506	active
kyutai-labs/unmute Unmute is a system that lets any text LLM listen and speak by wrapping it with Kyutai's low-latency speech-to-text and text-to-speech model…	60	1506	active
stepfun-ai/Step-Audio2 Step-Audio 2 is an end-to-end multimodal large language model for industry-strength audio understanding and speech conversation, with open-…	50	1503	active
jgravelle/AutoGroq AutoGroq is a Python application that dynamically generates tailored teams of AI agents for Autogen and similar frameworks based on user pr…	15	1503	active
appleboy/CodeGPT CodeGPT is a Go-based CLI tool that uses LLM APIs (OpenAI, Gemini, Anthropic, Groq, Ollama, OpenRouter) to generate git commit messages and…	92	1502	active
TIGER-AI-Lab/TheoremExplainAgent An agentic AI system that generates long-form (5+ minute) Manim animation videos explaining mathematical and STEM theorems using LLM agents…	35	1499	active
microsoft/ai-dev-gallery AI Dev Gallery is a Windows application from Microsoft that lets developers explore over 25 interactive samples powered by local AI models …	66	1497	active
QwenAudio/Fun-ASR Fun-ASR is a family of open-source LLM-based end-to-end speech recognition models from Tongyi Lab, covering Chinese, dialects, accents, and…	81	1496	active
WecoAI/aideml AIDE ML is an open-source Python package implementing an LLM-driven agent that uses agentic tree search to autonomously write, debug, and i…	65	1494	active
beehive-lab/TornadoVM TornadoVM is a Java programming framework that JIT-compiles JVM bytecode into GPU kernels targeting NVIDIA CUDA, OpenCL, and Apple Metal at…	97	1491	active
fynnfluegge/rocketnotes Rocketnotes is a web-based Markdown note-taking app with integrated AI features such as chat with your documents, text completion, voice-to…	69	1491	active
ggml-org/llama.vscode A VS Code extension providing local LLM-assisted text and code completion, AI chat, and agentic coding powered by llama.cpp. It supports au…	83	1490	active
yusufcanb/tlm tlm is a local CLI copilot powered by Ollama that suggests shell commands, explains them, and answers questions with context using open-sou…	58	1489	active
octotools/octotools OctoTools is an open-source, training-free agentic framework from Stanford that augments large language models with extensible tools for co…	56	1487	active
ekwek1/soprano Soprano is an ultra-lightweight 80M-parameter text-to-speech model and Python library for fast, expressive, high-fidelity speech synthesis …	44	1486	active
OpenGenerativeAI/llm-colosseum LLM Colosseum is a benchmarking tool that pits large language models against each other in real-time Street Fighter III matches to evaluate…	28	1484	active
mattzh72/articraft Articraft is a research tool that generates articulated 3D assets from text prompts or reference images by having an LLM write Python using…	67	1482	active
NPC-Worldwide/npcpy npcpy is a Python library providing primitives for building applications with multimodal LLMs, agentic AI, and knowledge graphs. It support…	85	1480	active
Bklieger/infinite-bookshelf Infinite Bookshelf is a Streamlit web app that generates entire nonfiction books from a one-line prompt using Llama models on Groq. It scaf…	35	1480	active
salesforce/CodeTF CodeTF is a Python transformer library for code large language models, providing unified interfaces for training, fine-tuning, and inferenc…	10	1480	active
Nutlope/napkins Napkins.dev is an open-source web application that turns screenshots or wireframes of website designs into working React + Tailwind code us…	63	1477	active
Om-Alve/smolGPT A minimal pure-PyTorch implementation for training small GPT-style LLMs from scratch, featuring flash attention, RMSNorm, SwiGLU, RoPE, and…	23	1475	active
ggml-org/Llama-macOS A native macOS menu bar app for running local LLMs, built on llama.cpp by the GGML org. It exposes an OpenAI-compatible local server at loc…	85	1472	active
openonion/connectonion ConnectOnion is a Python framework for building production-ready AI agents with tools, plugins, event hooks, and multi-agent collaboration.…	81	1470	active