function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| lmstudio-ai/lmstudio-js lmstudio-js is the official TypeScript/JavaScript client SDK for LM Studio, enabling use of locally running LLMs for chat, completions, emb… | 72 | 1761 | active |
| delibae/claude-prism ClaudePrism is an open-source, offline-first desktop application for scientific and academic writing, built with Tauri 2 and Rust. It combi… | 75 | 1759 | active |
| wwbin2017/bailing Bailing is an open-source voice assistant application similar to GPT-4o, built with an ASR + VAD + LLM + TTS pipeline (FunASR, silero-vad, … | 54 | 1757 | active |
| High-Logic/Genie-TTS GENIE is a lightweight Python inference engine for the open-source GPT-SoVITS text-to-speech project, optimized for fast CPU-based speech s… | 61 | 1754 | active |
| webonnx/wonnx Wonnx is a GPU-accelerated ONNX inference runtime written entirely in Rust, built on wgpu and usable natively or in the browser via WebGPU … | 10 | 1754 | active |
| facebookresearch/metaseq Metaseq is a PyTorch codebase from Meta AI for training and working with large-scale Open Pre-trained Transformers (OPT), forked from fairs… | 10 | 6548 | maintenance |
| GoogleCloudPlatform/agent-starter-pack A Python package from Google Cloud Platform that provides production-ready templates for building and deploying GenAI agents on Google Clou… | 74 | 6545 | maintenance |
| Prism-Shadow/penguin-harness PenguinHarness is an open-source, local-first multi-agent application development platform that automates building, optimizing, and deployi… | 80 | 1751 | active |
| octo-models/octo Octo is an open-source generalist robot policy: a transformer-based diffusion policy pretrained on 800k robot trajectories from the Open X-… | 17 | 1751 | active |
| Stability-AI/StableCascade Official codebase for Stable Cascade, a text-to-image generation model built on the Würstchen architecture with a highly compressed latent … | 26 | 6540 | maintenance |
| ExtensityAI/symbolicai SymbolicAI is a Python neuro-symbolic framework that blends classical programming with LLM capabilities through composable Symbol primitive… | 94 | 1750 | active |
| wm94i/Work-Review Work Review is a local-first desktop application that automatically tracks which apps you used, websites you visited, window titles, usage … | 82 | 1748 | active |
| sybil-solutions/local-studio Local Studio is a local-first desktop application for launching, managing, and chatting with self-hosted LLM backends like vLLM, SGLang, ll… | 78 | 1741 | active |
| aiwithremy/claude-skills-llm-council A Claude Code skill that runs a user's question through five independent AI advisors who peer-review each other's answers, then synthesizes… | 50 | 1741 | active |
| magenta/magenta-realtime Magenta RealTime 2 (MRT2) is an open-weights model for real-time, interactive music generation controllable via text, MIDI, and audio. It s… | 73 | 1739 | active |
| 0xSero/turboquant TurboQuant is a Python library implementing near-optimal KV cache quantization for LLM inference, compressing keys to 3-bit and values to 2… | 48 | 1739 | active |
| tile-ai/TileRT TileRT is a tile-based runtime for ultra-low-latency LLM inference, achieving hundreds to over 1000 tokens/s decode speeds on frontier mode… | 77 | 1738 | active |
| Xiaojiu-z/EasyControl EasyControl is the official implementation of an ICCV 2025 paper adding efficient and flexible conditional control to Diffusion Transformer… | 35 | 1737 | active |
| jenslys/opencode-gemini-auth A TypeScript plugin for the Opencode CLI that authenticates with Google via OAuth so users can use their existing Gemini plan and quotas (i… | 77 | 1736 | active |
| TypeWhisper/typewhisper-mac TypeWhisper is a macOS application for local, on-device speech-to-text dictation built on WhisperKit, with optional cloud transcription via… | 78 | 1735 | active |
| tom-doerr/zsh_codex A ZSH plugin that brings AI-powered code completion to the command line, supporting OpenAI Codex, Google Gemini, and other LLM providers. U… | 38 | 1734 | active |
| trpc-group/trpc-agent-go tRPC-Agent-Go is a Go-native framework for building production LLM agent systems, offering agents, graph workflows (LangGraph-equivalent), … | 81 | 1733 | active |
| Zoo-Code-Org/Zoo-Code Zoo Code is an open-source AI coding assistant delivered as a VS Code extension (with JetBrains/Open VSX support), continuing development o… | 81 | 1731 | active |
| HKUDS/Auto-Deep-Research Auto-Deep-Research is an open-source, cost-efficient alternative to OpenAI's Deep Research, built on the AutoAgent framework. It runs as a … | 41 | 1731 | active |
| NVIDIA/FasterTransformer NVIDIA's highly optimized C++/CUDA library for fast inference of Transformer-based models such as BERT, GPT, and encoder-decoder models, wi… | 23 | 6447 | maintenance |
| jesselau76/ebook-GPT-translator A Python CLI and GUI tool that translates ebooks (TXT, EPUB, DOCX, PDF, optional MOBI) using LLM providers like OpenAI, Azure, Codex, Claud… | 63 | 1728 | active |
| freestylefly/director_ai An AI-powered mobile application for creating comic-style short dramas, generating scripts, storyboards, and composed videos from a single … | 52 | 1726 | active |
| 14790897/paper-ai A web application that helps users write academic papers with AI while searching real literature from Semantic Scholar, arXiv, and PubMed a… | 69 | 1720 | active |
| bespokelabsai/curator Bespoke Curator is a Python library for bulk LLM inference and scalable synthetic data curation for post-training and structured data extra… | 79 | 1719 | active |
| anthropics/claude-agent-sdk-typescript The Claude Agent SDK is a TypeScript/Node.js library for programmatically building AI agents powered by Claude Code's capabilities. It enab… | 80 | 1718 | active |
| kingoflolz/mesh-transformer-jax A JAX/Haiku library implementing model-parallel training and inference of transformer models using xmap/pjit operators, similar to Megatron… | 32 | 6380 | maintenance |
| deer-flow/llm-space LLM Space is a local-first desktop application for building, tracing, debugging, and evaluating LLM agents. It lets developers prototype ag… | 80 | 1712 | active |
| zubair-trabzada/ai-legal-claude A Claude Code skill pack providing an AI legal assistant with 14 slash commands and 5 parallel agents for contract review, risk analysis, N… | 48 | 1710 | active |
| fish2018/YPrompt YPrompt is a self-hosted web application that uses AI-guided conversation to elicit user requirements and automatically generate profession… | 44 | 1709 | active |
| LLPhant/LLPhant LLPhant is a comprehensive PHP generative AI framework inspired by LangChain and LlamaIndex, providing tools for working with LLMs, embeddi… | 92 | 1708 | active |
| nat/openplayground A self-hosted LLM playground that runs on your laptop, providing a web UI for prompting models from OpenAI, Anthropic, Cohere, HuggingFace,… | 60 | 6351 | maintenance |
| glidea/zenfeed zenfeed is a self-hosted AI-powered RSS reader and information hub built in Go. It aggregates RSS feeds, uses LLMs to filter, summarize, an… | 68 | 1707 | active |
| sopaco/deepwiki-rs Litho (deepwiki-rs) is a Rust-based AI-powered documentation generator that analyzes codebases and produces C4-model architecture documenta… | 76 | 1703 | active |
| software-mansion/react-native-executorch React Native ExecuTorch is a declarative React Native library for running AI models on-device, powered by Meta's ExecuTorch runtime. It shi… | 88 | 1702 | active |
| undreamai/LLMUnity LLMUnity is an open-source Unity plugin that runs large language models locally in Unity games via llama.cpp, enabling AI-driven characters… | 75 | 1701 | active |
| AgentDock/AgentDock AgentDock is an open-source, backend-first TypeScript framework for building AI agents with configurable determinism, plus a Next.js refere… | 61 | 1701 | active |
| AvaLovelace1/BrickGPT BrickGPT is the official implementation of an ICCV 2025 Best Paper approach that generates physically stable, buildable toy brick (LEGO) mo… | 57 | 1701 | active |
| xingyaoww/code-act CodeAct is a research framework and agent system that uses executable Python code as a unified action space for LLM agents, enabling multi-… | 26 | 1699 | active |
| youssofal/MTPLX MTPLX is a native macOS app and CLI that runs local LLMs on Apple Silicon roughly twice as fast using native multi-token prediction (MTP) s… | 81 | 1698 | active |
| OpenMOSS/MOSS-Transcribe-Diarize MOSS-Transcribe-Diarize 0.9B is an open-source end-to-end audio understanding model that jointly performs multi-speaker speech transcriptio… | 58 | 1695 | active |
| mithun50/openclaw-termux OpenClaw is an AI gateway that can be run on Android as a standalone Flutter app with a built-in terminal, web dashboard, and one-tap setup… | 75 | 1692 | active |
| leochlon/hallbayes Berry is a local MCP server that detects hallucinations by measuring whether an LLM's claims are actually supported by the cited evidence, … | 57 | 1692 | active |
| gnehs/subtitle-translator-electron A cross-platform Electron desktop app that translates subtitle files (.ass, .srt, .ssa, .vtt) using LLMs like ChatGPT. It preserves context… | 94 | 1690 | active |
| hrishioa/lumentis Lumentis is an AI-powered CLI tool that generates comprehensive, easy-to-skim documentation websites from transcripts, notes, and unstructu… | 28 | 1689 | active |
| adamlyttleapps/claude-skill-aso-appstore-screenshots A Claude Code skill that generates high-converting App Store screenshots for iOS apps by analyzing the codebase to identify core benefits a… | 48 | 1687 | active |
| cubewhy/skid-homework A browser-based, AI-powered homework solver built with Next.js that sends images and PDFs of homework problems to a Gemini or OpenAI-compat… | 61 | 1683 | active |
| trymirai/uzu Uzu is a high-performance inference engine written in Rust for running AI models directly on-device, with Python, TypeScript, and Swift bin… | 85 | 1678 | active |
| he-yufeng/CoreCoder CoreCoder is a minimal AI coding agent written in roughly 1,000 lines of Python, inspired by Claude Code and designed to work with any LLM.… | 78 | 1678 | active |
| MorDavid/BruteForceAI BruteForceAI is a Python-based penetration testing tool that uses LLMs (via Ollama or Groq) to automatically analyze login page HTML and id… | 60 | 1677 | active |
| nextify-limited/libra Libra AI is an open-source alternative to V0 and Lovable that generates production-grade full-stack web applications from natural language … | 37 | 1675 | active |
| LeapLabTHU/cooragent Cooragent is an AI agent collaboration community and framework where users can create powerful agents from a single sentence and have them … | 57 | 1674 | active |
| googleapis/js-genai The official Google Gen AI SDK for TypeScript and JavaScript, providing access to the Gemini Developer API and Vertex AI (Gemini Enterprise… | 84 | 1668 | active |
| deepsense-ai/ragbits Ragbits is a Python framework of modular building blocks for rapidly developing generative AI applications, covering LLM interaction, promp… | 75 | 1668 | active |
| intentee/paddler Paddler is an open-source LLM/VLM load balancer and serving platform built around the llama.cpp (ggml) ecosystem, packaged as a single Rust… | 87 | 1664 | active |
| elixir-nx/bumblebee Bumblebee is an Elixir library providing pre-trained neural network models built on Axon, with integration for downloading models from Hugg… | 89 | 1662 | active |
| lotus-data/lotus LOTUS is a Python library providing a Pandas-like API of LLM-powered semantic operators (map, filter, extract, aggregate, top-k) for bulk p… | 83 | 1661 | active |
| Dicklesworthstone/pi_agent_rust pi_agent_rust is a native, single-binary AI coding agent CLI written in Rust with zero unsafe code, ported from Mario Zechner's Pi Agent. I… | 82 | 1657 | active |
| Nutlope/turboseek TurboSeek is an open-source AI search engine web application inspired by Perplexity, built with Next.js and TypeScript. It searches the web… | 65 | 1655 | active |
| nolanx-ai/nolanx.ai NolanX is an open-source multi-modal agent platform for AI filmmaking that orchestrates text, image, audio, and video models into long-runn… | 52 | 1655 | active |
| LLM-Red-Team/kimi-cc A shell-based installer that configures Claude Code to run on Moonshot's Kimi K2 model instead of Anthropic's API. It lets developers use C… | 32 | 1654 | active |
| devenjarvis/lathe Lathe is a Go CLI plus local web UI that generates hands-on, multi-part technical tutorials on demand using LLM skills run inside coding ag… | 77 | 1653 | active |
| weijunext/smart-excel-ai SmartExcel is a web application that generates Excel formulas from natural-language descriptions using the ChatGPT API with streaming respo… | 40 | 1651 | active |
| LlamaEdge/LlamaEdge LlamaEdge is a lightweight Rust and WasmEdge-based runtime for running open-source LLMs locally or on edge devices, with CLI chat apps and … | 65 | 1650 | active |
| bytedance/pasa PaSa is an LLM-powered paper search agent from ByteDance that autonomously invokes search tools, reads papers, and selects relevant referen… | 31 | 1649 | active |
| steven-jianhao-li/zotero-AI-Butler A Zotero plugin that acts as an AI-powered research butler: it automatically scans PDFs added to your Zotero library and sends them to a us… | 79 | 1648 | active |
| google-deepmind/concordia Concordia is a Python library for building generative agent-based social simulations, using a tabletop role-playing game pattern where a Ga… | 84 | 1647 | active |
| vllm-project/vllm-metal A community-maintained hardware plugin that enables vLLM to run LLM inference on Apple Silicon Macs using MLX as the primary compute backen… | 79 | 1647 | active |
| Anemll/Anemll ANEMLL is an open-source Python library and toolchain for converting and running large language models on the Apple Neural Engine (ANE) via… | 61 | 1647 | active |
| AgentEra/Agently Agently is a Python framework for building production-grade GenAI applications with structured outputs, observable actions, MCP/tool integr… | 94 | 1644 | active |
| airockchip/rknn-llm RKLLM is Rockchip's software stack for converting, quantizing, and running large language models on Rockchip NPU chips such as RK3588 and R… | 79 | 1644 | active |
| FSoft-AI4Code/CodeWiki CodeWiki is an open-source Python framework that uses LLM-powered multi-agent processing to generate holistic, structured documentation for… | 60 | 1644 | active |
| AK391/ai-gradio A Python package built on Gradio that provides a unified interface for building AI apps with 15+ providers like OpenAI, Anthropic, and Gemi… | 28 | 1644 | active |
| huggingface/aisheets Hugging Face AI Sheets is an open-source no-code web application for building, enriching, and transforming datasets using AI models. It can… | 58 | 1642 | active |
| crazyykhllc-bit/CyberPPT CyberPPT is a Codex Skill that converts documents, research materials, and business data into high-density, editable, consulting-style Powe… | 55 | 1642 | active |
| pengxiao-song/LaWGPT LaWGPT is a series of open-source large language models tuned with Chinese legal knowledge, built on Chinese-LLaMA/ChatGLM base models with… | 30 | 6059 | maintenance |
| tenstorrent/tt-metal TT-Metal is Tenstorrent's open-source software stack containing TT-NN, a Python and C++ neural network operator library, and TT-Metalium, a… | 97 | 1640 | active |
| badlogic/lemmy A TypeScript monorepo providing a unified LLM interface library across Anthropic, OpenAI, and Google providers, plus a terminal UI framewor… | 35 | 1639 | active |
| SevaSk/ecoute Ecoute is a live transcription application that captures audio from both the user's microphone and speakers and displays real-time transcri… | 64 | 6047 | maintenance |
| Open-Curiosity/gini-agent Gini Agent is a personal AI agent runtime that remembers conversations, learns skills from task outcomes, and exposes chat, CLI, web, and m… | 74 | 1637 | active |
| kwaroran/Risuai RisuAI is a cross-platform AI roleplay and character chatting application built with Svelte, TypeScript, and Tauri. It acts as a frontend f… | 93 | 1634 | active |
| Gitlawb/zero Zero is a terminal-based AI coding agent written in Go that can inspect repositories, edit files, run commands, and manage durable local se… | 80 | 1631 | active |
| Prat011/free-cluely An open-source Electron desktop app that acts as an invisible AI assistant, providing real-time answers and insights during meetings, inter… | 53 | 1631 | active |
| ShengranHu/ADAS ADAS (Automated Design of Agentic Systems) is the official research code for the ICLR 2025 paper introducing Meta Agent Search, where a met… | 24 | 1631 | active |
| vigorX777/ai-daily-digest A zero-dependency TypeScript CLI tool (runnable as an OpenCode skill) that fetches the latest posts from 90 top tech blogs recommended by A… | 45 | 1630 | active |
| Jeffser/Alpaca Alpaca is a GTK4/GNOME desktop client for Ollama that lets users manage, download, and chat with multiple local AI models, with support for… | 84 | 1626 | active |
| langchain-ai/langmem LangMem is a Python library that helps AI agents learn and adapt from their interactions over time by extracting information from conversat… | 64 | 1626 | active |
| qiutongxue/oba-live-tool A desktop Electron application that automates live-streaming e-commerce tasks on Chinese platforms including Douyin, Xiaohongshu, Kuaishou,… | 81 | 1624 | active |
| MurrayHill1127/reopencode ReOpenCode is a Rust-based AI coding assistant providing a full agent loop with 18 callable tools, a terminal TUI (ratatui), and an HTTP RE… | 54 | 1624 | active |
| yincongcyincong/MuseBot MuseBot is a self-hosted Go chatbot application that connects messaging platforms (Telegram, Discord, Slack, Lark/Feishu, DingTalk, WeCom, … | 78 | 1623 | active |
| HumanSignal/Adala Adala is a Python framework for building autonomous data labeling agents powered by LLMs. Agents learn skills iteratively from a ground tru… | 62 | 1623 | active |
| a16z-infra/companion-app A tutorial starter stack for building and hosting AI companions with personality, backstory, and conversational memory. It combines Next.js… | 29 | 5977 | maintenance |
| mlcommons/inference Reference implementations of the MLPerf Inference benchmark suite, the industry-standard benchmarks for measuring how fast ML systems run m… | 83 | 1621 | active |
| nasa-jpl/rosa ROSA is a Python library and AI agent built on LangChain that lets developers interact with ROS1 and ROS2 robotics systems using natural la… | 58 | 1619 | active |
| nilsherzig/LLocalSearch LLocalSearch is a self-hosted search aggregator that uses locally running LLM agents to answer questions with web-sourced information, show… | 10 | 5947 | maintenance |
| NVIDIA/RULER RULER is a benchmark framework from NVIDIA that generates synthetic test examples to measure the true effective context size of long-contex… | 66 | 1609 | active |