function: llm-inference
3362 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| microsoft/genaiscript GenAIScript is a JavaScript/TypeScript framework for programmatically assembling and orchestrating LLM prompts, tools, and data in code. It… | 10 | 2892 | maintenance |
| stanford-crfm/helm HELM is a Python framework from Stanford's CRFM for holistic, reproducible evaluation of foundation models including LLMs and multimodal mo… | 87 | 2887 | maintenance |
| shreyashankar/gpt3-sandbox A Python library that wraps the OpenAI GPT-3 API to make it easy to prime the model with few-shot examples and launch a local web demo with… | 32 | 2868 | maintenance |
| OpenPipe/OpenPipe OpenPipe is an open-source fine-tuning and model-hosting platform that turns expensive LLM prompts into cheaper fine-tuned models. It offer… | 29 | 2830 | maintenance |
| yanqiangmiffy/Chinese-LangChain A Chinese-language local knowledge base question-answering application built on ChatGLM-6B and LangChain, with a Gradio web UI. It supports… | 30 | 2824 | maintenance |
| microsoftarchive/promptbench PromptBench is a unified Python library from Microsoft for evaluating and understanding large language models across many datasets, models,… | 10 | 2819 | maintenance |
| Alpha-VLLM/LLaMA2-Accessory LLaMA2-Accessory is an open-source Python toolkit for pretraining, finetuning, and deploying large language models and multimodal LLMs, inc… | 29 | 2800 | maintenance |
| henrywoo/pyllama A Python library that is a modified version of Meta's LLaMA implementation, making it easy to download model weights and run LLaMA models o… | 21 | 2778 | maintenance |
| OpenBMB/BMTools BMTools is an open-source Python library that extends large language models with external tools, letting developers build plugins by writin… | 30 | 2771 | maintenance |
| microsoft/prompt-engine An NPM utility library from Microsoft for building and maintaining prompts for Large Language Models, with generic, code, and chat prompt e… | 10 | 2752 | maintenance |
| paperswithcode/galai galai is the official Python library for loading and running inference with the GALACTICA general-purpose scientific language models from P… | 23 | 2741 | maintenance |
| apple/ml-ane-transformers A reference PyTorch implementation of the Transformer architecture optimized for Apple's Neural Engine (ANE), including optimized Hugging F… | 23 | 2736 | maintenance |
| ohmplatform/FreedomGPT FreedomGPT is a React and Electron desktop application that runs LLM models locally on Mac, Windows, and Linux using llama.cpp, providing a… | 21 | 2670 | maintenance |
| microsoft/DirectML DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning that provides GPU acceleration across DirectX … | 58 | 2580 | maintenance |
| wenge-research/YAYI YAYI is a family of open-source Chinese-optimized large language models (7B/13B) instruction-tuned from LLaMA 2 and BLOOM base models by We… | 10 | 2527 | maintenance |
| IntelligenzaArtificiale/Free-Auto-GPT A simplified open-source reimplementation of Auto GPT that runs autonomous AI agents without any paid APIs, using free access to models via… | 30 | 2526 | maintenance |
| young-geng/EasyLM EasyLM is a JAX/Flax-based framework for pre-training, finetuning, evaluating, and serving large language models like LLaMA. It scales trai… | 32 | 2514 | maintenance |
| Anning01/AIMedia AIMedia is a heavyweight integrated application that automatically scrapes trending news, generates AI-written articles with AI-generated i… | 63 | 2436 | maintenance |
| Stability-AI/stability-sdk A Python SDK and command-line client for interacting with Stability AI's gRPC-based APIs, primarily for Stable Diffusion image generation a… | 60 | 2436 | maintenance |
| zai-org/CogVLM2 CogVLM2 is an open-source multi-modal vision-language model family built on Meta-Llama-3-8B-Instruct, offering image and video understandin… | 28 | 2433 | maintenance |
| AkariAsai/self-rag Original implementation of Self-RAG, a framework that trains a language model to adaptively retrieve passages on demand and critique its ow… | 27 | 2421 | maintenance |
| microsoft/Codex-CLI A Python CLI tool that uses OpenAI's GPT-3 Codex model to translate natural language commands into Bash, Zsh, or PowerShell commands. Users… | 32 | 2413 | maintenance |
| OpenBMB/CPM-Bee CPM-Bee is a fully open-source, commercially usable 10-billion-parameter bilingual (Chinese-English) foundation large language model traine… | 71 | 2406 | maintenance |
| SoraWebui/SoraWebui SoraWebui is an open-source web client for generating videos from text prompts using OpenAI's Sora model, built with TypeScript and deploya… | 16 | 2404 | maintenance |
| IST-DASLab/gptq Reference implementation of GPTQ, a one-shot post-training weight quantization method for large generative transformer models based on appr… | 32 | 2360 | maintenance |
| glowingjade/obsidian-smart-composer Smart Composer is an Obsidian plugin that adds an AI chat assistant with contextual awareness of your vault, smart writing assistance, and … | 64 | 2320 | maintenance |
| TigerResearch/TigerBot TigerBot is a multi-language, multi-task large language model project from TigerResearch, providing pretrained and chat-tuned model weights… | 29 | 2259 | maintenance |
| Turing-Project/AntiFraudChatBot A prompt-based Chinese conversational AI framework that connects a fine-tuned large language model (YUAN-1.0) to WeChat via the wechaty fra… | 22 | 2220 | maintenance |
| zuoqing1988/ZQCNN ZQCNN is a lightweight deep learning inference framework written in C/C++ that runs on Windows, Linux, and ARM-Linux. It ships with demos f… | 61 | 2214 | maintenance |
| hwchase17/notion-qa A Python application that lets you ask natural-language questions about a Notion database, built on LangChain and OpenAI. It ingests export… | 32 | 2154 | maintenance |
| facebookresearch/chameleon Repository for Meta Chameleon, an early-fusion token-based mixed-modal foundation model that understands and generates interleaved images a… | 10 | 2103 | maintenance |
| camenduru/text-generation-webui-colab A collection of Google Colab notebooks that launch the oobabooga text-generation-webui with a Gradio interface for running large language m… | 30 | 2091 | maintenance |
| ai-forever/ru-gpts A repository of Russian GPT-3 language models (ruGPT3XL/Large/Medium/Small and ruGPT2Large) with usage and fine-tuning examples. It provide… | 32 | 2087 | maintenance |
| sonnylazuardi/chat-ai-desktop An unofficial open-source ChatGPT desktop app for Mac, Windows, and Linux that lives in the menubar, built with Tauri and Rust. It wraps th… | 22 | 1997 | maintenance |
| deepseek-ai/DeepSeek-MoE DeepSeekMoE 16B is a Mixture-of-Experts language model with fine-grained expert segmentation and shared expert isolation, released with bas… | 26 | 1969 | maintenance |
| cloudflare/moltworker A Cloudflare Workers project that runs the OpenClaw (formerly Moltbot/Clawdbot) self-hosted personal AI assistant on Cloudflare's Sandbox c… | 52 | 9957 | experimental |
| QwenLM/Qwen-Audio Official repository for Qwen-Audio, Alibaba Cloud's large audio-language model with pretrained and chat variants. It provides model weights… | 27 | 1945 | maintenance |
| liltom-eth/llama2-webui A Gradio-based web UI for running Llama 2 models (7B/13B/70B, GPTQ, GGML, GGUF, CodeLlama) locally on GPU or CPU across Linux, Windows, and… | 19 | 1937 | maintenance |
| nat/natbot natbot is a Python tool that drives a web browser using GPT-3, letting an LLM agent navigate and interact with web pages. It serializes the… | 32 | 1933 | maintenance |
| uTensor/uTensor uTensor is an extremely lightweight machine learning inference framework built on TensorFlow and optimized for Arm Cortex-M microcontroller… | 32 | 1928 | maintenance |
| S-LoRA/S-LoRA S-LoRA is a serving system for large language models that scales to thousands of concurrent LoRA adapters on one or more GPUs. It uses Unif… | 10 | 1923 | maintenance |
| OkGoDoIt/OpenAI-API-dotnet An unofficial C#/.NET SDK wrapping the OpenAI API, covering chat completions (GPT-3.5/4), DALL-E image generation, embeddings, moderation, … | 23 | 1893 | maintenance |
| Akegarasu/ChatGLM-webui A Gradio-based web UI for running the THUDM ChatGLM-6B large language model locally. It provides a chat interface with adjustable parameter… | 30 | 1889 | maintenance |
| marella/ctransformers CTransformers provides Python bindings for transformer language models implemented in C/C++ using the GGML library. It offers a unified int… | 20 | 1885 | maintenance |
| Forethought-Technologies/AutoChain AutoChain is a lightweight, extensible Python framework for building custom LLM-powered generative agents with custom tools, inspired by La… | 55 | 1879 | maintenance |
| salesforce/ctrl CTRL is a 1.63 billion-parameter conditional Transformer language model from Salesforce that generates text controllable via control codes … | 10 | 1879 | maintenance |
| clue-ai/ChatYuan ChatYuan is an open-source bilingual (Chinese and English) dialogue large language model in ChatGPT style, with a lightweight variant runna… | 31 | 1863 | maintenance |
| Helixform/CodeCursor CodeCursor is a Visual Studio Code extension that brings the Cursor AI code editor experience into VS Code, offering AI code generation, co… | 10 | 1828 | maintenance |
| openai/gpt-discord-bot An example Discord bot written in Python that uses the OpenAI chat completions API to hold conversations with the gpt-3.5-turbo model and t… | 31 | 1825 | maintenance |
| xmartlabs/Bender Bender is a Swift library that provides an abstraction layer over Apple's MetalPerformanceShaders for defining and running neural networks … | 23 | 1802 | maintenance |
| hieunc229/copilot-clone Captain Stack is a VSCode extension that provides code suggestions similar to GitHub Copilot. Instead of relying solely on AI, it searches … | 32 | 1782 | maintenance |
| Fictiverse/Redream A Windows desktop application that performs near-realtime Stable Diffusion img2img generation from a captured screen area, using the Automa… | 22 | 1778 | maintenance |
| ttengwang/Caption-Anything Caption-Anything combines Segment Anything image segmentation, visual captioning, and ChatGPT to generate tailored captions for any object … | 30 | 1777 | maintenance |
| Synaptrix/ChatGPT-Desktop A cross-platform desktop ChatGPT client built with Tauri, Vue 3, and Arco Design that wraps the OpenAI GPT-3.5-turbo API. It offers a fast … | 10 | 1773 | maintenance |
| flowtyone/flowty-realtime-lcm-canvas A real-time sketch-to-image demo application that uses Latent Consistency Models (LCM) with a Gradio web UI. Users draw on a canvas and see… | 27 | 1771 | maintenance |
| nashsu/FreeAskInternet FreeAskInternet is a self-hosted, Perplexity-like search aggregator and answer generator that runs multi-engine searches via SearXNG and fe… | 25 | 8746 | experimental |
| OpenCodeInterpreter/OpenCodeInterpreter OpenCodeInterpreter is a suite of open-source code generation models and deployment code that integrates code execution and iterative refin… | 25 | 1757 | maintenance |
| microsoft/TypeChat TypeChat is a TypeScript library from Microsoft that replaces prompt engineering with 'schema engineering' for building natural language in… | 73 | 8680 | experimental |
| SecureAI-Tools/SecureAI-Tools A self-hosted web application for private AI productivity, offering chat with AI models and chat-with-documents (PDF) backed by local infer… | 17 | 1726 | maintenance |
| 101dotxyz/GPTeam GPTeam is an open-source multi-agent simulation that uses GPT-4 to create multiple AI agents with memory who collaborate and communicate to… | 75 | 1723 | maintenance |
| cvlab-columbia/viper ViperGPT is a research codebase that composes vision-and-language models with code generated by large language models (GPT-3.5/GPT-4) to pe… | 30 | 1716 | maintenance |
| Maratyszcza/NNPACK NNPACK is a C99 acceleration package providing high-performance SIMD and multi-core CPU implementations of neural network layers, especiall… | 32 | 1710 | maintenance |
| techleadhd/chatgpt-retrieval A simple Python script that lets you ask ChatGPT questions about your own files using LangChain and a Chroma vector store. It loads text or… | 29 | 1701 | maintenance |
| ELS-RD/transformer-deploy A Python tool that optimizes Hugging Face transformer models for production inference using ONNX Runtime and NVIDIA TensorRT, deploying the… | 23 | 1689 | maintenance |
| TheExplainthis/ChatGPT-Line-Bot A Python application that integrates ChatGPT into the LINE messaging platform, letting users chat with GPT models and generate DALL·E 2 ima… | 21 | 1689 | maintenance |
| zlwaterfield/scramble Scramble is an open-source browser extension that uses LLMs to enhance writing directly on any webpage, acting as a customizable Grammarly … | 35 | 1686 | maintenance |
| eyurtsev/kor Kor is a Python library for extracting structured data from text using large language models. Users define a schema (Kor-style or Pydantic)… | 23 | 1685 | maintenance |
| maojindao55/botgroup.chat A multi-user AI group chat application built with React and deployable to Cloudflare Pages, where multiple AI characters with distinct pers… | 72 | 1678 | maintenance |
| mit-han-lab/smoothquant SmoothQuant is a training-free post-training quantization (PTQ) method that enables W8A8 (8-bit weight, 8-bit activation) quantization for … | 32 | 1677 | maintenance |
| lxfater/Portal Portal is a cross-platform desktop tool that integrates ChatGPT capabilities into any application via a global keyboard shortcut. It sends … | 10 | 1674 | maintenance |
| hamelsmu/evals-skills A collection of skills (prompts/instructions) that guide AI coding agents like Claude Code through building and auditing LLM evaluation pip… | 10 | 1664 | maintenance |
| pydantic/monty Monty is a minimal, secure Python interpreter written in Rust designed for safely executing LLM-generated code without container-based sand… | 95 | 8113 | experimental |
| langchain-ai/langgraph-supervisor-py A Python library for building hierarchical multi-agent systems on top of LangGraph, where a central supervisor agent orchestrates and deleg… | 70 | 1646 | maintenance |
| altryne/chatGPT-telegram-bot A Telegram bot that lets users chat with ChatGPT, originally by driving the ChatGPT web interface in Chromium via Playwright, with plans to… | 22 | 1641 | maintenance |
| mmz-001/knowledge_gpt KnowledgeGPT is a Streamlit web application that lets users upload documents (primarily PDFs) and ask questions about them, receiving answe… | 10 | 1635 | maintenance |
| johnson7788/MultiAgentPPT MultiAgentPPT is a Python-based multi-agent system that generates editable presentation slides from a topic using A2A, MCP, and Google ADK … | 60 | 1633 | maintenance |
| WisdomShell/codeshell CodeShell is a family of 7-billion-parameter code large language models developed by PKU-KCL, trained on 500 billion tokens with an 8192-to… | 27 | 1621 | maintenance |
| wangzhaode/mnn-llm mnn-llm is a C++ library for deploying large language models on-device using Alibaba's MNN inference engine, with demos for CLI, web, Andro… | 22 | 1616 | maintenance |
| divamgupta/stable-diffusion-tensorflow A TensorFlow/Keras implementation of Stable Diffusion with weights ported from the original implementation. It provides a Python API and Co… | 32 | 1608 | maintenance |
| sobelio/llm-chain llm-chain is a Rust crate for building chains of large language model calls to complete complex tasks such as text summarization. It provid… | 21 | 1604 | maintenance |
| RomanHotsiy/commitgpt commitgpt is a Node.js CLI tool that automatically generates commit messages using ChatGPT. It runs `git diff --cached`, sends the diff to … | 32 | 1593 | maintenance |
| finic-ai/rag-stack RAGstack is a self-hostable private ChatGPT alternative that deploys open-source LLMs (Llama 2, Falcon, GPT4All) alongside the Qdrant vecto… | 29 | 1586 | maintenance |
| jina-ai/thinkgpt ThinkGPT is a Python library implementing Chain of Thought techniques for LLMs, providing memory, self-refinement, knowledge compression, a… | 30 | 1583 | maintenance |
| TsinghuaAI/CPM-1-Generate CPM-Generate provides inference code for CPM-LM, a 2.6B-parameter Chinese pre-trained language model based on the GPT-2 architecture and Me… | 32 | 1578 | maintenance |
| Frrrrrrrrank/auto_job__find__chatgpt__rpa A free Python RPA script that uses OpenAI's API to automatically generate cover letters from your resume and job descriptions, then sends m… | 26 | 1574 | maintenance |
| microsoft/TinyTroupe TinyTroupe is an experimental Python library for simulating people with specific personalities, interests, and goals using LLM-powered agen… | 72 | 7556 | experimental |
| pytorch/QNNPACK QNNPACK is a mobile-optimized C library of high-performance kernels for 8-bit quantized neural network operators such as convolution, pooli… | 10 | 1551 | maintenance |
| GoogleCloudPlatform/localllm A CLI tool and Docker image for running quantized LLMs locally on Google Cloud Workstations using llama-cpp-python's web server. It package… | 10 | 1550 | maintenance |
| Tencent/TurboTransformers A fast, user-friendly C++/Python runtime for transformer model inference (BERT, ALBERT, RoBERTa, GPT2, decoders) on CPU and GPU. It works a… | 37 | 1548 | maintenance |
| bes-dev/stable_diffusion.openvino A Python CLI implementation of Stable Diffusion text-to-image generation optimized for Intel CPUs and GPUs via OpenVINO. It supports text-t… | 32 | 1533 | maintenance |
| Stability-AI/sd3.5 A minimal inference-only reference implementation of Stable Diffusion 3.5 and SD3, including text encoders, VAE decoder, MM-DiT core, and S… | 22 | 1529 | maintenance |
| Tameyer41/liftoff Liftoff is a web application that simulates technical mock interviews and provides AI-powered feedback using OpenAI Whisper for speech tran… | 29 | 1523 | maintenance |
| run-llama/llama-lab Llama Lab is a collection of experimental projects built on top of LlamaIndex for exploring cutting-edge LLM agent capabilities. It include… | 30 | 1512 | maintenance |
| alexrozanski/LlamaChat LlamaChat is a native macOS app for chatting with LLaMA, Alpaca, GPT4All and similar models running entirely locally on your Mac. It is bui… | 60 | 1511 | maintenance |
| OWD-AI/AutoAgents AutoAgents is a Python framework that uses an LLM to automatically generate multiple specialized agent roles that collaborate to accomplish… | 46 | 1493 | maintenance |
| rotemweiss57/gpt-newspaper GPT Newspaper is a Python application built on LangGraph that runs a multi-agent pipeline to search, curate, write, critique, design, and p… | 26 | 1469 | maintenance |
| CStanKonrad/long_llama LongLLaMA is a large language model handling long contexts of 256k tokens or more, built on OpenLLaMA and fine-tuned with the Focused Trans… | 29 | 1465 | maintenance |
| joshpxyne/gpt-migrate GPT-Migrate is a Python CLI tool that uses LLMs (GPT-4 via OpenAI or OpenRouter) to automatically migrate codebases from one framework or l… | 29 | 6978 | experimental |
| instructlab/instructlab InstructLab (ilab) is an open-source CLI and Python package for enhancing large language models through community-contributed taxonomy-base… | 10 | 1419 | maintenance |
| mckaywrigley/clarity-ai Clarity AI is an open-source clone of Perplexity.ai that answers user queries by scraping Google search results, parsing the webpages, and … | 31 | 1418 | maintenance |