SqueezeAILab/LLMCompiler

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1880 stars · 136 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

LLMCompiler is a Python framework from an ICML 2024 paper that orchestrates parallel function calling with LLMs. It uses a planner, task fetching unit, and executor to automatically decompose tasks into parallelizable function calls, reducing latency and cost versus sequential approaches like ReAct.

Use cases

• run multiple llm tool calls in parallel to reduce latency
• orchestrate function calling with dependencies between tasks
• cut token costs when an agent makes many function calls
• benchmark parallel function calling against ReAct
• use open-source LLaMA models with parallel tool calling
• serve custom models with vLLM for function calling

When to choose

• your LLM agent makes many independent tool calls sequentially and latency matters
• you want lower cost and higher accuracy than ReAct-style agents
• you need to support both OpenAI and open-source models like LLaMA

When to avoid

• your agent only makes one or two function calls at a time
• you need a production-hardened agent framework with broad integrations
• your tasks are strictly sequential with no parallelizable steps

Facets

framework · maturity active

agent-framework llm-inference workflow-automation sdk large-language-models machine-learning python cross-platform function-calling parallel-execution llm-orchestration compiler-inspired research-paper benchmarking vllm openai llama ai-agents natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="SqueezeAILab/LLMCompiler")