# SqueezeAILab/LLMCompiler

[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling

Repository: https://github.com/SqueezeAILab/LLMCompiler
Canonical: https://ross.abutalabs.com/products/llmcompiler
Homepage: https://arxiv.org/abs/2312.04511
Language: Python
License: MIT
License Family: permissive
Topics: function-calling, llm, llm-agent, llm-agents, llms, parallel-function-call, efficient-inference, large-language-models, llama, llama2, llm-framework, natural-language-processing, nlp, transformer
Last push: 2024-07-10T04:39:34+00:00

## Health v2 (maintenance only)
Score: 26/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 35, longevity 71
- inputs: {"age_days": 1001, "days_push": 784, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1880, forks 136 (observed 2026-08-28T04:05:48.562087+00:00)

## What it is
LLMCompiler is a Python framework from an ICML 2024 paper that orchestrates parallel function calling with LLMs. It uses a planner, task fetching unit, and executor to automatically decompose tasks into parallelizable function calls, reducing latency and cost versus sequential approaches like ReAct.

## Use cases
- run multiple llm tool calls in parallel to reduce latency
- orchestrate function calling with dependencies between tasks
- cut token costs when an agent makes many function calls
- benchmark parallel function calling against ReAct
- use open-source LLaMA models with parallel tool calling
- serve custom models with vLLM for function calling

## When to choose
- your LLM agent makes many independent tool calls sequentially and latency matters
- you want lower cost and higher accuracy than ReAct-style agents
- you need to support both OpenAI and open-source models like LLaMA

## When to avoid
- your agent only makes one or two function calls at a time
- you need a production-hardened agent framework with broad integrations
- your tasks are strictly sequential with no parallelizable steps

## Facets
- artifact type: framework
- maturity: active
- function: agent-framework, llm-inference, workflow-automation, sdk
- domain: large-language-models, machine-learning
- platform: python, cross-platform
- tags: function-calling, parallel-execution, llm-orchestration, compiler-inspired, research-paper, benchmarking, vllm, openai, llama, ai-agents, natural-language-processing

## Member repositories
- SqueezeAILab/LLMCompiler (main) score 26

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:48.562087+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:14:00.102189+00:00, confidence not recorded.
  - readme: https://github.com/SqueezeAILab/LLMCompiler (fetched 2026-08-28T04:05:48.562087+00:00, sha 04a25fd0059f)
  - homepage: https://arxiv.org/abs/2312.04511 (fetched 2026-08-29T10:53:16.383191+00:00, sha fc097cc4e340)
  - site_page: https://info.arxiv.org/about/donate.html (fetched 2026-08-29T10:53:16.393385+00:00, sha cca9c3a11c56)
  - site_page: https://info.arxiv.org/about/ourmembers.html (fetched 2026-08-29T10:53:16.397529+00:00, sha 47cbc55ff1de)
  - site_page: https://info.arxiv.org/about (fetched 2026-08-29T10:53:16.399523+00:00, sha a1f16f915a9a)
  - site_page: https://info.arxiv.org/labs/index.html (fetched 2026-08-29T10:53:16.395745+00:00, sha b14a8d05a0ec)
- Data as of 2026-08-30T08:39:29.467469+00:00.