ray-project/ray-llm

Health v2 · maintenance only

Flags: archived no_license

Adoption not part of the score

1261 stars · 90 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

RayLLM was a library for serving and deploying large language models on top of Ray, providing LLM APIs built on Ray Serve. It has been archived and its functionality was upstreamed into the main Ray repository as ray.serve.llm and ray.data.llm.

Use cases

• serve LLMs on a Ray cluster
• deploy open-source language models as APIs
• scale LLM inference with Ray Serve
• run batch LLM inference pipelines
• host multiple LLM backends behind one endpoint

When to choose

• you want the modern equivalent — use ray.serve.llm in the main Ray repo instead
• you need historical reference for the original RayLLM design

When to avoid

• starting a new project — the repo is archived and unmaintained
• you need bug fixes or support
• you are not already invested in the Ray ecosystem

Facets

library · maturity abandoned

llm-inference deployment api-framework large-language-models machine-learning developer-tools python cloud ray llm-serving archived ray-serve model-deployment docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ray-project/ray-llm")