# ray-project/ray-llm

RayLLM - LLMs on Ray (Archived). Read README for more info.

Repository: https://github.com/ray-project/ray-llm
Canonical: https://ross.abutalabs.com/products/ray-llm
Homepage: https://docs.ray.io/en/latest/
License Family: other
Topics: ray, llm, llm-serving
Archived: true
Last push: 2025-03-13T01:13:38+00:00

## Health v2 (maintenance only)
Score: 10/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 11, release rhythm 8, longevity 85
- inputs: {"age_days": 1190, "days_push": 539, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: archived, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1261, forks 90 (observed 2026-08-28T04:04:10.207657+00:00)

## What it is
RayLLM was a library for serving and deploying large language models on top of Ray, providing LLM APIs built on Ray Serve. It has been archived and its functionality was upstreamed into the main Ray repository as ray.serve.llm and ray.data.llm.

## Use cases
- serve LLMs on a Ray cluster
- deploy open-source language models as APIs
- scale LLM inference with Ray Serve
- run batch LLM inference pipelines
- host multiple LLM backends behind one endpoint

## When to choose
- you want the modern equivalent — use ray.serve.llm in the main Ray repo instead
- you need historical reference for the original RayLLM design

## When to avoid
- starting a new project — the repo is archived and unmaintained
- you need bug fixes or support
- you are not already invested in the Ray ecosystem

## Facets
- artifact type: library
- maturity: abandoned
- function: llm-inference, deployment, api-framework
- domain: large-language-models, machine-learning, developer-tools
- platform: python, cloud
- tags: ray, llm-serving, archived, ray-serve, model-deployment, docker

## Member repositories
- ray-project/ray-llm (main) score 10

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:04:10.207657+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T05:04:20.240536+00:00, confidence not recorded.
  - readme: https://github.com/ray-project/ray-llm (fetched 2026-08-28T04:04:10.207657+00:00, sha d1ea6f8624a1)
- Data as of 2026-08-30T08:39:29.467469+00:00.