# Alluxio/alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud

Repository: https://github.com/Alluxio/alluxio
Canonical: https://ross.abutalabs.com/products/alluxio
Homepage: https://www.alluxio.io
Language: Java
License: Apache-2.0
License Family: permissive
Topics: alluxio, memory-speed, hadoop, spark, presto, tensorflow, data-analysis, data-orchestration, virtual-distributed-filesystem
Last push: 2025-04-29T16:46:58+00:00

## Health v2 (maintenance only)
Score: 31/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 19, release rhythm 8, longevity 100
- inputs: {"age_days": 5003, "days_push": 491, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 7231, forks 2933 (observed 2026-08-28T04:09:57.659621+00:00)

## What it is
Alluxio is an open-source distributed caching and data orchestration platform that sits between compute frameworks (Spark, Presto, Trino, PyTorch) and storage systems (S3, GCS, Azure Blob, HDFS). It caches hot data on local NVMe/SSD of compute nodes, exposing it via POSIX, S3-compatible API, or FSSpec so workloads read at local storage speed instead of repeatedly crossing the network to object storage.

## Use cases
- speed up Spark and Presto queries against S3 or HDFS
- cache training datasets on GPU nodes to accelerate PyTorch or TensorFlow epochs
- eliminate model-loading cold starts in inference serving by caching model weights
- accelerate checkpoint writes to cloud object storage with write-back caching
- unify access to multiple storage systems behind a single virtual file system
- reduce cloud egress and storage costs by caching data near compute
- support multi-cloud and hybrid data access for analytics clusters

## When to choose
- your analytics or ML workloads repeatedly read the same data from slow remote object storage
- you run data-intensive engines like Spark, Presto, or Trino against S3, GCS, or HDFS and want memory/SSD-speed access
- you need a virtual distributed file system layer unifying multiple underlying storage systems
- you want to reduce S3 request costs and egress traffic in large clusters

## When to avoid
- your datasets are small and fit entirely in local storage or your database
- you cannot operate an additional distributed Java service cluster
- you need the enterprise features (tens of billions of files, FUSE POSIX for AI) that are only in the paid edition
- your workload is write-heavy with strict consistency requirements against object storage

## Facets
- artifact type: service
- maturity: stable
- function: caching, file-system, object-storage, data-science, etl
- domain: big-data, machine-learning, microservices, cloud-computing
- platform: cloud, jvm
- tags: distributed-caching, data-orchestration, virtual-file-system, spark, presto, trino, s3, hdfs, gpu-acceleration, analytics-acceleration, data-engineering, linux, docker, kubernetes

## Member repositories
- Alluxio/alluxio (main) score 31

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:09:57.659621+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:39:46.784424+00:00, confidence not recorded.
  - readme: https://github.com/Alluxio/alluxio (fetched 2026-08-28T04:09:57.659621+00:00, sha c65b5b9bfbe6)
  - homepage: https://www.alluxio.io (fetched 2026-08-29T08:34:38.396685+00:00, sha 822bd89e6986)
  - site_page: https://www.alluxio.io/alluxio-ai/alluxio-ai-features (fetched 2026-08-29T08:34:38.405906+00:00, sha 4b46bea51612)
  - site_page: http://documentation.alluxio.io/ (fetched 2026-08-29T08:34:38.408886+00:00, sha aa8d353ef1fa)
  - site_page: https://www.alluxio.io/about (fetched 2026-08-29T08:34:38.411297+00:00, sha 450275747934)
  - site_page: https://www.alluxio.io/pricing (fetched 2026-08-29T08:34:38.413139+00:00, sha fbe2a4c3c009)
- Data as of 2026-08-30T08:39:29.467469+00:00.