fluid-cloudnative/fluid

Health v2 · maintenance only

Adoption not part of the score

1965 stars · 1273 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Fluid is a Kubernetes-native distributed dataset orchestrator and accelerator for data-intensive applications such as big data and AI workloads. It provides dataset abstraction, elastic distributed caching (e.g., via Alluxio), and data-aware scheduling, and is a CNCF incubating project.

Use cases

• accelerate AI training jobs reading large datasets on Kubernetes
• cache big data datasets close to compute pods
• manage heterogeneous data sources behind a unified Dataset abstraction
• warm up datasets before running Spark or deep learning jobs
• schedule pods to nodes where their data is cached
• observe and scale distributed cache runtimes in the cloud

When to choose

• you run data-intensive big data or AI workloads on Kubernetes and need faster data access
• you want a Kubernetes-native way to orchestrate and cache datasets across heterogeneous storage
• you need data-aware scheduling to co-locate compute with cached data

When to avoid

• your workloads are small and storage access latency is not a bottleneck
• you are not running on Kubernetes
• you need a general-purpose object storage system rather than a caching/orchestration layer

Facets

framework · maturity active

caching workflow-automation data-science scheduling monitoring big-data machine-learning cloud-computing microservices go cloud self-hosted dataset-orchestration distributed-cache alluxio data-acceleration cncf data-aware-scheduling cloud-native containers data-engineering kubernetes docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="fluid-cloudnative/fluid")