openlake-project/openlake

Health v2 · maintenance only

Flags: young

Adoption not part of the score

2330 stars · 413 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

OpenLake is a high-performance distributed storage engine written in Rust (built on io_uring, RDMA, and GPUDirect) designed to feed GPUs during LLM inference and training. It provides petabyte-scale KV cache offload, fast checkpointing, vector serving, and S3-compatible object storage with sub-millisecond, million-IOPS performance.

Use cases

• offload LLM KV cache from GPU memory to persistent storage
• speed up model checkpointing for RL and ML training runs
• reduce GPU idle time during training with fast small-file I/O
• store and reuse KV cache across vLLM or SGLang inference requests
• build a fast vector database for retrieval workloads
• store massive conversation context and memory for AI agents
• replace S3 bottlenecks in GPU data pipelines

When to choose

• you run LLM inference at scale and KV cache recompute is hurting throughput
• training jobs are bottlenecked on checkpoint or small-file storage I/O
• you need GPU-direct data paths (RDMA/GPUDirect) to saturate accelerators
• you want S3-compatible APIs with far higher throughput on GPU hosts

When to avoid

• you need a general-purpose cloud object store with multi-region replication
• your workloads are CPU-bound with no GPU acceleration
• you need a turnkey managed service rather than self-hosted infrastructure
• you run primarily on Windows or macOS, since it targets Linux with io_uring

Facets

library · maturity active

caching object-storage serialization gpu-computing llm-inference llm-training rag large-language-models databases gpu-computing infrastructure-as-code microservices machine-learning artificial-intelligence rust self-hosted storage-engine kv-cache-offload io-uring rdma gpudirect checkpointing vector-database model-serving high-performance-storage linux gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="openlake-project/openlake")