lakesoul-io/LakeSoul

Health v2 · maintenance only

Adoption not part of the score

3247 stars · 423 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

LakeSoul is a cloud-native, real-time lakehouse framework with a Rust-native core providing ACID table format, concurrent upserts, incremental reads, and vector search. It integrates with Spark, Flink, Presto, Ray, Daft, and DuckDB, with PostgreSQL-based metadata management and built-in compaction and RBAC.

Use cases

• build a real-time lakehouse with streaming ingestion from Kafka and Flink CDC
• run concurrent upserts and incremental reads on data lake tables
• query lakehouse tables with SQL from Spark, Flink, Presto, or DuckDB
• prepare tabular training datasets for AI and PyTorch workloads
• perform vector search over lakehouse data
• unify batch and stream processing on one table format
• avoid stitching together separate catalogs, compaction services, and auth layers

When to choose

• you need a batteries-included lakehouse platform rather than just a table format
• you require high-concurrency writes with ACID guarantees and auto conflict resolution
• you want one Rust-native core shared consistently across Java, Python, and C++ engines
• you need real-time incremental pipelines on Hadoop or Kubernetes clusters
• you want built-in compaction, RBAC, and vector retrieval out of the box

When to avoid

• you only need a widely adopted table format with the broadest ecosystem support, such as Apache Iceberg
• your stack relies on engines not in LakeSoul's compatibility matrix
• you prefer a fully serverless managed warehouse over self-managed lakehouse infrastructure
• your workloads are small-scale and don't need lakehouse complexity

Facets

framework · maturity active

database streaming etl vector-database search-engine serialization data-science big-data databases analytics machine-learning jvm python rust cloud self-hosted lakehouse table-format spark flink apache-arrow datafusion upsert acid-transactions cdc incremental-processing olap ray daft rust-core vector-search data-engineering real-time docker kubernetes linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lakesoul-io/LakeSoul")