lance-format/lance

Health v2 · maintenance only

Adoption not part of the score

6973 stars · 817 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Lance is an open lakehouse data format (file format, table format, and catalog spec) for multimodal AI, implemented in Rust with Python and Java SDKs. It offers 100x faster random access than Parquet, vector and full-text search, zero-copy versioning with ACID transactions, and integrations with Pandas, DuckDB, Polars, PyArrow, Spark, Ray, and PyTorch.

Use cases

• convert parquet datasets to a faster format for ML training
• store and query image, video, audio, and embedding data
• run vector similarity search over large embedding collections
• build a feature store with hybrid vector and full-text search
• version ML training datasets with time travel and branches
• serve random-access row lookups for ML inference
• add columns to large tables without full rewrites

When to choose

• you need fast random access to individual rows in large columnar datasets
• you are training ML models on multimodal data stored in object storage
• you want vector search, full-text search, and SQL analytics over one dataset
• you need dataset versioning and ACID transactions without extra infrastructure

When to avoid

• you only need simple analytical scans where Parquet is sufficient
• you need a mature, battle-tested lakehouse format like Iceberg or Delta Lake with broad vendor support
• your workload is small-scale and doesn't benefit from the format's optimizations

Facets

library · maturity active

vector-database search-engine serialization database data-science machine-learning machine-learning databases large-language-models computer-vision python rust jvm cross-platform cloud lakehouse-format columnar-format parquet-alternative vector-search acid-transactions multimodal-data apache-arrow time-travel data-versioning pylance data-engineering search

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lance-format/lance")