sfu-db/connector-x

Health v2 · maintenance only

Adoption not part of the score

2659 stars · 227 forks observed · 2026-10-10

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

ConnectorX is a Rust-based library with Python bindings that loads data from databases into DataFrames (pandas, Arrow, Polars) as fast and memory-efficiently as possible. It supports parallel partitioned loading and experimental federated queries across multiple databases with a single read_sql call.

Use cases

• load a large postgres table into a pandas dataframe fast
• export sql query results to arrow or parquet
• parallelize database reads by partitioning on a numeric column
• join tables from two different databases in one query
• replace pd.read_sql for better performance and memory usage
• bulk extract data from a database for etl pipelines

When to choose

• you need the fastest possible database-to-DataFrame loading in Python
• you are loading multi-GB tables and memory efficiency matters
• you want parallel reads from a single database via partitioning
• you need to combine data from multiple databases in one query

When to avoid

• you need writes or general ORM functionality - it is read-only
• your queries are small and simple where driver overhead is negligible
• you need partitioning on non-numeric or nullable columns
• you need a pure-Python dependency-free solution

Facets

library · maturity active

database-driver etl data-science databases data-science analytics python rust cross-platform dataframe sql parallel-loading pandas arrow federated-queries data-engineering

Member repositories

For agents

markdown · JSON · MCP: product_card(name="sfu-db/connector-x")