dlt-hub/dlt

Health v2 · maintenance only

Adoption not part of the score

5778 stars · 590 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

dlt (data load tool) is an open-source Python library for building ELT data pipelines that extract data from REST APIs, SQL databases, cloud storage, or Python data structures and load it into destinations like DuckDB, BigQuery, Snowflake, or S3. It automatically infers schemas, normalizes nested data, and handles incremental loading, schema evolution, and schema contracts.

Use cases

• load data from a REST API into a database
• build an ELT pipeline in Python
• ingest JSON from APIs into BigQuery or Snowflake
• load SQL database tables into a data warehouse
• move files from S3 into a data lake
• incrementally sync new and updated records
• unnest nested JSON into relational tables
• run data pipelines in Airflow or serverless functions

When to choose

• you need a Python-first, code-based alternative to GUI ETL tools like Fivetran or Airbyte
• you want automatic schema inference, typing, and normalization of messy source data
• you need incremental loading and schema evolution without maintaining pipeline code by hand
• you want pipelines that run anywhere Python runs, from notebooks to Airflow to Lambda

When to avoid

• you need heavy in-pipeline transformations rather than extract-and-load (dlt focuses on EL, with transformations via dltHub or external tools)
• your team prefers no-code/low-code managed connectors over writing Python
• you need streaming (sub-second) ingestion rather than batch or micro-batch loads

Facets

library · maturity stable

etl data-science streaming database sdk big-data databases analytics python cross-platform serverless cloud elt data-loading data-pipelines schema-inference incremental-loading rest-api reverse-etl data-warehouse data-lake data-engineering automation

Member repositories

For agents

markdown · JSON · MCP: product_card(name="dlt-hub/dlt")