# moj-analytical-services/splink

Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

Repository: https://github.com/moj-analytical-services/splink
Canonical: https://ross.abutalabs.com/products/splink
Homepage: https://moj-analytical-services.github.io/splink/
Language: Python
License: MIT
License Family: permissive
Topics: record-linkage, spark, em-algorithm, deduplication, deduplicate-data, entity-resolution, data-matching, fuzzy-matching, data-science, duckdb, uk-gov-data-science
Last push: 2026-08-26T21:27:25+00:00

## Health v2 (maintenance only)
Score: 90/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 74, longevity 100
- inputs: {"age_days": 2476, "days_push": 7, "days_rel": 175, "gap_med": 23, "n_releases_24m": 16}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2364, forks 259 (observed 2026-08-28T04:06:41.509834+00:00)

## What it is
Splink is a Python library for fast, scalable probabilistic record linkage and entity resolution, based on the Fellegi-Sunter model with unsupervised EM training. It deduplicates and links records lacking unique identifiers, running on DuckDB locally or Spark/Athena for 100M+ records.

## Use cases
- deduplicate customer records without a unique ID
- link records across two datasets probabilistically
- entity resolution on millions of rows
- fuzzy match names and dates of birth
- train a record linkage model without labelled data
- scale data linkage to Spark or Athena

## When to choose
- you need to deduplicate or link tabular data lacking unique identifiers
- you want unsupervised probabilistic matching with no training labels
- you need to scale linkage from a laptop to hundreds of millions of records
- you want interactive diagnostics to understand match quality

## When to avoid
- your data has a reliable unique identifier - just join directly
- your columns are highly correlated or few in number
- you need deep-learning or embedding-based entity resolution
- your data is unstructured text rather than structured columns

## Facets
- artifact type: library
- maturity: active
- function: data-science, etl, search-engine
- domain: data-science, analytics, big-data
- platform: python, cli, cloud
- tags: record-linkage, entity-resolution, deduplication, probabilistic-matching, fellegi-sunter, duckdb, spark, fuzzy-matching, em-algorithm, data-engineering, docker

## Member repositories
- moj-analytical-services/splink (main) score 90

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:41.509834+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:36:01.710223+00:00, confidence not recorded.
  - readme: https://github.com/moj-analytical-services/splink (fetched 2026-08-28T04:06:41.509834+00:00, sha ccdbd0effbee)
  - homepage: https://moj-analytical-services.github.io/splink/ (fetched 2026-08-29T10:16:31.774703+00:00, sha e3e3c92308c4)
  - registry_pypi: https://pypi.org/pypi/splink/json (fetched 2026-08-29T10:16:31.784270+00:00, sha 1962d98be02d)
- Data as of 2026-08-30T08:39:29.467469+00:00.