capitalone/DataProfiler

Health v2 · maintenance only

Adoption not part of the score

1578 stars · 187 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

DataProfiler is a Python library that loads CSV, AVRO, Parquet, JSON, text, or URL data into a pandas-compatible DataFrame and profiles it with a single command. It extracts schema, statistics, and entities, including sensitive data (PII/NPI) detection via a pre-trained deep learning model.

Use cases

• profile a csv dataset to get schema and statistics
• detect PII and sensitive data in datasets for GDPR compliance
• automatically load csv, avro, parquet, or files into a pandas dataframe
• identify entities like names, emails, and phone numbers in tabular or text data
• generate data quality reports for downstream monitoring
• profile graph or network data
• extend entity recognition with custom labels or regex pipelines

When to choose

• you need automated data profiling with statistics and schema extraction
• you must scan datasets for PII/NPI to meet privacy or compliance requirements
• you want a pre-trained sensitive-data detection model without building your own NLP pipeline
• you work with heterogeneous file formats and want a unified pandas-compatible loader

When to avoid

• you need a lightweight tool and cannot install TensorFlow or heavy ML dependencies
• you only need simple descriptive statistics already covered by pandas describe()
• you need real-time streaming profiling rather than batch analysis

Facets

library · maturity active

data-science nlp machine-learning privacy security analytics data-science privacy security developer-tools python cross-platform data-profiling pii-detection sensitive-data schema-extraction pandas csv avro parquet gdpr entity-recognition natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="capitalone/DataProfiler")