# CIRCL/AIL-framework

AIL framework - Analysis Information Leak framework. Project moved to https://github.com/ail-project

Repository: https://github.com/CIRCL/AIL-framework
Canonical: https://ross.abutalabs.com/products/ail-framework
Homepage: https://github.com/ail-project/ail-framework
Language: Python
License: AGPL-3.0
License Family: copyleft
Topics: ail-framework, information-leak, information-security, analysis, data-mining, security, security-incidents, leak, privacy
Last push: 2026-08-19T12:54:34+00:00

## Health v2 (maintenance only)
Score: 67/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 98, release rhythm 8, longevity 100
- inputs: {"age_days": 4411, "days_push": 14, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1378, forks 288 (observed 2026-08-28T04:04:33.548234+00:00)

## What it is
AIL framework is an open-source Python platform for collecting, crawling, processing, and analyzing unstructured data from the clear web, Tor, I2P, chat platforms, files, and external feeds. Originally developed at CIRCL, it helps analysts extract, detect, correlate, and share intelligence such as leaked credentials and threat information.

## Use cases
- monitor dark web marketplaces and Tor hidden services for leaked data
- detect leaked credentials and sensitive information mentions
- retro-hunt historical data with keywords, regex, and YARA rules
- crawl and analyze chat platform communications for threat intelligence
- correlate extracted indicators and export them to MISP
- process unstructured text with OCR, decoding, and tagging pipelines

## When to choose
- you need continuous monitoring of leaks, pastes, chats, or hidden services for security intelligence
- you want an extensible Python framework with detection, correlation, and MISP integration
- you run a SOC or CERT and need automated retro-hunting with YARA and regex

## When to avoid
- you need a simple one-off web scraper without analysis workflows
- you cannot self-host a multi-component Python framework with crawlers and feeders
- your focus is structured data analytics rather than unstructured threat intelligence

## Facets
- artifact type: framework
- maturity: active
- function: web-scraping, nlp, search-engine, monitoring, security, data-science, ocr, etl
- domain: security, osint, crawlers, privacy
- platform: python, self-hosted
- tags: threat-intelligence, leak-analysis, tor-crawler, yara, misp-integration, dark-web-monitoring, retro-hunting, natural-language-processing, linux, docker

## Member repositories
- CIRCL/AIL-framework (main) score 67

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:04:33.548234+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T04:40:20.498860+00:00, confidence not recorded.
  - readme: https://github.com/CIRCL/AIL-framework (fetched 2026-08-28T04:04:33.548234+00:00, sha 1695e6ae1589)
  - homepage: https://github.com/ail-project/ail-framework (fetched 2026-08-29T11:56:23.369702+00:00, sha 64ba322a1af0)
- Data as of 2026-08-30T08:39:29.467469+00:00.