tinyfish-io/bigset-oss
Open-source BigSet — self-hostable live datasets populated by TinyFish web agents observed · 2026-08-28
Health v2 · maintenance only
76/100
- Activity 99
- Release rhythm 87
- Longevity 7
Flags: young
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: 0.0
- age_days: 110
- days_rel: 85
- days_push: 9
- n_releases_24m: 3
Adoption not part of the score
1684 stars · 201 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded
BigSet is a self-hostable application that turns a natural-language sentence into a structured, regularly refreshed dataset by dispatching autonomous web agents to research, verify, deduplicate, and export the data. It infers schemas automatically and supports refresh cadences from 30 minutes to weekly, exporting to CSV or XLSX.
Use cases
- build a dataset of YC companies hiring engineers with funding stage and location
- keep a price list refreshed daily from the live web
- generate verified lead lists from company websites
- turn a one-sentence data request into a CSV table
- schedule agents to re-crawl and update a dataset every 6 hours
- extract structured job postings across many sites without writing scrapers
When to choose
- you need fresh structured data from the web on a recurring schedule
- you want schema inference and verification handled automatically instead of stitching scrapers, search APIs, and cron jobs
- you prefer self-hosting an open-source data pipeline
When to avoid
- you need a battle-tested production tool with stable APIs
- you only need simple scraping of known URLs
- your project requires a permissive license since BigSet is AGPL-3.0
Facets
application · maturity experimental
web-scraping data-generation agent-framework etl search-engine data-science crawlers artificial-intelligence self-hosted live-datasets autonomous-agents schema-inference scheduled-refresh csv-export xlsx-export tinyfish ai-agents automation web-server nodejs docker
1 source
- readme: https://github.com/tinyfish-io/bigset-oss · fetched 2026-08-28 · 755c4d95e349
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| tinyfish-io/bigset-oss | main | 76 |
For agents
markdown · JSON · MCP: product_card(name="tinyfish-io/bigset-oss")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem