mvdbos/php-spider
A configurable and extensible PHP web spider observed · 2026-08-28
Health v2 · maintenance only
75/100
- Activity 96
- Release rhythm 35
- Longevity 100
Flags: no_releases
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: n/a
- age_days: 4937
- days_rel: n/a
- days_push: 29
- n_releases_24m: 0
Adoption not part of the score
1341 stars · 229 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded
A configurable and extensible PHP web spider library for crawling websites. It supports breadth-first and depth-first traversal, URI discovery via XPath/CSS selectors, filters, caching, and HTTP authentication.
Use cases
- crawl a website and extract links
- build a site link checker in PHP
- scrape pages using XPath or CSS selectors
- respect robots.txt while crawling
- cache downloaded pages during a crawl
When to choose
- you need a PHP-native crawler with composable discoverers and filters
- you want crawl limits, politeness, and persistence handlers out of the box
- your targets are static HTML without JavaScript rendering
When to avoid
- you need to crawl JavaScript-heavy single-page applications
- you need distributed or very large-scale crawling
- you are not working in PHP
Facets
library · maturity active
web-scraping http-client caching crawlers web-development developer-tools php cli spider php-library composer robots-txt breadth-first-search depth-first-search xpath css-selectors
1 source
- readme: https://github.com/mvdbos/php-spider · fetched 2026-08-28 · 6b19a50017df
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| mvdbos/php-spider | main | 75 |
For agents
markdown · JSON · MCP: product_card(name="mvdbos/php-spider")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem