mvdbos/php-spider
Some data may be outdated — see provenance labels.
A configurable and extensible PHP web spider observed · 2026-09-22
Health v2 · maintenance only
72/100
- Activity 88
- Release rhythm 35
- Longevity 100
Flags: no_releases
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-10-21. Adoption (stars, forks) is never an input.
- gap_med: n/a
- age_days: 4985
- days_rel: n/a
- days_push: 77
- n_releases_24m: 0
Adoption not part of the score
1340 stars · 228 forks observed · 2026-09-22
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded
A configurable and extensible PHP web spider library for crawling websites. It supports breadth-first and depth-first traversal, URI discovery via XPath/CSS selectors, filters, caching, and HTTP authentication.
Use cases
- crawl a website and extract links
- build a site link checker in PHP
- scrape pages using XPath or CSS selectors
- respect robots.txt while crawling
- cache downloaded pages during a crawl
When to choose
- you need a PHP-native crawler with composable discoverers and filters
- you want crawl limits, politeness, and persistence handlers out of the box
- your targets are static HTML without JavaScript rendering
When to avoid
- you need to crawl JavaScript-heavy single-page applications
- you need distributed or very large-scale crawling
- you are not working in PHP
Facets
library · maturity active
web-scraping http-client caching crawlers web-development developer-tools php cli spider php-library composer robots-txt breadth-first-search depth-first-search xpath css-selectors
1 source
- readme: https://github.com/mvdbos/php-spider · fetched 2026-09-22 · 6b19a50017df
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| mvdbos/php-spider | main | 72 |
For agents
markdown · JSON · MCP: product_card(name="mvdbos/php-spider")
Data as of 2026-10-19T08:05:51.212177+00:00 · Report a problem