function: web-scraping
1985 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| DaidoujiChen/Dai-Hentai Dai-Hentai is an iOS app for browsing, reading, and favoriting content from the e-Hentai/EXHentai adult manga sites. It is written in Objec… | 23 | 1266 | maintenance |
| useragents/Zefoy-TikTok-Automator A Python/Selenium automation script that drives zefoy.com to send TikTok followers, views, likes, shares, favorites, and comment likes. It … | 32 | 1265 | maintenance |
| kiddyuchina/Beanbun Beanbun is a multi-process web crawler framework written in PHP, built on Workerman with Guzzle as the default downloader. It supports dist… | 61 | 1259 | maintenance |
| CyberPunkMetalHead/gateio-crypto-trading-bot-binance-announcements-new-coins A Python crypto trading bot that scrapes the Binance Announcements page for new coin listings and automatically places buy orders on Gate.i… | 32 | 1242 | maintenance |
| dchrastil/ScrapedIn A Python CLI tool that scrapes LinkedIn without API restrictions to enumerate employees of a target company for red team or social engineer… | 32 | 1235 | maintenance |
| istresearch/scrapy-cluster Scrapy Cluster is a distributed web scraping framework built on Scrapy that uses Redis to coordinate crawl requests and Kafka as a data bus… | 10 | 1225 | maintenance |
| vysecurity/LinkedInt LinkedInt is a Python CLI tool for LinkedIn reconnaissance that scrapes employee profiles for a target company and generates an HTML report… | 10 | 1214 | maintenance |
| yqchilde/JDMemberCloseAccount A Python/Selenium-based automation tool that automatically quits all JD (Jingdong) shop memberships. It handles SMS verification codes via … | 23 | 1209 | maintenance |
| Viralmaniar/I-See-You ISeeYou is a Bash and JavaScript tool that captures a target's exact GPS coordinates (latitude/longitude) during social engineering or phis… | 32 | 1199 | maintenance |
| yhy0/github-cve-monitor A Python application that monitors GitHub every few minutes for newly published CVEs, security tool updates, and activity on watched reposi… | 32 | 1195 | maintenance |
| Mapaler/PixivUserBatchDownload A userscript (browser extension-style script) that batch-downloads all public works of a Pixiv artist directly from the Pixiv website, send… | 47 | 1191 | maintenance |
| mkoehnke/WKZombie WKZombie is a Swift headless browser framework for iOS and macOS that navigates websites and collects data without a user interface or API,… | 23 | 1185 | maintenance |
| timwhitez/crawlergo_x_XRAY A Python glue script that combines the crawlergo dynamic crawler with the XRAY passive vulnerability scanner, replaying crawled URLs throug… | 32 | 1182 | maintenance |
| yutto-dev/bilili bilili is a Python CLI tool for downloading Bilibili videos (including bangumi/season content) along with danmaku comments and subtitles. I… | 10 | 1180 | maintenance |
| csns1/bypass-paywalls-chrome-clean-magnolia1234 A Chromium browser extension that bypasses paywalls on supported news sites, allowing users to read articles for free. It supports custom s… | 32 | 1175 | maintenance |
| MLNLP-World/AI-Paper-Collector A tool from the MLNLP community that automatically collects AI-related papers from sources like DBLP, ACL Anthology, NIPS, and OpenReview, … | 32 | 1173 | maintenance |
| yorickpeterse/oga Oga is an XML/HTML parser library for Ruby with a simple API for parsing, modifying, and querying documents via XPath. It requires no syste… | 62 | 1171 | maintenance |
| Lucifer1993/SatanSword SatanSword is a Python-based red team penetration testing framework that integrates web fingerprinting, PoC-based vulnerability detection, … | 32 | 1171 | maintenance |
| knadh/tg-archive tg-archive is a Python CLI tool that syncs Telegram group chat messages into a local SQLite database and generates a static website archive… | 66 | 1165 | maintenance |
| WebSpiderUtils/verification_code A research repository documenting approaches and code for solving mainstream CAPTCHA systems such as Geetest, NetEase Yidun, and Aliyun CAP… | 32 | 1165 | maintenance |
| tongcheng-security-team/NextScan NextScan (飞刃) is an enterprise-grade distributed black-box vulnerability scanning platform built in Go, composed of Server, Agent, and Web … | 21 | 1164 | maintenance |
| dixudx/tumblr-crawler A Python script that downloads all photos and videos from specified Tumblr blogs. It supports batch site lists, proxy configuration, and sk… | 59 | 1157 | maintenance |
| holgerd77/django-dynamic-scraper A Django app that builds on the Scrapy framework and lets you create and manage Scrapy spiders through the Django admin interface. It remov… | 23 | 1157 | maintenance |
| jlobos/instagram-web-api A Node.js client library for Instagram's private web API, supporting login, profile access, photo upload, likes, comments, follows, and sea… | 23 | 1151 | maintenance |
| s045pd/DarkNet_ChineseTrading A Python-based real-time crawler that monitors Chinese-language darknet marketplaces over Tor, with automatic account registration, login, … | 10 | 1149 | maintenance |
| charlesbel/Microsoft-Rewards-Farmer A Python bot that uses Selenium to automatically earn Microsoft Rewards points by completing searches and daily tasks. It supports multiple… | 10 | 1138 | maintenance |
| yize8888/maotai A Node.js collection of scheduled reservation scripts for the i-Moutai app and several Guizhou e-commerce platforms (Huluxia group-buying s… | 24 | 1134 | maintenance |
| juancarlospaco/faster-than-requests A Python 3 HTTP client library claiming to be much faster than the popular Requests library, implemented with a compiled core and minimal d… | 67 | 1129 | maintenance |
| bonfy/github-trending A Python script that scrapes GitHub's trending page daily and archives the most popular repositories as Markdown files. It is designed to r… | 77 | 1128 | maintenance |
| moyada/stealer A self-hosted Python web application that parses and downloads watermark-free videos from Chinese short-video platforms such as Douyin, Kua… | 32 | 1128 | maintenance |
| kohlschutter/boilerpipe boilerpipe is a Java library for removing boilerplate (ads, navigation, headers) from HTML pages and extracting the main full text content.… | 32 | 1127 | maintenance |
| filamentgroup/glyphhanger Glyphhanger is a Node.js CLI tool for web font optimization. It crawls web pages to determine which unicode-ranges are actually used, and c… | 10 | 1123 | maintenance |
| medialab/artoo artoo.js is a JavaScript library injected into a webpage's context (typically via a bookmarklet) that provides client-side web scraping uti… | 23 | 1119 | maintenance |
| EricJMarti/inventory-hunter A Python-based stock monitoring bot that watches retail websites for inventory of items like CPUs, GPUs, and game consoles, and sends notif… | 32 | 1118 | maintenance |
| KathanP19/JSFScan.sh JSFScan.sh is a shell script that automates JavaScript reconnaissance for bug bounty hunting. Given a list of subdomains, it gathers JS fil… | 32 | 1107 | maintenance |
| cezheng/Fuzi Fuzi is a fast, lightweight Swift library for parsing XML and HTML documents, built on libxml2 as a Swift port of Ono. It provides a modern… | 23 | 1105 | maintenance |
| QuiteRSS/quiterss QuiteRSS is an open-source cross-platform RSS/Atom news feed reader written in C++ with Qt. It provides a fast, comfortable desktop interfa… | 23 | 1086 | maintenance |
| exorde-labs/exorde-client The Exorde client is a Python CLI worker node for the Exorde Network, a decentralized protocol where participants scrape social media and w… | 60 | 1085 | maintenance |
| agentcoinorg/evo.ninja Evo.ninja is a versatile generalist AI agent that adapts in real time by selecting pre-defined agent personas (researcher, CSV analyst, dev… | 28 | 1081 | maintenance |
| acikyazilimagi/afet-org An open-source earthquake relief platform (depremyardim.com / afetharita.com) that aggregates calls for help from Twitter, WhatsApp, Telegr… | 31 | 1079 | maintenance |
| wireghoul/htshells A collection of self-contained .htaccess files that turn Apache servers into web shells or launch various attacks when uploaded. It include… | 32 | 1076 | maintenance |
| metachris/pdfx PDFx is a Python command-line tool and library that extracts metadata, text, and references (PDFs, URLs, DOIs, arXiv IDs) from PDF document… | 10 | 1076 | maintenance |
| admintony/Prepare-for-AWD A collection of Python and PHP scripts for AWD (Attack with Defense) CTF competitions, including batch attack scripts for planting and trig… | 32 | 1075 | maintenance |
| devxprite/infoooze Infoooze is a Node.js-based OSINT (Open-Source Intelligence) CLI tool for quickly gathering information about websites, IP addresses, usern… | 23 | 1071 | maintenance |
| fuergaosi233/gitbook2pdf A Python CLI tool that asynchronously crawls GitBook documentation sites and converts their contents into a single PDF. It preserves the or… | 10 | 1067 | maintenance |
| jonbakerfish/TweetScraper TweetScraper is a Scrapy-based crawler that scrapes tweets and user information from Twitter Search without using Twitter's official APIs. … | 23 | 1062 | maintenance |
| KnockOutEZ/wigolo wigolo is a local-first MCP server that gives AI coding agents web intelligence tools: search across 18 engines, fetch with anti-bot escala… | 79 | 4694 | experimental |
| zaytoun/scihub.py An unofficial Python API and command-line tool for Sci-Hub that can search Google Scholar and download academic papers by DOI, PMID, or URL… | 32 | 1055 | maintenance |
| daprofiler/DaProfiler DaProfiler is a Python-based OSINT CLI tool that traces a person's digital identity from a first and last name, recovering email addresses,… | 10 | 1040 | maintenance |
| maanavshah/stock-market-india A Node.js/Express API service that fetches and serves stock market data from India's NSE and BSE exchanges. It exposes REST endpoints for m… | 32 | 1038 | maintenance |
| gdtool/zhaopp A Google Drive search engine built on Elasticsearch that indexes millions of files from shared team drives and lets users search and direct… | 32 | 1038 | maintenance |
| yassineaboukir/sublert Sublert is a Python CLI security and reconnaissance tool that uses certificate transparency logs to monitor new subdomains issued TLS/SSL c… | 32 | 1034 | maintenance |
| averagesecurityguy/scripts A collection of Python scripts written for use during penetration testing engagements, organized into categories like brute forcing, enumer… | 32 | 1033 | maintenance |
| falling-sky/source Falling Sky is the source code for the test-ipv6.com website, a browser-based IPv6 connectivity testing service built mostly with HTML and … | 71 | 1031 | maintenance |
| supzhang/epg A Python/Django-based system that scrapes TV program schedules (EPG) from multiple Chinese and Hong Kong/Taiwan sources and generates XMLTV… | 30 | 1031 | maintenance |
| olyble/callPhoneBoom A Python script that automates browser-based form submissions (via Selenium/DrissionPage) against scraped business websites to spam a targe… | 22 | 1031 | maintenance |
| admintony/svnExploit SvnExploit is a Python CLI tool that exploits SVN source code disclosure vulnerabilities, supporting both SVN <1.7 and >1.7 repository form… | 32 | 1029 | maintenance |
| zepen/predict_Lottery_ticket A Python application that scrapes historical draw data for Chinese lotteries (双色球/Double Color Ball and 大乐透/Super Lotto) and trains deep le… | 34 | 1025 | maintenance |
| Malfrats/xeuledoc A Python CLI tool that fetches information about public Google documents across Drive, Docs, Sheets, Slides, and other Google Workspace ser… | 32 | 1023 | maintenance |
| utkarshkukreti/select.rs A Rust library for extracting useful data from HTML documents, built around predicate-based DOM queries. It is designed for web scraping ta… | 38 | 1020 | maintenance |
| ReversecLabs/awspx awspx is a graph-based security tool that visualizes effective access and resource relationships in AWS environments. It resolves IAM polic… | 23 | 1018 | maintenance |
| bit4woo/teemo Teemo is a Python command-line reconnaissance tool that collects domains, subdomains, and email addresses for a target organization. It agg… | 32 | 1016 | maintenance |
| s7ckTeam/Glass Glass is a Python CLI tool for rapid fingerprint identification of asset lists, querying Fofa, ZoomEye, Shodan, and 360 Quake APIs to gathe… | 32 | 1010 | maintenance |
| TideSec/FuzzScanner FuzzScanner is a Ruby/Python-based reconnaissance toolset that batch-collects information about target websites, including subdomains, open… | 32 | 1008 | maintenance |
| anvaka/vs A web application that visualizes Google's autocomplete suggestions as an interactive graph. It renders search query expansions as connecte… | 72 | 1002 | maintenance |
| Algebra-FUN/WeReadScan A Python library that uses Selenium headless browsers to scan purchased books from WeRead (WeChat Reading) and convert them into local PDF … | 32 | 1002 | maintenance |
| bitmagnet-io/bitmagnet bitmagnet is a self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with a web UI, GraphQL API and Ser… | 62 | 4318 | experimental |
| browser-use/workflow-use Workflow Use is a Python tool by the Browser Use team for recording and running deterministic browser workflows with variables, falling bac… | 71 | 4149 | experimental |
| MIgHTy-alIeN/ai-trader-bot A Solidity smart contract plus Python automation script that claims to find and execute arbitrage opportunities between Uniswap pools and r… | 57 | 2689 | experimental |
| JSREI/ast-hook-for-js-RE A browser memory roaming tool for JavaScript reverse engineering that hooks variable assignments via AST-transformed proxy responses. It le… | 23 | 1912 | experimental |
| richardyc/Chrome-GPT Chrome-GPT is an experimental AutoGPT agent that uses LangChain and Selenium to control a full Chrome browser session on your desktop. It c… | 30 | 1742 | experimental |
| xyzeva/k-id-age-verifier A TypeScript tool that automatically completes K-ID age verification as an adult on platforms like Discord, Twitch, Kick, and Quora. It aut… | 45 | 1730 | experimental |
| PedroHBessa/backscan A Node.js web application that serves a fake 'Comprovante' (payment receipt) page and exfiltrates the visitor's geolocation to an attacker-… | 28 | 1691 | experimental |
| tinyfish-io/bigset-oss BigSet is a self-hostable application that turns a natural-language sentence into a structured, regularly refreshed dataset by dispatching … | 76 | 1684 | experimental |
| paulpierre/informer A Python library built on Telethon that masquerades as real Telegram user accounts to monitor and log messages from hundreds of channels pe… | 54 | 1662 | experimental |
| maplibre/maplibre-rs maplibre-rs is a portable vector map renderer written in Rust that uses WebGPU for cross-platform rendering on web, mobile, and desktop. It… | 76 | 1568 | experimental |
| danielgross/LlamaAcademy LlamaAcademy is a Python pipeline that crawls API documentation, generates synthetic training data with GPT-3.5/GPT-4, and fine-tunes a Vic… | 30 | 1198 | experimental |
| kkyon/botflow Botflow is a Python dataflow programming framework for building data pipelines using pipes and routes, with parallelism via coroutines and … | 62 | 1196 | experimental |
| theacodes/kicanvas KiCanvas is an interactive, browser-based viewer for KiCad schematics and PCB boards, rendered with WebGL and the Canvas element. It is wri… | 68 | 1127 | experimental |
| distribyted/distribyted Distribyted is a torrent client written in Go that exposes torrent contents as standard filesystems via FUSE, WebDAV, or HTTP endpoints. It… | 67 | 1127 | experimental |
| oxylabs/ai-scraper-py AI-Scraper is a Python library and scrape agent from Oxylabs AI Studio that extracts data from webpages using natural language prompts inst… | 51 | 1055 | experimental |
| ariya/phantomjs PhantomJS is a headless WebKit browser scriptable with JavaScript, supporting page automation, screen capture, headless web testing, and ne… | 10 | 29449 | abandoned |
| leiurayer/downkyi Downkyi (哔哩下载姬) was a Windows desktop GUI application for downloading videos, danmaku (bullet comments), subtitles, and audio from Bilibili… | 64 | 24424 | abandoned |
| segment-boneyard/nightmare Nightmare is a high-level browser automation library from Segment that exposes simple methods like goto, type, and click with a synchronous… | 23 | 19768 | abandoned |
| twintproject/twint Twint is a Python CLI tool and library that scrapes tweets, followers, following, and likes from Twitter without using the official API or … | 10 | 16398 | abandoned |
| nilaoda/BBDown BBDown is a command-line tool for downloading videos from Bilibili, written in C#. It supports fetching video and audio streams, subtitles,… | 10 | 13900 | abandoned |
| schickling/chromeless Chromeless is a TypeScript library providing a simple API for automating Chrome, either locally or headlessly on AWS Lambda. It supports br… | 10 | 13223 | abandoned |
| wechat-article/wechat-article-exporter An online batch downloader for WeChat Official Account articles that exports posts in HTML, JSON, Excel, TXT, Markdown, and DOCX formats, i… | 68 | 12789 | abandoned |
| xiandanin/magnetW MagnetW is a cross-platform desktop application built with Electron and Vue that aggregates magnet link search results from multiple torren… | 10 | 11271 | abandoned |
| venera-app/venera Venera is a cross-platform comic reader app built with Flutter that supports reading both local and network comics. It allows users to exte… | 10 | 10884 | abandoned |
| seven332/EhViewer An unofficial Android client for browsing E-Hentai, supporting manga, comics, and doujinshi galleries. The project is deprecated and no lon… | 10 | 9795 | abandoned |
| yangyangwithgnu/hardseed hardseed is a C++ command-line tool that scrapes adult forum threads (aicheng, caoliu) to batch-download images and torrent seed files, wit… | 32 | 9198 | abandoned |
| clips/pattern Pattern is a Python web mining module bundling tools for scraping (Google, Twitter, Wikipedia APIs, crawler, HTML DOM parser), NLP (POS tag… | 66 | 8860 | abandoned |
| fuck-xuexiqiangguo/Fuck-XueXiQiangGuo A cross-platform Electron desktop application that automates studying and point-earning on the Chinese 'Xuexi Qiangguo' (学习强国) app by simul… | 32 | 8332 | abandoned |
| TonyChen56/WeChatRobot A C++ WeChat hooking toolkit and robot framework providing wxhook APIs, WeChat database decryption, and Official Account (公众号) scraping, pl… | 58 | 7188 | abandoned |
| casperjs/casperjs CasperJS is a navigation scripting and testing utility for the PhantomJS and SlimerJS headless browsers, providing high-level APIs for defi… | 10 | 7159 | abandoned |
| smuyyh/BookReader An Android e-book reader app ('任阅') for online novels and local txt/pdf/epub books, featuring 3D page-turning effects, bookshelf management… | 32 | 6936 | abandoned |
| beakerbrowser/beaker Beaker Browser was an experimental peer-to-peer web browser built on Electron that added APIs for hosting 'hostless' applications using the… | 10 | 6744 | abandoned |
| wuhan2020/wuhan2020 A volunteer-built information collection platform for the 2020 Wuhan COVID-19 outbreak, aggregating data on hospitals, hotels, logistics, d… | 32 | 5913 | abandoned |
| xchaoinfo/fuck-login A Python library that simulates programmatic login to popular Chinese websites like Zhihu, Weibo, Baidu, and Douban, built on requests, Pil… | 10 | 5870 | abandoned |