crawling
Here are 40 public repositories matching this topic...
Declarative web scraping
-
Updated
Apr 1, 2024 - Go
A Chrome DevTools Protocol driver for web automation and scraping.
-
Updated
Aug 19, 2024 - Go
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
-
Updated
Jan 23, 2024 - Go
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
-
Updated
Aug 17, 2024 - Go
Extract structured data from web sites. Web sites scraping.
-
Updated
Mar 7, 2023 - Go
Second-order subdomain takeover scanner
-
Updated
Mar 6, 2023 - Go
🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)
-
Updated
May 19, 2021 - Go
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
-
Updated
May 31, 2020 - Go
Download a large list of files concurrently
-
Updated
Oct 27, 2019 - Go
A fast, modern and intelligent proxy rotator perfect for crawling and scraping public data.
-
Updated
Mar 16, 2018 - Go
Fast, highly configurable, cloud native dark web crawler.
-
Updated
Jul 3, 2023 - Go
Content Discovery Development Platform. A tool to create your own CD solution. This is the new official repo for the project, old C++ and Rust versions are now closed, please follow this repo for updates.
-
Updated
Aug 22, 2024 - Go
Advanced declarative web scraping
-
Updated
Feb 25, 2023 - Go
An open source web crawling platform
-
Updated
May 6, 2018 - Go
Improve this page
Add a description, image, and links to the crawling topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the crawling topic, visit your repo's landing page and select "manage topics."