Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

Histórico de datos sobre aparcamientos públicos de Málaga (Andalucía, España).

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Extraction, versioning and machine-readable provisioning of public data.

Booklify.me is an open-source platform for keeping track of everything in your bookshelf.

Headless Chrome .NET API

Scrapy Extension for monitoring spiders execution.

Sasori is a dynamic web crawler powered by Puppeteer, designed for lightning-fast endpoint discovery.

sitemapr is a library that generates sitemaps for SPA websites by reading site structures defined in declarative configuration.

Content Discovery Development Platform. A tool to create your own CD solution. This is the new official repo for the project, old C++ and Rust versions are now closed, please follow this repo for updates.

🕷 Automatically detect changes made to the official Telegram sites, clients and servers.

Lightweight and efficient web crawling using Python

A Chrome DevTools Protocol driver for web automation and scraping.

Scrapy, a fast high-level web crawling & scraping framework for Python.

This project is a B2B Link Shortener platform, offering businesses a customizable and feature-rich solution for URL shortening.

Run a high-fidelity browser-based crawler in a single Docker container

📅🇨🇳中国法定节假日数据 自动每日抓取国务院公告

A simple and easy to use web crawler for Python

🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖