This mini search engine should be programmed to perform parsing, crawling, indexing, and query-serving functions and return the results on a result page.
-
Updated
Jul 28, 2024 - Java
This mini search engine should be programmed to perform parsing, crawling, indexing, and query-serving functions and return the results on a result page.
Incremental crawling capabilities for Apache Tika. Crawl content out of e.g. file systems, http(s) sources (webcrawling) imap(s) servers or your own arbitrary data sources. LeechCrawler offers additional Tika parsers providing these crawling capabilities.
Crawljax
Search engine using a Trie Tree structure
ProxyCrawl Java library for scraping and crawling
Data Web Crawlers Benchmark for the HOBBIT platform
Simple search engine application that is capable of crawling articles from a website, store them in predefined format and later index them. These documents are available to be searched for by full-text querries from user interface.
Burp Suite's extension to scan and crawl Single Page Applications
Add a description, image, and links to the crawling topic page so that developers can more easily learn about it.
To associate your repository with the crawling topic, visit your repo's landing page and select "manage topics."