Java Web Crawler: Difference between revisions
Jump to navigation
Jump to search
| Line 2: | Line 2: | ||
==References== | ==References== | ||
* [https://stackoverflow.com/questions/10399557/is-it-possible-to-run-selenium-firefox-web-driver-without-a-gui#:~:text=run%20Selenium%20on%20Firefox%20headlessly Selenium web driver without a GUI] | |||
* [https://stackoverflow.com/questions/11282503/java-web-crawler-libraries#:~:text=Selenium%20Components Java Web Crawler Libraries] | * [https://stackoverflow.com/questions/11282503/java-web-crawler-libraries#:~:text=Selenium%20Components Java Web Crawler Libraries] | ||
* [https://www.cloudflare.com/en-gb/learning/bots/what-is-a-web-crawler/#:~:text=A%20web%20crawler%2C%20or%20spider,appear%20in%20search%20engine%20results What Is a Web Crawler?] | * [https://www.cloudflare.com/en-gb/learning/bots/what-is-a-web-crawler/#:~:text=A%20web%20crawler%2C%20or%20spider,appear%20in%20search%20engine%20results What Is a Web Crawler?] | ||
* [https://www.selenium.dev/projects/ Selenium Projects] | * [https://www.selenium.dev/projects/ Selenium Projects] | ||
Revision as of 04:37, 24 September 2020
A web crawler, or spider, is a type of bot that's typically operated by search engines like Google and Bing. Their purpose is to index the content of websites all across the Internet so that those websites can appear in search engine results.