Web Crawling
By Christopher Olston, Marc Najork. Encyclopedia of Social Network Analysis and Mining, 2014
480 citations recorded by the source corpus.
View Paper: https://www.semanticscholar.org/paper/000d6c5b079b543fb1d17eab935c29026016922f
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Izzeddin Gur
- Ofir Nachum
- Mustafa Safdari
- Aakanksha Chowdhery
- Aleksandra Faust
- Sharan Narang
- Yingjie Miao
- Noah Fiedel
Citations in the corpus
- Understanding HTML with Large Language Models (2022, background): “Web crawling (Olston et al., 2010), form-filling (Diaz et al., 2013; Gur et al., 2021), or information retrieving web agents (Nogueira and Cho, 2016) are important for both automating and assisting users in web-based tasks.”
- Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discovery and Extraction (2021, background): “As web crawling involves discarding much of the downloaded content (Olston and Najork, 2010), especially link filtering and prioritization can prove to be tricky for contexts in which data collection is just the first step of a project, so that time resources for this task are scarce.”