Automatic Wrappers for Large Scale Web Extraction

By Nilesh N. Dalvi, Ravi Kumar, Mohamed A. Soliman. 2011

147 citations recorded by the source corpus.

We present a generic framework to make wrapper induction algorithms tolerant to noise in the training data. This enables us to learn wrappers in a completely unsupervised manner from automatically and cheaply obtained noisy training data, e.g., using dictionaries and regular expressions. By removing the site-level supervision that wrapper-based techniques require, we are able to perform information extraction at web-scale, with accuracy unattained with existing unsupervised extraction techniques. Our system is used in production at Yahoo! and powers live applications.

Subfields: Information Systems, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Signal Processing

Citations in the corpus

A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, background): “As the Web contains large numbers of structured datasets, there have been significant efforts to automatically extract the useful ones [32], [33], [34].”
Crowdsourcing large scale wrapper inference (2014, background): “More recently, a few wrapper inference approaches aiming at scaling at the Web by improving the automation level have been proposed: the system discussed in [10] automatically annotates the pages needed by a supervised technique tolerant to noise in the training data.”
Multimodal Learning for Web Information Extraction (2017, methodology): “While programs like wrapper [9] can extract information from well structured text with high precision, they usually fail when the text becomes less structured.”
Zero-shot Entity Extraction from Web Pages (2014, unknown): “We expect that this problem can be solved by normalizing the web page, using an alternative web page representation (Cohen et al., 2002; Wang and Co-hen, 2009; Fumarola et al., 2011), or leveraging more expressive extraction predicates (Dalvi et al., 2011).”
Web Data Extraction, Applications and Techniques: A Survey (2012, background): “To this purpose, some authors [30,31] introduced the concept of wrapper robustness: they proposed a strategy to find, among all the XPath expressions capable of extracting the same information from a Web page, the one that is less influenced by potential changes in the structure of the page and such”
Automatic web-scale information extraction (2012, unknown): “Design, Experimentation, Performance”