Boilerplate detection using shallow text features

By Christian Kohlschütter, Péter Fankhauser, Wolfgang Nejdl. 2010

580 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

End-to-End Neural Ad-hoc Ranking with Kernel Pooling (2017, methodology): “e full texts of testing documents were crawled and parsed using Boilerpipe [13] for our word-based baselines (described in Section 4.”
Web Data Extraction, Applications and Techniques: A Survey (2012, methodology): “An approach for boilerplate detection using shallow text features was introduced in [70].”
Domain Adaptation for Medical Text Translation using Web Resources (2014, methodology): “Therefore, we use the Boilerpipe tool (Kohlschütter et al., 2010) to filter these noisy data and preserve the useful content that is marked by the tag, <canonicalDocument>.”
Context-aware Image Tweet Modelling and Recommendation (2016, background): “It is worth noting that the fusion strategy helps to reduce the acquisition cost of contexts by 18.0% in our dataset (when treating all API calls as a unit cost), and provides better semantic modelling for image tweets (demonstrated in Section 5.2).”
ZenCrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking (2012, methodology): “After the full text of the articles has been extracted from the HTML page [22], 489 entities were extracted from it using the Stanford Parser [21] as entity extractor.”
AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages (2020, methodology): “For many news websites, we used BoilerPipe3, a tool to automatically extract the main article content for structured pages without any site-specific learning and customizations (Kohlschütter et al., 2010).”
The Klarna Product Page Dataset: A Realistic Benchmark for Web Representation Learning (2021, background): “It was gathered in 2008 and consists of 621 English Google News articles from 408 different websites [17].”
Machine Translation of Medical Texts in the Khresmoi Project (2014, methodology): “The Boilerpipe (Kohlsch¨utter et al., 2010) and Justext (Pomik´alek, 2011) tools were used to remove boil-erplate texts and extract just the main content from the web pages.”
An analysis of the coherence of descriptors in topic modeling (2015, methodology): “The article body text was extracted using the Java Boilerpipe library 3 (Kohlschütter, Fankhauser, & Nejdl, 2010), with articles containing empty body text being ﬁltered.”
We Value Your Privacy ... Now Take Some Cookies (2018, methodology): “After retrieving a total of 112,041 privacy policies, we preprocessed these files with Boilerpipe, an HTML text extraction library, to remove unnecessary HTML code from the documents [21].”
Learning Semantic Relationships between Entities in Twitter (2011, methodology): “We implemented the relation discovery framework as a Java library using the Twitter streaming API(5) to monitor tweets, boilerpipe(6) [15] for extracting the main content of news articles from websites such as BBC or CNN and OpenCalais(7) for extracting entities from tweets and news articles respect”
iNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages (2020, methodology): “For many news websites, we used BoilerPipe 3 , a tool to automatically extract the main article content for structured pages without any site-specific customizations (Kohlschütter et al., 2010).”