news-please - A Generic News Crawler and Extractor

By Felix Hamborg, Norman Meuschke, Corinna Breitinger, Bela Gipp. 2017

160 citations recorded by the source corpus.

Subfields: Information Retrieval & Search, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

1. Danqi Chen
2. Luke S. Zettlemoyer
3. Minjoon Seo
4. Michael Lewis
5. Omer Levy
6. Sohee Yang
7. Hannaneh Hajishirzi
8. Sebastian Riedel

In Information Extraction

1. Veselin Stoyanov
2. Luke S. Zettlemoyer
3. Michael Lewis
4. Omer Levy
5. Myle Ott
6. Danqi Chen

Citations in the corpus

• RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019, methodology): “We use news-please (Hamborg et al., 2017) to collect and extract CC-NEWS.”
• Towards Continual Knowledge Learning of Language Models (2021, methodology): “Dataset Construction The data for continual pretraining, CC-RECENTNEWS, is constructed using news-please (Hamborg et al., 2017).”
• Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning (2023, unknown): “, 2015), Stories (Trinh & Le, 2018), CCNews (Hamborg et al., 2017), the Pile (Gao et al.”
• One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2022, unknown): “…(Bowman et al., 2015) Symmetric 50,000 npr (Li, 2020) Asymmetric 25,000 specter_train_triples (Cohan et al., 2020) Symmetric 50,000 ccnews_title_text (Hamborg et al., 2017) Asymmetric 25,000 triviaqa (Joshi et al., 2017) Asymmetric 50,000 zero_shot_re (Levy et al., 2017) Asymmetric 15,000…”
• Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discovery and Extraction (2021, background): “…provides additional functions but no structured text or comment extraction • news-please 21 is a news crawler that extracts structured information (Hamborg et al., 2017) • readability-lxml 22 cleans the page and preserves some markup The tools are compared to the raw page source and to a…”
• Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text (2024, unknown): “Drawing samples of human-written text from CCNews (Hamborg et al., 2017), PubMed (Sen et al., 2008), and CNN (Hermann et al., 2015), we generate corresponding, machine-generated completions using LLaMA-2-7B and Falcon-7B (see details in Appendix A.1.1).”
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation (2024, unknown): “Particularly, the unsupervised data is curated by extracting the rich-semantic structures, e.g., title-body, title-abstract, instruction-output, etc., within a wide variety of multi-lingual corpora, including Wikipedia, S2ORC (Lo et al., 2020), xP3 (Muennighoff et al., 2023), mC4 (Raffel et al., 202”
• Nomic Embed: Training a Reproducible Long Context Text Embedder (2024, unknown): “(Lewis et al., 2021b) 52,953,088 0.23 Amazon Reviews (Ni et al., 2019) 38,682,624 0.16 S2ORC Title Abstract (Lo et al., 2020) 35438592 0.15 WikiAnswers (Fader et al., 2014) 9,912,320 0.04 S2ORC Citation Titles (Lo et al., 2020) 7,585,792 0.03 S2ORC Abstract Citation (Lo et al., 2020) 7,503,872 0.03 ”