WebSRC: A Dataset for Web-Based Structural Reading Comprehension

By Lu Chen, Xingyu Chen, Zihan Zhao, Danyang Zhang, Jiabao Ji, Ao Luo, Kai Yu. 2021

123 citations recorded by the source corpus.

Web search is an essential way for humans to obtain information, but it’s still a great challenge for machines to understand the contents of web pages. In this paper, we introduce the task of web-based structural reading comprehension. Given a web page and a question about it, the task is to find an answer from the web page. This task requires a system not only to understand the semantics of texts but also the structure of the web page. Moreover, we proposed WebSRC, a novel Web-based Structural Reading Comprehension dataset. WebSRC consists of 400K question-answer pairs, which are collected from 6.4K web pages with corresponding HTML source code, screenshots, and metadata. Each question in WebSRC requires a certain structural understanding of a web page to answer, and the answer is either a text span on the web page or yes/no. We evaluate various strong baselines on our dataset to show the difficulty of our task. We also investigate the usefulness of structural information and visual features. Our dataset and baselines have been publicly available.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science, Evaluation & Benchmarks, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Furu Wei
2. Li Dong
3. Shaohan Huang
4. Shuming Ma
5. Wenhui Wang
6. Mohit Bansal
7. Jianfeng Gao
8. Xia Song

In Information Systems

1. Furu Wei
2. Li Dong
3. Shuming Ma
4. Shaohan Huang
5. Vishrav Chaudhary
6. Lei Cui
7. Sheng Wang
8. Baobao Chang

In Sociology and Political Science

1. Douglas Eck
2. Qiang Liu
3. Ofir Nachum

In Evaluation & Benchmarks

1. Furu Wei
2. Li Dong
3. Zewen Chi
4. Saksham Singhal
5. Vishrav Chaudhary
6. Lei Cui
7. Shuming Ma
8. Liang Chen

In Question Answering & Reading Comprehension

1. Li Dong
2. Furu Wei
3. Xiaohan Zou

Citations in the corpus

• Language Is Not All You Need: Aligning Perception with Language Models (2023, methodology): “We compare the performance on the Web-based Structural Reading Comprehension (WebSRC) dataset [43].”
• Understanding HTML with Large Language Models (2022, methodology): “WebSRC (Chen et al., 2021) introduced a QA task from web documents.”
• MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning (2023, background): “Web Page Question Answering (Web QA) includes the Websrc (Chen et al., 2021a; Huang et al., 2023a) datasets.”
• LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models (2024, unknown): “The judge model assigns a score from [1 , 10] based on the provided ground-truth answer, detailed in Section 4.2.3.”
• A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis (2023, methodology): “To emphasize the advantage of our modular approach, we test WebAgent on a static website comprehension benchmark, WebSRC (Chen et al., 2021b), which is a contextual QA dataset with HTML documents.”