Mining data records in Web pages

By B. Liu, Robert L. Grossman, Yanhong Zhai. Knowledge Discovery and Data Mining, 2003

570 citations recorded by the source corpus.

A large amount of information on the Web is contained in regularly structured objects, which we call data records. Such data records are important because they often present the essential information of their host pages, e.g., lists of products or services. It is useful to mine such data records in order to extract information from them to provide value-added services. Existing automatic techniques are not satisfactory because of their poor accuracies. In this paper, we propose a more effective technique to perform the task. The technique is based on two observations about data records on the Web and a string matching algorithm. The proposed technique is able to mine both contiguous and non-contiguous data records. Our experimental results show that the proposed technique outperforms existing techniques substantially.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/c8cdce10fc4c9505ca22796920d1cd4b0f82dc76

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Percy Liang
2. B. Liu
3. Ji-rong Wen
4. Panupong Pasupat
5. Yanhong Zhai
6. Bing Liu
7. Mohit Bansal
8. Jianfeng Gao

In Computer Networks and Communications

1. Weiyi Meng
2. Xiaofeng Meng
3. Wei David Liu
4. B. Liu
5. Yanhong Zhai
6. Mohammed Kayed
7. Moheb R. Girgis
8. Khaled Shaalan

In Information Systems

1. B. Liu
2. Weiyi Meng
3. Yanhong Zhai
4. Wei David Liu
5. Xiaofeng Meng
6. Panupong Pasupat
7. Mohammed Kayed
8. Moheb R. Girgis

Citations in the corpus

• Fully automatic wrapper generation for search engines (2005, methodology): “Several automated or nearly automated IE methods have been proposed recently and the most representative ones are Omini [5], the method in [12], IEPAD [6], MDR [20], RoadRunner [10], EXALG [2], DeLa [26], PickUp [8].”
• Zero-shot Entity Extraction from Web Pages (2014, background): “In contrast to information extraction systems that extract homogeneous records from web pages (Liu et al., 2003; Zheng et al., 2009), our system must choose the correct field from each record and also identify the relevant part of the page based on the query.”
• Object-level ranking: bringing order to Web objects (2005, background): “Since the information about an object is normally represented as a block of a Web page [12, 14].”
• A Survey of Web Information Extraction Systems (2006, methodology): “The described algorithm, called MDR [38], works in three steps.”
• ViDE: A Vision-Based Approach for Deep Web Data Extraction (2010, methodology): “We also choose MDR [17] and DEPTA [29] to compare with ViDRE and ViDIE, respectively.”
• Web data extraction based on partial tree alignment (2005, methodology): “We do not compare it with the method in [5] and the method in [8] here as it is shown in [21] that MDR is already more effective than them.”
• Automatic web news extraction using tree edit distance (2004, methodology): “have developed an effective algorithm for mining data records from Web pages [14].”