Syntactic Clustering of the Web

By Andrei Z. Broder, S. Glassman, Mark S. Manasse, Geoffry Zweig. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

RUPEE: A fast and accurate purely geometric protein structure search (2018, methodology): “In IR, the bag of shingles representation of documents is used in the near dupe 161 clustering of documents [27].”
Towards Making Systems Forget with Machine Unlearning (2015, background): “In the training stage, OSNSF first clusters OSN messages based on the text shingling distance [26] and the URL elements of the messages, and then extracts the features of each cluster, such as cluster size, average social degree, and average time interval of messages.”
Winnowing: local algorithms for document fingerprinting (2003, methodology): “This idea was later used to show that it was possible to cluster documents on the Web by similarity [6].”
Discovering Large Dense Subgraphs in Massive Graphs (2005, methodology): “Shingles were introduced in [9], and have since seen wide usage to estimate the similarity of web pages us-ing a particular feature extraction scheme based on overlapping windows of terms (motivating the name “shingles”); they were also used to detect mirror sites (see Chakrabarti [11] and Bharat…”
Mining Query Logs: Turning Search Usage Data into Knowledge (2010, methodology): “Shingles have already been used in the past for clustering text collections, [50] and for detecting duplicate pages [63, 102].”
Efficient semi-streaming algorithms for local triangle counting in massive graphs (2008, methodology): “In the next section, we revisit the general technique [8, 9, 6, 7] to estimate the Jaccard coefficient and thus the size of the intersection of two sets A and B, defined over the same Table 1: Datasets used in the experiments.”
Methods for Identifying Versioned and Plagiarized Documents (2003, methodology): “Words can be separated by punctuation, whitespace, or control characters [2, 5, 18].”
Automated Crowdturfing Attacks and Defenses in Online Review Systems (2017, methodology): “Similarity between two reviews is computed using Jaccard Similarity [5] of their fingerprints generated from Winnowing.”
Fast locality-sensitive hashing (2011, background): “Likewise, popular operations such as near-duplicate detection [7, 28, 19], all-pairs similarity [6], similarity join/record-linkage [25], temporal correlation [10], are made easier.”
Detecting duplicate web documents using clickthrough data (2011, methodology): “One of the commonly used methods for detecting near duplicates is based on shingling [5, 7].”
Detecting Phishing Web Pages with Visual Similarity Assessment Based on Earth Mover's Distance (EMD) (2006, background): “The Web page obfuscation can be carried out in three basic ways: 1. using the downloaded Web page from the real Web site to make the phishing Web page appear and react exactly the same as the real one does, 2. using a script or images to cover the address bar to scam users into believing they have…”
Efficient similarity joins for near duplicate detection (2008, methodology): “, systematic sampling of shingles or super-shingling [6] were designed to trade result quality for efficiency, and are most likely to have even worse precision and recall values.”
Duplicate Record Detection: A Survey (2007, background): “Hence, we do not cover solutions for various other problems, such as that of mirror detection , in which the goal is to detect similar or identical Web pages (e.g., see [4], [5]).”
Detecting near-duplicates for web crawling (2007, methodology): “Concomitant with the development of our system in 2004—2005, Monika Henzinger conducted a study that compared simhash with Broder’s shingle-based fingerprints [14].”