White Paper: The Deep Web: Surfacing Hidden Value
By Michael K. Bergman. 2001
547 citations recorded by the source corpus.
Traditional search engines create their indices by spidering or crawling surface Web pages. To be discovered, the page must be static and linked to other pages. Traditional search engines can not "see" or retrieve content in the deep Web — those pages do not exist until they are created dynamically as the result of a specific search. Because traditional search engine crawlers can not probe beneath the surface, the deep Web has heretofore been hidden. The deep Web is qualitatively different from the surface Web. Deep Web sources store their content in searchable databases that only produce results dynamically in response to a direct request. But a direct query is a "one at a time" laborious way to search. BrightPlanet's search technology automates the process of making dozens of direct queries simultaneously using multiple-thread technology and thus is the only search technology, so far, that is capable of identifying, retrieving, qualifying, classifying, and organizing both "deep" and "surface" content.
Subfields: Sociology and Political Science
View Paper: https://www.semanticscholar.org/paper/ea572747901d2d5ed23e3e58773e11bad784ff3a
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Sociology and Political Science
- Adrian Kirkwood
- Linda Price
- Panagiotis G. Ipeirotis
- Anindya Ghose
- Andreas Munzel
- Sangwon Park
- Bing Liu
Citations in the corpus
- Efficient Filtering on Hidden Document Streams (2014, background): “Exploring Hidden Web Databases The Hidden Web (Bergman 2001) is the part of the Web that is accessible through forms, instead of hyperlinks.”
- Learners and learning in the twenty‐first century: what do we know about students’ attitudes towards and experiences of information and communication technologies that will help us design courses? (2005, background): “It has been estimated (Bergman, 2001) that most common search engines access less than 20% of the ‘Surface Web’ and that information in specialist databases and archives forms a ‘Deep Web’ that is 400 to 550 times larger than the ‘Surface Web’.”