The Creation and Analysis of a Website Privacy Policy Corpus

By Shomir Wilson, Florian Schaub, Aswarth Abhilash Dara, Frederick Liu, Sushain Cherivirala, Pedro Giovanni Leon, Mads Schaarup Andersen, Sebastian Zimmeck, Kanthashree Mysore Sathyendra, N. Russell and 4 more. Annual Meeting of the Association for Computational Linguistics, 2016

346 citations recorded by the source corpus.

Website privacy policies are often ignored by Internet users, because these documents tend to be long and difficult to understand. However, the significance of privacy policies greatly exceeds the attention paid to them: these documents are binding legal agreements between website operators and their users, and their opaqueness is a challenge not only to Internet users but also to policy regulators. One proposed alternative to the status quo is to automate or semi-automate the extraction of salient details from privacy policy text, using a combination of crowdsourcing, natural language processing, and machine learning. However, there has been a relative dearth of datasets appropriate for identifying data practices in privacy policies. To remedy this problem, we introduce a corpus of 115 privacy policies (267K words) with manual annotations for 23K fine-grained data practices. We describe the process of using skilled annotators and a purpose-built annotation tool to produce the data. We provide findings based on a census of the annotations and show results toward automating the annotation procedure. Finally, we describe challenges and opportunities for the research community to use this corpus to advance research in both privacy and language technologies.

Subfields: Artificial Intelligence, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/e14883353786ca0f38f743cc064b1dc4b2ff73bd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hamza Harkous
2. Kang G. Shin
3. Karl Aberer
4. Kassem Fawaz
5. Rémi Lebret
6. Florian Schaub

In Sociology and Political Science

1. Karl Aberer
2. Florian Schaub

Citations in the corpus

• Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning (2018, methodology): “Although the OPP-115 taxonomy covers a wide variety of privacy practices [11], there are certain types of applications that it does not fully capture.”
• The Medium Is Not the Message: Deconfounding Document Embeddings via Linear Concept Erasure (2025, unknown): “…(Wang et al., 2025; Koreeda and Manning, 2021), LegalBenchCorporateLobbying (Guha et al., 2023; Holzenberger and Van Durme, 2021; Lippi et al., 2019; Ravichander et al., 2019; Wang et al., 2023; Wilson et al., 2016; Zheng et al., 2021; Zimmeck et al., 2019), LegalSummarization (Manor and Li, 2019).”