Automated Crowdturfing Attacks and Defenses in Online Review Systems

By Yuanshun Yao, Bimal Viswanath, Jenna Cryan, Haitao Zheng, Ben Y. Zhao. 2017

169 citations recorded by the source corpus.

Malicious crowdsourcing forums are gaining traction as sources of spreading misinformation online, but are limited by the costs of hiring and managing human workers. In this paper, we identify a new class of attacks that leverage deep learning language models (Recurrent Neural Networks or RNNs) to automate the generation of fake online reviews for products and services. Not only are these attacks cheap and therefore more scalable, but they can control rate of content output to eliminate the signature burstiness that makes crowdsourced campaigns easy to detect. Using Yelp reviews as an example platform, we show how a two phased review generation and customization attack can produce reviews that are indistinguishable by state-of-the-art statistical detectors. We conduct a survey-based user study to show these reviews not only evade human detection, but also score high on "usefulness" metrics by users. Finally, we develop novel automated defenses against these attacks, by leveraging the lossy transformation introduced by the RNN training and generation cycle. We consider countermeasures against our mechanisms, show that they produce unattractive cost-benefit tradeoffs for attackers, and that they can be further curtailed by simple constraints imposed by online service providers.

Subfields: Computer Science Applications, Information Systems, Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Information Systems

In Sociology and Political Science

In Safety, Robustness & Interpretability

Citations in the corpus

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks (2018, background): “Such models are commonly used as the basis of everything from sentiment analysis to compression [36, 53].”
The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets (2018, background): “Language models are well-studied in other domains, and have been shown to be effective at many different tasks [38, 42, 53].”
Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning (2019, unknown): “Besides the above, there exist a wide range of other attacks and defenses on machine learning models [4,5,8,9,13,14,16, 17, 22, 23, 25, 26, 35, 41, 43, 44, 46, 48–50, 52, 54–56]”
Keep Calm and Switch On! Preserving Sentiment and Fluency in Semantic Text Exchange (2019, methodology): “The MRT is similar to the temperature parameter used to control the “novelty” of generated text in works such as Yao et al. (2017).”
The AI-Based Cyber Threat Landscape (2020, methodology): “[88] proposed an automated review attack by leveraging a DNN-based language model to produce realistic content for online reviews that are indistinguishable from those created by real humans.”
False Information on Web and Social Media: A Survey (2018, background): “However, several research studies have conducted experiments to measure the ability of humans to detect false information including hoaxes, fake reviews, and fake news, and have shown that humans are not particularly good at discerning false from true information [50, 70, 74, 114].”
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning (2018, background): “Attacks against such ML models can be targeted towards compromising the integrity of the model (such as by evasion attacks [5,13,18,32,63,76,79,84,87]), or its privacy and con-ﬁdentiality .”