Generating Fake Cyber Threat Intelligence Using Transformer-Based Models

By Priyanka Ranade, Aritran Piplai, Sudip Mittal, Anupam Joshi. 2021

100 citations recorded by the source corpus.

Cyber-defense systems are being developed to automatically ingest Cyber Threat Intelligence (CTI) that contains semi-structured data and/or text to populate knowledge graphs. A potential risk is that fake CTI can be generated and spread through Open-Source Intelligence (OSINT) communities or on the Web to effect a data poisoning attack on these systems. Adversaries can use fake CTI examples as training input to subvert cyber defense systems, forcing their models to learn incorrect inputs to serve the attackers' malicious needs. In this paper, we show how to automatically generate fake CTI text descriptions using transformers. Given an initial prompt sentence, a public language model like GPT-2 with fine-tuning can generate plausible CTI text that can mislead cyber-defense systems. We use the generated fake CTI text to perform a data poisoning attack on a Cybersecurity Knowledge Graph (CKG) and a cybersecurity corpus. The attack introduced adverse impacts such as returning incorrect reasoning outputs, representation poisoning, and corruption of other dependent AI-based cyber defense systems. We evaluate with traditional approaches and conduct a human evaluation study with cyber-security professionals and threat hunters. Based on the study, professional threat hunters were equally likely to consider our fake generated CTI and authentic CTI as true.

Subfields: Artificial Intelligence, Sociology and Political Science, Natural Language Processing, Privacy & Security, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Iason Gabriel
2. Po-Sen Huang
3. Jonathan Uesato
4. William S. Isaac
5. John F. J. Mellor
6. Borja Balle
7. Geoffrey Irving
8. Amelia Glaese

In Sociology and Political Science

1. William S. Isaac
2. Iason Gabriel
3. Borja Balle
4. Myra Cheng
5. Lisa Anne Hendricks
6. T. Stepleton
7. Atoosa Kasirzadeh
8. Abeba Birhane

In Natural Language Processing

1. Iason Gabriel
2. John F. J. Mellor
3. Borja Balle
4. Geoffrey Irving
5. Amelia Glaese
6. Myra Cheng
7. Lisa Anne Hendricks
8. Po-Sen Huang

In Privacy & Security

1. Borja Balle
2. T. Stepleton
3. Peter Kairouz
4. Aurélien Bellet
5. Badih Ghazi
6. Yu-Xiang Wang

In Signal Processing

1. Borja Balle
2. Po-Sen Huang
3. Jonathan Uesato

Citations in the corpus

• Ethical and social risks of harm from Language Models (2021, background): “For example, (Ranade et al., 2021) found that GPT-2 could generate fake information on cyber threats which were believed to be real threats by expert cybersecurity analysts.”
• Taxonomy of Risks posed by Language Models (2022, background): “Risks of disinformation also intersect with concerns about LMs creating new cyber security threats, as it was found that disinformation can be generated in target domains, such as cyber security, to distract the attention of specialists from addressing real vulnerabilities [155].”