SafeText: A Benchmark for Exploring Physical Safety in Language Models

By Sharon Levy, Emily Allaway, Melanie Subbiah, Lydia B. Chilton, Desmond Upton Patton, K. McKeown, William Yang Wang. 2022

51 citations recorded by the source corpus.

Understanding what constitutes safe text is an important issue in natural language processing and can often prevent the deployment of models deemed harmful and unsafe. One such type of safety that has been scarcely studied is commonsense physical safety, i.e. text that is not explicitly violent and requires additional commonsense knowledge to comprehend that it leads to physical harm. We create the first benchmark dataset, SafeText, comprising real-life scenarios with paired safe and physically unsafe pieces of advice. We utilize SafeText to empirically study commonsense physical safety across various models designed for text generation and commonsense reasoning tasks. We find that state-of-the-art large language models are susceptible to the generation of unsafe text and have difficulty rejecting unsafe advice. As a result, we argue for further studies of safety and the assessment of commonsense physical safety in models before release.

Subfields: Artificial Intelligence, Information Systems, Radiological and Ultrasound Technology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Minlie Huang
2. Tatsunori B. Hashimoto
3. Dan Jurafsky
4. Hao Sun
5. Federico Bianchi
6. Zhexin Zhang
7. Paul Röttger
8. Jiawen Deng

In Information Systems

1. Minlie Huang
2. Dan Jurafsky
3. Tatsunori B. Hashimoto
4. Federico Bianchi
5. Mirac Suzgun

In Safety, Robustness & Interpretability

1. Zhexin Zhang
2. Hao Sun
3. Minlie Huang
4. Dan Jurafsky
5. Paul Röttger
6. Jiawen Deng
7. Jiale Cheng
8. Tatsunori B. Hashimoto

Citations in the corpus

• Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, background): “For instance, recent studies have emphasized the mental harm caused by inappropriate advice and the public opinion risks brought about by controversial re-marks (Dinan et al., 2021; Sun et al., 2022; Levy et al., 2022).”
• Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models (2023, background): “PhysicalUnSafe (100 samples) aims to generate unsafe instructions regarding common sense physical safety (Levy et al., 2022).”
• Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions (2023, methodology): “We take examples from Levy et al. (2022) and rephrase them to create instructions.”
• XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models (2023, unknown): “As a consequence, we believe that XST EST is most useful as a complement to other test sets, especially those that test model behaviour in response to unsafe prompts (e.g. Levy et al., 2022; Vidgen et al., 2023; Bianchi et al., 2024).”