NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails

By Traian Rebedea, R. Dinu, Makesh Narsimhan Sreedhar, Christopher Parisien, J. Cohen. Conference on Empirical Methods in Natural Language Processing, 2023

423 citations recorded by the source corpus.

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems. Guardrails (or rails for short) are a specific way of controlling the output of an LLM, such as not talking about topics considered harmful, following a predefined dialogue path, using a particular language style, and more. There are several mechanisms that allow LLM providers and developers to add guardrails that are embedded into a specific model at training, e.g. using model alignment. Differently, using a runtime inspired from dialogue management, NeMo Guardrails allows developers to add programmable rails to LLM applications - these are user-defined, independent of the underlying LLM, and interpretable. Our initial results show that the proposed approach can be used with several LLM providers to develop controllable and safe LLM applications using programmable rails.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/b428e9e0e8ac36b3ae29a7ab9b9554c39cedb283

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ziwei Xu
2. Mohan Kankanhalli
3. S. Jain

Citations in the corpus

• Hallucination is Inevitable: An Innate Limitation of Large Language Models (2024, background): “Guardrails and fences can be formally programmed [53] to explicitly affect LLMs’ behaviours.”
• HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (2024, neutral): “These include input and output filtering (Markov et al., 2023; Inan et al., 2023; Computer, 2023; Li et al., 2023; Cao et al., 2023; Jain et al., 2023), input sanitization (Jain et al., 2023) and modification (Zhou et al., 2024), and constrained inference (Rebedea et al., 2023).”
• ARBITER: AI-Driven Filtering for Role-Based Access Control (2025, unknown): “For that we introduce ARBITER ( A I R ole B ased I ntelligent T ext E valuato R ) – a system that leverages NeMo Guardrails [8] to create a multi-layered filter chain without relying on fine-tuned models.”