AI Alignment: A Comprehensive Survey
By Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang and 15 more. arXiv.org, 2023
376 citations recorded by the source corpus.
AI alignment aims to make AI systems behave in line with human intentions and values. As AI systems grow more capable, so do risks from misalignment. To provide a comprehensive and up-to-date overview of the alignment field, in this survey, we delve into the core concepts, methodology, and practice of alignment. First, we identify four principles as the key objectives of AI alignment: Robustness, Interpretability, Controllability, and Ethicality (RICE). Guided by these four principles, we outline the landscape of current alignment research and decompose them into two key components: forward alignment and backward alignment. The former aims to make AI systems aligned via alignment training, while the latter aims to gain evidence about the systems' alignment and govern them appropriately to avoid exacerbating misalignment risks. On forward alignment, we discuss techniques for learning from feedback and learning under distribution shift. On backward alignment, we discuss assurance techniques and governance practices. We also release and continually update the website (www.alignmentsurvey.com) which features tutorials, collections of papers, blog posts, and other resources.
Subfields: Information Systems
View Paper: https://www.semanticscholar.org/paper/a3d1954a57110f199ad58c24a6e588ee73135170
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Information Systems
Citations in the corpus
- Safety Alignment Should Be Made More Than Just a Few Tokens Deep (2024, positive): “…Gemini Team, 2023) heavily hinges on AI alignment approaches (Leike et al., 2018; Christian, 2020; Kenton et al., 2021; Leike & Sutskever, 2023; Ji et al., 2023)—typically a mixture of supervised Fine-tuning (SFT) (Wei et al., 2021) and preference-based optimization methods like Reinforcement…”
- ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution (2024, neutral): “The rise of LLMs introduces new prospects for diverse fields (Xi et al., 2023; Wang et al., 2023c; Zhao et al., 2023; Ji et al., 2023; Lu et al., 2023; Zhang et al., 2023a).”
- FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts (2023, methodology): “We propose FigStep , a straightforward yet effective black-box jailbreaking algorithm against VLMs. Briefly, FigStep first embeds the harmful content into an image and then pairs it with a benign text instruction, wasting the huge previous efforts on the safety alignment of the underlying LLMs [20].”
- Safe RLHF: Safe Reinforcement Learning from Human Feedback (2023, positive): “LLMs Alignment and Safety AI alignment focuses on ensuring that AI systems, particularly LLMs, adhere to human intentions and values (Ji et al., 2023a).”
- Federation of Agents: A Semantics-Aware Communication Fabric for Large-Scale Agentic AI (2025, unknown): “5, we found it practical to set-up A-1 as a pre-aligned small language model ( ≤ 20 B params) for fast-feedback execution of multiple refinement rounds over long-term horizons [41, 47, 27].”
- Statistical inference on black-box generative models in the data kernel perspective space (2024, unknown): “The question is chosen due to its subjectiveness – there is no correct answer to the question of a person’s greatness – and its duality – Ronald A. Fisher is considered one of the most influential statisticians in history and is considered an advocate of the 20th century Eugenics movement (Bodmer…”