Alignment & RLHF
Instruction tuning, RLHF, preference optimization, helpful/harmless assistants.
A topic in the Science corpus, drawn from 99 papers.
Trusted voices in Alignment & RLHF
Ordered by the trust Alignment & RLHF's own network concentrates on each researcher, most first.
- Christian P. Robert — trusted by 56 researchers here
- Dario Amodei — trusted by 69 researchers here
- Paul Christiano — trusted by 71 researchers here
- Jan Leike — trusted by 34 researchers here
- Shane Legg — trusted by 30 researchers here
- Tom B. Brown — trusted by 31 researchers here
- Nick Bostrom — trusted by 19 researchers here
- Angelica Chen — trusted by 23 researchers here
- Miljan Martic — trusted by 23 researchers here
- Jeff Wu — trusted by 22 researchers here
- Stuart Armstrong — trusted by 14 researchers here
- W. B. Knox — trusted by 17 researchers here
Most-cited papers in Alignment & RLHF
Ordered by the recorded citation count from the source corpus; a citation is not an endorsement.
- Training language models to follow instructions with human feedback (2022, Neural Information Processing Systems) — 22,660 citations
- Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, arXiv.org) — 17,670 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023, Neural Information Processing Systems) — 9,702 citations
- Deep Reinforcement Learning from Human Preferences (2017, Neural Information Processing Systems) — 5,646 citations
- Scaling Instruction-Finetuned Language Models (2022, Journal of machine learning research) — 4,301 citations
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback (2022, arXiv.org) — 4,254 citations
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning (2023) — 3,705 citations
- Self-Instruct: Aligning Language Models with Self-Generated Instructions (2022, Annual Meeting of the Association for Computational Linguistics) — 3,444 citations
- Constitutional AI: Harmlessness from AI Feedback (2022, arXiv.org) — 3,377 citations
- Learning to summarize from human feedback (2020) — 3,335 citations
- Fine-Tuning Language Models from Human Preferences (2019) — 2,638 citations
- LIMA: Less Is More for Alignment (2023) — 1,362 citations
- Towards a Unified View of Parameter-Efficient Transfer Learning (2021) — 1,223 citations
- A General Language Assistant as a Laboratory for Alignment (2021) — 1,175 citations
- Scaling Laws for Reward Model Overoptimization (2022) — 1,126 citations
- WizardCoder: Empowering Code Large Language Models with Evol-Instruct (2023) — 1,012 citations
- Aligning AI With Shared Human Values (2020) — 970 citations
- The Flan Collection: Designing Data and Methods for Effective Instruction Tuning (2023) — 953 citations
- AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback (2023) — 921 citations
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022) — 837 citations
- Instruction Tuning with GPT-4 (2023) — 837 citations
- Otter: A Multi-Modal Model With In-Context Instruction Tuning (2023) — 692 citations
- Improving alignment of dialogue agents via targeted human judgements (2022) — 676 citations
- Scalable agent alignment via reward modeling: a research direction (2018, arXiv.org) — 615 citations
- SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization (2019) — 613 citations
- RRHF: Rank Responses to Align Language Models with Human Feedback without tears (2023) — 545 citations
- How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources (2023) — 510 citations
- Reward learning from human preferences and demonstrations in Atari (2018) — 490 citations
- Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor (2022) — 487 citations
- Policy Shaping: Integrating Human Feedback with Reinforcement Learning (2013) — 484 citations
- Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision (2023) — 457 citations
- PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization (2023) — 400 citations
- Two Cheers for Rebooting AI: Building Artificial Intelligence We Can Trust (2020) — 364 citations
- The alignment problem from a deep learning perspective (2022) — 333 citations
- MIMIC-IT: Multi-Modal In-Context Instruction Tuning (2023) — 310 citations
- Pretraining Language Models with Human Preferences (2023) — 302 citations
- A Long Way to Go: Investigating Length Correlations in RLHF (2023) — 284 citations
- Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive (2024) — 257 citations
- Combining manual feedback with subsequent MDP reward signals for reinforcement learning (2010) — 242 citations
- Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control (2016) — 222 citations
- The Capacity for Moral Self-Correction in Large Language Models (2023) — 214 citations
- Don’t Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training (2019) — 212 citations
- Reward-rational (implicit) choice: A unifying formalism for reward learning (2020) — 208 citations
- Memory-assisted prompt editing to improve GPT-3 after deployment (2022) — 194 citations
- ImageBind-LLM: Multi-modality Instruction Tuning (2023) — 181 citations
- Supervising strong learners by amplifying weak experts (2018) — 178 citations
- Corrigibility (2015) — 177 citations
- Preference-based reinforcement learning: a formal framework and a policy iteration algorithm (2012) — 177 citations
- Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021) — 176 citations
- SVIT: Scaling up Visual Instruction Tuning (2023) — 176 citations
Most active researchers
By papers in this topic: a volume measure, not trust.
- Dario Amodei — 11 papers
- Ethan Perez — 8 papers
- Tom B. Brown — 7 papers
- Amanda Askell — 6 papers
- Angelica Chen — 6 papers
- Paul Christiano — 6 papers
- T. Henighan — 5 papers
- Yuntao Bai — 5 papers
- John Kernion — 5 papers
- Dawn Drain — 5 papers
- Kamal Ndousse — 5 papers
- Kyunghyun Cho — 5 papers