Reinforcement Learning from Human Feedback: A Tutorial at ICML 2023

Reinforcement learning from human feedback (RLHF) has dramatically improved the real-world performance and user experience of large machine learning models. Still, this approach has primarily been applied at a scale of compute and data curation that limits academic availability. In this tutorial, we will describe the general framework of RLHF and explain the technical procedures required to apply this framework. The tutorial begins with a detailed conceptual overview and continues with an explanation of human-in-the-loop data collection procedures used when scaling state-of-the-art systems.