DPO
Also: direct preference optimisation
Training directly on pairs of preferred and rejected responses with a classification-style loss, skipping the separate reward model and the reinforcement learning loop. Far simpler to run, and competitive for most preference tuning.
A definition is not an understanding. This is taught properly in Preference tuning with DPO, hands on, part of Alignment & RLHF.
Learn this properlyTerms from the same topic