Alignment

Steering outputs toward what a person would accept.

3 entries

  1. 2023NeurIPS 2023PublishedCurated

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model

    Rafailov et al.

    Reformulates preference learning so the reward model can be folded into the policy in closed form. The result is a single supervised objective over preference pairs, with no separate reward model and no reinforcement learning loop.

    AlignmentAdaptation
    arXiv
  2. 2022PreprintCurated

    Constitutional AI: Harmlessness from AI Feedback

    Bai et al.

    Trains a model to critique and revise its own outputs against a written set of principles, then learns from the revised outputs. The aim is a helpful assistant that can explain why it declined, rather than one that simply refuses.

    Alignment
    arXiv
  3. 2022NeurIPS 2022PublishedCurated

    Training Language Models to Follow Instructions with Human Feedback

    Ouyang et al.

    Fine-tunes language models on human demonstrations and then on human preference comparisons collected through reinforcement learning. The paper is the origin of the RLHF pipeline that produced the first widely used instruction-following assistants.

    Alignment
    arXiv

All research