Adaptation

Changing what a model does without retraining all of it.

3 entries

  1. 2023NeurIPS 2023PublishedCurated

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model

    Rafailov et al.

    Reformulates preference learning so the reward model can be folded into the policy in closed form. The result is a single supervised objective over preference pairs, with no separate reward model and no reinforcement learning loop.

    AlignmentAdaptation
    arXiv
  2. 2023NeurIPS 2023PublishedCurated

    QLoRA: Efficient Finetuning of Quantized LLMs

    Dettmers et al.

    Backpropagates through a frozen 4-bit quantized base model into a small set of low-rank adapters. The approach brings fine-tuning of large models down to a single consumer GPU while holding quality close to full 16-bit tuning.

    Adaptation
    arXiv
  3. 2021ICLR 2022PublishedCurated

    LoRA: Low-Rank Adaptation of Large Language Models

    Hu et al.

    Freezes the pretrained weights and injects trainable low-rank matrices into each layer instead. The number of trainable parameters drops by several orders of magnitude, adapters are small enough to swap at runtime, and training cost falls with them.

    Adaptation
    arXiv

All research