Backpropagates through a frozen 4-bit quantized base model into a small set of low-rank adapters. The approach brings fine-tuning of large models down to a single consumer GPU while holding quality close to full 16-bit tuning.
@inproceedings{dettmers2023qlora,
author = {Tim Dettmers and Artidoro Pagnoni and Ari Holtzman and Luke Zettlemoyer},
title = {QLoRA: Efficient Finetuning of Quantized LLMs},
booktitle = {Advances in Neural Information Processing Systems 36 (NeurIPS 2023)},
year = {2023},
eprint = {2305.14314},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2305.14314}
}Ejentic did not author this paper. Credit belongs to the authors named above; the note is ours.
2023
Cut the RLHF pipeline down to one training run. For teams without the infrastructure to run a reward model and PPO, this is usually the practical route into preference tuning.
2021
Swappable adapters mean one base model can serve many clients. That is the operational reason this paper matters more than its accuracy numbers.