Language Models are Few-Shot Learners

Abstract

Demonstrates that a sufficiently large language model can perform a new task from a handful of examples in the prompt, with no gradient updates. The paper established in-context learning as the default interface to a language model.

@inproceedings{brown2020language,
  author    = {Tom B. Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared Kaplan and Prafulla Dhariwal and Arvind Neelakantan and others},
  title     = {Language Models are Few-Shot Learners},
  booktitle = {Advances in Neural Information Processing Systems 33 (NeurIPS 2020)},
  year      = {2020},
  eprint    = {2005.14165},
  archivePrefix = {arXiv},
  url       = {https://arxiv.org/abs/2005.14165}
}