2020
Language Models are Few-Shot Learners
Where prompt engineering starts. The limitation matters as much as the result: few-shot performance is sensitive to the examples you pick and to their order.
Introduces the transformer: a sequence model built entirely from attention, with no recurrence and no convolution. Because every position is computed in parallel, the architecture scales to far larger training runs than the recurrent models it replaced.
@inproceedings{vaswani2017attention,
author = {Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin},
title = {Attention Is All You Need},
booktitle = {Advances in Neural Information Processing Systems 30 (NeurIPS 2017)},
year = {2017},
eprint = {1706.03762},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1706.03762}
}Ejentic did not author this paper. Credit belongs to the authors named above; the note is ours.
2020
Where prompt engineering starts. The limitation matters as much as the result: few-shot performance is sensitive to the examples you pick and to their order.
2018
Still the right answer for classification, extraction and embedding at small scale. Not every problem needs a generative model.