Toolformer: Language Models Can Teach Themselves to Use Tools

Abstract

A model decides for itself when to call an external API, inserts the calls into its own training data, and keeps only the ones that actually reduce its prediction loss. No hand-written tool demonstrations are needed.

@inproceedings{schick2023toolformer,
  author    = {Timo Schick and Jane Dwivedi-Yu and Roberto Dessi and Roberta Raileanu and Maria Lomeli and Luke Zettlemoyer and Nicola Cancedda and Thomas Scialom},
  title     = {Toolformer: Language Models Can Teach Themselves to Use Tools},
  booktitle = {Advances in Neural Information Processing Systems 36 (NeurIPS 2023)},
  year      = {2023},
  eprint    = {2302.04761},
  archivePrefix = {arXiv},
  url       = {https://arxiv.org/abs/2302.04761}
}