Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)28 January 2022
Abstract
Shows that asking a model to write out intermediate reasoning steps before answering substantially improves multi-step arithmetic, commonsense and symbolic reasoning, and that the behaviour appears without any fine-tuning at sufficient model scale.
@inproceedings{wei2022chainofthought,
author = {Jason Wei and Xuezhi Wang and Dale Schuurmans and Maarten Bosma and Brian Ichter and Fei Xia and Ed Chi and Quoc Le and Denny Zhou},
title = {Chain-of-Thought Prompting Elicits Reasoning in Large Language Models},
booktitle = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
year = {2022},
eprint = {2201.11903},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2201.11903}
}
Ejentic did not author this paper. Credit belongs to the authors named above; the note is ours.
The interesting part is the failure it fixes: a good step inside a bad run gets no credit. Anyone who has debugged an agent by reading successful traces only has already lost the most informative data, which lives in the runs that failed at step nine of twelve.
More tokens and more latency per answer, so it earns its place only where a wrong answer is expensive. We do not reach for it on classification or extraction.