The papers behind how we build

All entries

6 entries

1. 2022PreprintCurated

   Constitutional AI: Harmlessness from AI Feedback

   Bai et al.

   Trains a model to critique and revise its own outputs against a written set of principles, then learns from the revised outputs. The aim is a helpful assistant that can explain why it declined, rather than one that simply refuses.

   Alignment

   arXiv
2. 2022PreprintCurated

   Holistic Evaluation of Language Models

   Liang et al.

   Argues that accuracy alone is a misleading summary of a language model, and evaluates a set of models across many scenarios and many metrics at once, including calibration, robustness, fairness and efficiency, so the trade-offs stay visible.

   Evaluation

   arXiv
3. 2022ICLR 2023PublishedCurated

   ReAct: Synergizing Reasoning and Acting in Language Models

   Yao et al.

   Interleaves verbal reasoning traces with tool actions, so the model thinks about what to do next, does it, then reasons about what came back. The reasoning trace also becomes an auditable record of why the agent took each step.

   Agents & Tool UseReasoning

   arXiv
4. 2022ICLR 2023PublishedCurated

   Self-Consistency Improves Chain of Thought Reasoning in Language Models

   Wang et al.

   Samples several independent reasoning paths for the same question and takes the most common answer, instead of trusting a single greedy chain. The intuition is that a correct derivation can be reached many ways while a wrong one is usually reached once.

   Reasoning

   arXiv
5. 2022NeurIPS 2022PublishedCurated

   Training Language Models to Follow Instructions with Human Feedback

   Ouyang et al.

   Fine-tunes language models on human demonstrations and then on human preference comparisons collected through reinforcement learning. The paper is the origin of the RLHF pipeline that produced the first widely used instruction-following assistants.

   Alignment

   arXiv
6. 2022NeurIPS 2022PublishedCurated

   Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

   Wei et al.

   Shows that asking a model to write out intermediate reasoning steps before answering substantially improves multi-step arithmetic, commonsense and symbolic reasoning, and that the behaviour appears without any fine-tuning at sufficient model scale.

   Reasoning

   arXiv

Browse by topic