Research desk
The papers behind how we build
An annotated reading list. Every entry is credited to its original authors, with a note on what it changes once the idea meets a real system.
All entries
1 entry
- 2023TACL 2024PublishedCurated
Lost in the Middle: How Language Models Use Long Contexts
Language models attend strongly to the beginning and end of a long context and weakly to the middle. The authors show accuracy degrading in a characteristic U-shape as the position of the relevant passage moves through the input, even in models explicitly built for long contexts.
Browse by topic
Foundations
The architectures and pre-training results everything else stands on.
3 entries
Retrieval & RAG
Grounding a model in documents you actually own.
5 entries
Reasoning
Getting a model to show its work, and checking the work.
6 entries
Agents & Tool Use
Models that call tools, hold state, and finish multi-step jobs.
8 entries
Adaptation
Changing what a model does without retraining all of it.
4 entries
Alignment
Steering outputs toward what a person would accept.
6 entries
Evaluation
How you know it works before a customer finds out it does not.
11 entries