The papers behind how we build

All entries

4 entries

1. 2026ICML 2026PublishedCurated

   Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning

   Cheng et al.

   Group-based reinforcement learning methods credit whole trajectories by final outcome, which makes it hard to see the contribution of individual steps — including valuable steps buried inside failed trajectories. The authors propose Graph-based Group Policy Optimization (GraphGPO), which aggregates all rollout trajectories into a unified state-transition graph, estimates the distance from each state to the task goal using global information encoded in the graph, and assigns credit to each edge by how much the transition reduces that distance. They report improved training efficiency and state-of-the-art performance across a range of benchmarks.

   ReasoningAdaptation

   arXiv
2. 2023NeurIPS 2023PublishedCurated

   Direct Preference Optimization: Your Language Model is Secretly a Reward Model

   Rafailov et al.

   Reformulates preference learning so the reward model can be folded into the policy in closed form. The result is a single supervised objective over preference pairs, with no separate reward model and no reinforcement learning loop.

   AlignmentAdaptation

   arXiv
3. 2023NeurIPS 2023PublishedCurated

   QLoRA: Efficient Finetuning of Quantized LLMs

   Dettmers et al.

   Backpropagates through a frozen 4-bit quantized base model into a small set of low-rank adapters. The approach brings fine-tuning of large models down to a single consumer GPU while holding quality close to full 16-bit tuning.

   Adaptation

   arXiv
4. 2021ICLR 2022PublishedCurated

   LoRA: Low-Rank Adaptation of Large Language Models

   Hu et al.

   Freezes the pretrained weights and injects trainable low-rank matrices into each layer instead. The number of trainable parameters drops by several orders of magnitude, adapters are small enough to swap at runtime, and training cost falls with them.

   Adaptation

   arXiv

Browse by topic