International Conference on Machine Learning (ICML) 202610 February 2026
Abstract
Proposes Agent World Model (AWM), a fully synthetic environment generation pipeline scaled to 1,000 environments covering everyday scenarios, in which agents interact with rich toolsets and obtain high-quality observations. The environments are code-driven and backed by databases rather than simulated by an LLM, which the authors argue gives more reliable and consistent state transitions than LLM-simulated environments, and enables reliable reward functions because the database state is accessible. Experiments on three benchmarks show that training exclusively in synthetic environments yields strong out-of-distribution generalisation.
Training an agent inside an LLM-simulated environment means the world model and
the agent are the same kind of thing. State transitions are plausible rather
than correct, and the reward signal inherits every bias of the simulator.
AWM makes environments code-driven with a database behind them. Transitions are
executed, not generated. State is queryable.
Why the database matters more than the scale
A thousand environments is the headline but the mechanism is the point: because
the state is in a database, the reward function can read it. You are no longer
asking a judge model to decide whether the agent succeeded, you are checking
whether the row changed.
That is the difference between a reward that can be gamed by fluent output and
one that cannot.
The transfer result
Training only in synthetic environments and evaluating on real benchmarks
produced strong out-of-distribution generalisation. If that holds up, it
changes the economics of agent training — the bottleneck has been environment
availability, and this treats environment generation as the thing you scale
rather than the thing you scavenge.
BibTeX
@inproceedings{wang2026agent,
author = {Zhaoyang Wang and Canwen Xu and Boyi Liu and Yite Wang and Siwei Han and Zhewei Yao and Huaxiu Yao and Yuxiong He},
title = {Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning},
booktitle = {International Conference on Machine Learning (ICML) 2026},
year = {2026},
eprint = {2602.10090},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2602.10090}
}
Ejentic did not author this paper. Credit belongs to the authors named above; the note is ours.
Read this before you let an agent call anything a customer will see the output of. The failure mode is not a crash, it is an API returning 200 with a body that means "I did not understand you". There is no exception to catch and no status code to branch on. The result we quote most often is the last one: the fix was one line of schema, not a better model.
This is the exact shape of a deployed support or negotiation agent: it works for the business and talks to someone who does not. The split the benchmark finds is the one that matters commercially, because the failure is not leaking data to the counterparty, it is refusing the principal so broadly that the agent stops doing the job it was hired for.
The interesting part is the failure it fixes: a good step inside a bad run gets no credit. Anyone who has debugged an agent by reading successful traces only has already lost the most informative data, which lives in the runs that failed at step nine of twelve.