Have you ever wondered how to train GPT-6 Astra's amazing computer-use agent (CUA) to use computers?
Researchers from Microsoft Research (Microsfot Research), UNC-Chapel Hill, Georgia Tech (Gatech), and Northeastern University (Northeastern University) presented the latest results of WildAWM.
The title of the paper is Agent World Model in the Wild: Computer-Use Agent Environment Synthesis.
It's the first time I've seen this.
The main page of the project:https://msr-orchard.github.io/WildAWM/
Open source code:https://github.com/MSR-Orchard/WildAWM
Open source environments, tasks and models:https://huggingface.co/collections/ChilleD/agent-world-model-in-the-wild
The simplest and most direct way to get an agent to learn how to use a computer is to allow it to interact and train in a computer environment, but the real computer environment is expensive, difficult to open on a large scale, and can pose security risks. For example, OpenAI recently disclosed that an agent learning in an online environment accidentally attacked a huggingface server.
WildAWM uses synthetic personal computer environments (shells, file systems, browsers, and common applications of MCP services such as email) as an alternative to real-world training environments. And, in order to avoid indirectly distilling suspicions about private models, WildAWM synthesizes entirely using open-source models. After synthesizing the environment, it synthesizes high-quality agent tasks, corresponding verifier verifiers, and composes {environment, task, verifeir} training samples, training only two Qwen models (Qwen3.6-35b-a3b and Qwen3.5-4b) in these synthesized environments, while all of the eight common cross-agent, claw agent, and GUI computer-use agent OOD ratings are upgraded on the list.
From Agent World Model (AWM) to WildAWM
Recently, front-end laboratories have shown that RL is a great help in improving the performance of agents. Agents need to learn to interact with each other, need to interact with the environment. Real system interaction is not realistic: the environment cannot be copied in large quantities, and the state is difficult to replace.
In February this year, the team released the Agent World Model (AWM, ICML 2026). In contrast to the world model, which often uses neural network simulation environments to predict the next state as an environment, AWM uses code and databases to synthesize executable, interactive environments, generating a total of 1,000, covering a variety of everyday scenarios, for enhanced training for agents specifically used for multi-cycle calling tools.
WildAWM is a new work in the series that uses a general-purpose computer using a computer-use agent (CUA) to synthesize an entire personal computer instead of a single tool environment.
It's a computer environment.
A real computer-use agent should be an agent that can access any interface of the computer without being limited to one interface. An agent can use CLI to perform tasks with code as efficiently as a coding model, can use a GUI, manipulate a mouse, access a keyboard to a graphical interface web page, and can access an MCP service to find convenient software services (such as email).
Websites, applications, MCP services, and local files are all on the same machine and are connected to each other. Most existing synthesis methods do this by making each application an isolated environment, such as a specifically synthesized terminal environment, such as a specifically synthesized web environment, etc.
Each environment in WildAWM is a person's computer. Synthesis starts with a fictional character: first determine the person's career, contacts, frequent websites, and a few things he is busy with, and then organize them into a day-by-day plan. Then, use a coding agent (GLM-5.2 plus OpenCode) to put the plan into the environment daily, write records in a database, put files in a folder, and do it every day after the previous day's state, and natural data is synthesized more and more each day.
Figure 1: WildAWM synthesizes the world, including GUI web, MCP, and CLI three agents common computer-use interfaces
In order to bring this world back and forth, each day's states had to be independently replayed twice, with the two results being exactly the same, and all services and websites were normal. Another agent was launched to verify that the data and plans for the day were incorrect. The entire process was composed of 36 characters, 640 world states. By the last day, there were an average of more than 170,000 database records and about 50 local documents in one world, more complex than the synthetic environment class work of the same period.
High-quality synthesis Task is another challenge.
The environment is there, and the next thing to do is to tackle the task.
Task synthesized by WildAWM is called a cross-agent interface (XAI) task, and each task must use three interfaces: GUI, MCP, and CLI.
For example, one question: Jordan has to run half a horse on February 21st, want to confirm which racetrack to buy before the race. He remembers a few spare GPS watches in his training notebook and has already fixed one down, but Jordan wants to look at the criteria for the race, namely GPS accuracy, screen blindness and budget, to rearrange. The agent first writes a python script, sortes out the candidates from his notebook and files them. Then logs into Amazon. The searched watch may have more than one, the title doesn't have a model, only describes the appearance (black rounded, dark gray lines on the side of the screen), and then looks at the merchandise.
These tasks include an average of 4.2 sub-tasks, with 3.1 interfaces to be switched, 74.5% of the tasks to be completed in the middle and return to previously used interfaces, some requiring more than 100 steps, and are long-horizon agent tasks.
Figure 2: An agent completes synthesized tasks across multiple computer-use interfaces: CLI, GUI, and MCP.
So how do you synthesize such a high-quality task, especially without relying on any closed-source front-end model:
Task Solvability, ensuring that the synthesized tasks can be completed properly;
Task Difficulty), ensuring that the synthesized task has enough difficulty, after all, most benchmarks on the market are stagnant, and we have to synthesize tasks that are difficult enough, like the difficult task of GPT-6 Astra.
Reliability of task verifier (Verifier Reliability), synthetic task has a verifier that is used to automatically determine whether an agent has completed a task, provides a reward signal for training, and must be sure to be accurate and reliable enough to give an effective agent RL reinforcement learning signal.
Figure 3: The overall process of WildAWM. On the left is the synthetic world, in the middle details the various stages of the synthetic task, and on the right is the synthetic environment, task and verifier for reinforcement learning.
WildAWM's approach is to introduce a large number of different agents in the synthesis phase to actually test these topics. The research team uses three open source models (GLM-5.2 ‒ DeepSeek-V4-Flash ‒ MiniMax-M3) with six agent harnesses, of which OpenCode ‒ Codex ‒ Claude Code is the programming agent, Nanobot ‒ OpenClaw ‒ Hermes Agent is the claw agent.
Each run is evaluated by three verification methods: the generator-written verification script, the verifier, the LLM judge, and a coding agent judge with the highest permissions, who can view the back and forth state of the environment themselves. They gradually increase the time and calculation of the test time, which should be more accurate. But in RL training, we need an efficient verifier, usually a script.
With the actual execution process trajectory of these agents, a Task has to connect to five pathways to qualify:
Conclusions of the verifier, LLM judge and agent judge must be consistent in 6 executions.
Initial unfinished: When the agent does nothing, the verifier must declare failure, so as not to appear at the beginning of the task.
At least one of the six executions should be successful.
Difficulty: If all three are successful without giving a hint, the explanation of the problem is too simple, and the difficulty needs to be modified.
Counterfacts: The big model is easy to follow. The researchers intentionally tell the agent judge the opposite conclusion and have it re-examined, if it changes the judgement, indicating that the judgement is unreliable and needs to be revised.
After these links, 341 qualifying titles were left in the final draft of 2,551 missions.
Figure 4: Synthetic task vulnerabilities that filter out a lot of unqualified tasks every time you close them
What is the quality of these synthesized tasks?
Researchers used a combination of 24 models and frameworks (the previous three open-source models combined with the stronger closed-source model Opus-4.8, each with 6 frameworks) to do these sets of questions, each with at least one combination, and even with only three weaker open-source models in synthesis, more than 95% can be done.
Figure 5: How many synthetic tasks can be solved by each agent: Each task is solved successfully by at least one agent, but none can be solved by all agents.
It's not hard to see (Task Difficulty). When the synthesis phase gives no hint, the agent's average success rate is only 21.3%, with 273 of the 341 paths being labeled as problems. In the verification set, the GPT-6 Astra's first success rate is about Pass@1 60%, making it 8 times, while Pass@8 still has 27.3% of the problems that have not been solved. WildAWM uses an open source model entirely, but can synthesize that GPT-6 Astra is not all right. Note that each problem has been verified to be solved.)
Figure 6: Performance of various models on synthesized WildAWM environments and tasks, right showing performance under Test time scaling
Finally, look at the Verifier Reliability. In the 100 agent execution records drawn at random (including successful and unsuccessful), the Verifier script and the agent judge majority voted 95%, Cohen's κ was 0.90. A lightweight verifier script is almost identical to these heavy and expensive agent judges, which ensures that the reward used in RL training is reliable enough.
Figure 7: Assessment of consistency between verifier and agent judge
Training only in a synthetic environment
How can it be generalized to the real world?
341 paths are divided into three parts: 100 paths are used for SFT, 208 paths are used for reinforcement learning, and 33 paths are used for validation test sets. Training starts with GLM-5.2 for the 100 tracks on that part of the SFT, then runs up to 64 steps of reinforcement learning, and the training algorithm uses GRPO to train two Qwen models (Qwen 3.6-35b-a3b and Qwen 3.5-4b).
Eight OOD benchmarks that were completely missing from training were evaluated in three categories: web agents: WebArena, Online-Mind2Web, claw agents: QwenClawBench, ClawEval, GUI computer-use agents: CUA-Gym, OSWorld-Verified, OSWorld 2.0, Agents' Last Exam.
Figure 8: Models trained by WildAWM perform on eight non-distributed benchmarks
The two Qwen models all improved on eight benchmarks. Qwen 3.6-35B-A3B's average score increased from 42.84 shillings to 53.42, increased by 10.58 points, and improved by 16.67, 16.06, 13.72 and 12.22 points, respectively, on ClawEval, OSWorld-Verified, WebArena and CUA-Gym. After training, it equaled GLM-5.2 on WebArena (67.26), and surpassed GPT-5.5 on Online-Mind2Web (5.80.77 vs. 72.22); Qwen 3.5-4B's average score increased from 21.83 shillings to 30.04, increased by 8.21 points. On WildAWM's own 33 road test sets, 35B-A3B's pass rate has been mentioned by 5.68% and GLM-5.98%, and the GLM-5.2's score by 35.2x (202.22) is well below 37.88%.
Figure 9: After training in the synthetic world alone, Qwen 3.6-35B-A3B increases on each benchmark.
In order to explore what agent capabilities contribute to the synthetic environment of WildAWM, the authors conducted a more in-depth study, selecting two important studies:
Each time a researcher removes one of the melting experiments and re-synthesizes the SFT data to train Qwen 3.5-4B, the average score drops to between 2.60 and 9.30 points.
Figure 10: WildAWM Task synthesizes the results of the five-way fusion experiment
What changes did RL reinforcement training bring to the agent?
The ratio of switching interfaces between adjacent tool calls increased from 18.4% to 25.5%, closer to GLM-5.2. This indicates that agents are more skilled at using different computer interfaces and can switch to the most suitable interfaces for completing tasks for different needs.
RL enables the model to do longer tasks: in this 40 to 120 step section, only the SFT model solves more than 9.7 percentage points of the problem, and after complete training the +RL model solves more than 17.0 percentage points.
RL cannot give the model the ability to go beyond the Teacher model itself: to divide tasks into multiple barrels according to their difficulty, it can be found that even after RL training, the agent can only be promoted in easy, medium, hard, and not in the hardest problem that teacher model GLM-5.2 does not do. This finding is also consistent with current mainstream research.
Figure 11: Behavioral analysis of reinforced learning agents
It's all open source.
WildAWM's synthetic streamlines, 341 environments and missions, two well-trained Qwen models, and a well-organized data set in the Harbor format are all open source. From creating environments, questions to judgments, the entire process uses only open source models, not relying on any closed-source front-end models, which can be used directly or reproduced.
The team believes that this paves the way for an open-source model: self-synthesis can lead to continuously expanding environments and high-quality tasks in which to learn, iterate, and eventually become a powerful general-purpose computer using a computer-use agent, with the entire process not relying on any closed-source model and avoiding indirect distillation. This laid the foundation for Recursive Self-Improvement (RSI) for open-source models.
