In RoboColiseum’s new-season results, Claude Fable 5.2 ranked first overall with 18.30 points. It also led the fine-manipulation and cross-embodiment-transfer tracks, with scores of 33.04 and 17.99. Dexterous-hand tasks, however, were the overall leader’s weakest category.
It ranked first in fine manipulation, so why did it score only 3.73 in dexterous-hand tasks? Hanging clothes on a rack and holding an object securely are ordinary human actions. Why are they so difficult for large models?
RoboColiseum’s new season brings practical robot tasks into the benchmark, adding four tracks—dexterous manipulation, whole-body coordination, fine manipulation and cross-embodiment transfer—with 50 tasks in total. The evaluation measures both whether a task is completed from start to finish and how efficiently it is done.
Four New TracksFocused on Realistic Long-Horizon and Complex TasksCharging a phone, hanging clothes on a hanger, picking up litter and unlocking a door with a key are all tasks robots may encounter in the real world. Though they seem ordinary, they require coordinated use of fingers, both arms and the robot’s body.
RoboColiseum has moved these practical challenges into simulation, adding four tracks and 50 tasks to test whether a model can complete an activity end to end.
Dexterous manipulation: from coordinating multiple fingers to using tools
To hang a garment, a robot must grasp and spread the fabric, pass a hanger through it, then adjust the garment and place it on a rail.
Fabric is soft and easily deformed. The robot’s fingers must coordinate their grip, and both hands must work together to complete the sequence.
This track contains 10 tasks, including wok tossing, solving a Rubik’s Cube, tying a rubbish bag and carrying a package with both hands.
 |  |
 |  |
Whole-body coordination: the base, torso and arms must work together
To pick up litter, a robot must first move into position, adjust its body and reach out to grasp it. Erasing a blackboard requires the mobile base and torso to reposition so the arm can reach and clean different areas.
The track has eight tasks, including picking up litter, setting down plates, retrieving items from a conveyor, unlocking a door and erasing a blackboard. They test whether a robot can combine movement and manipulation in a continuous sequence.
 |  |
 |  |
Fine manipulation: can the connector be inserted once it is aligned?
To charge a phone, the robot must control the phone and connector with separate arms, aligning both position and orientation. The eight task types include phone charging, block assembly, plugging in a network cable, installing a bearing and packing a phone. They test precise alignment, contact and assembly of small parts, where accuracy matters most near completion.
 |
Cross-embodiment transfer: can a model keep working in a different “body”?
The track includes six task types, such as hanging a cup, replacing a keyboard keycap, writing with a pen and pouring water. Each is tested on four robots—ARX, AgileX, G1 and G2—for 24 evaluation tasks in total.
Robots differ in joint structure, range of motion and grippers, so a model must adapt its actions accordingly. The track tests whether the same model can work across different robot embodiments.
Scores for Eight ModelsDexterous-Hand Tasks Remain a Shared WeaknessScores for eight agents and three policy models
Claude Fable 5.2 ranked first overall with 18.30 points, followed by GPT-6 Astra at 13.73 and Claude Opus 5.5 at 11.77.
Fable also led the fine-manipulation and cross-embodiment-transfer tracks, scoring 33.04 and 17.99. In whole-body coordination, Astra and Opus were nearly tied at 18.96 and 18.94.
Dexterous-hand tasks were the weakest category for all eight agents. Fable scored 3.73, while Astra and GPT-6.1 Terra scored 0.00. Gemini 4 Argon had the agents’ highest score in this track, but managed only 5.19. Coordinating multiple fingers and securely grasping and manipulating objects remain common challenges for these models.
Policy models scored higher in the dexterous-hand track. π₀.₅ led with 16.79, followed by LingBot-VLA-2.0 at 10.66 and GR00T N1.7 at 5.53. All three outscored the eight agents in this category. π₀.₅ also ranked fourth overall with 11.22 points.
Three Safeguards Against Benchmark GamingAgents and Policies Are Evaluated SeparatelyThe results point to a clear opportunity for teams with strong manipulation policies: policy models currently have an advantage on tasks that general-purpose agents still struggle to perform, and that gap could widen.
Models can score well by repeatedly practicing the same questions. RoboColiseum’s new season introduces three safeguards against benchmark gaming and privileged information, aiming to make scores a better reflection of actual model capability.
First: a hidden test set keeps the practice scenes public while reserving the official test scenes.
Developers can use open-source scenes for development and debugging. Official server-side test scenes follow the same distribution but use different layouts, testing whether a model can handle scenarios it has not practiced.
Second: dynamic spot checks use a different set of tasks to test performance.
The platform retains a fixed official test set for scoring and also creates a dynamic set that changes with each submission. The dynamic set does not count toward the total score; it is compared with the official set to detect overfitting to fixed tasks. Large gaps between the two can restrict a submission from appearing on the leaderboard.
Third: human review checks for conduct that violates the evaluation rules.
Alongside automated checks, the platform uses human review to identify rule violations and attempts to obtain scores improperly.
Agents and policy models are evaluated separately, with each approach compared against its peers.
Agents can alternate between observation, reasoning and action. Policy approaches such as VLA and WAM typically execute a sequence of actions. Because their interaction patterns and pace differ, the update gives agents a separate leaderboard so teams can compare like with like.
Execution efficiency now affects the score: how many actions does a model take to finish a task?
The new rules limit the number of times an agent can act, with thresholds informed by the training data. Repeated observations, reasoning and attempts beyond a reasonable limit incur a penalty.
96.86% Correlation Between Simulation and Real-Robot ScoresReal-World Data Tests the CorrelationTask completion also matters. The score retains credit for intermediate progress and gives a bonus for completing a task within the prescribed number of steps. The final result reflects how far the model progressed, whether it finished the task and how many steps it took.
How closely do simulation results reflect real-world performance? That is the key question for simulation benchmarks. To measure this, RoboColiseum conducted large-scale comparisons: without adapting or tuning the simulation, it evaluated models trained in the real world directly in simulation in a zero-shot real-to-sim setting.
Across four robot embodiments and 22 complex tasks, the reported correlation between simulation and real-robot task scores was 96.86%.
The result suggests that simulation scores need not be merely virtual report cards. Defects found in simulation and strategies validated there may transfer to real robots with high reliability, reducing the cost of real-world testing and iteration.
Work Through an AI AssistantBring Your Model to CompeteComparison of real-robot and simulation results across 22 complex tasks
Preparing data, connecting a model, submitting an evaluation and viewing the results can all be handled through a conversation with an AI assistant.
RoboColiseum provides callable AI skills. After installing and configuring them, developers can use an AI assistant to download training data, connect inference services and launch evaluations, helping them get results sooner.
The New RoboColiseum Season Is Under WayChallenge the LeadersOnce a task is created and inference is launched, RoboColiseum manages the simulation, scene scheduling and scoring. It also provides task progress, category scores and result queries. This lets teams spend more time analyzing failures and improving their models.
Eight models have entered RoboColiseum’s embodied-AI arena. This round of evaluation shows that dexterous control, fine manipulation and cross-embodiment transfer still have room to improve. Those unresolved challenges leave opportunities for new entrants.
Four new tracks and 50 tasks are open. RoboColiseum invites teams developing policy models and agent systems to enter their models and compete with the eight leading systems. The next model to surpass them could be yours.
The new-season challenge is under way, and RoboColiseum plans to continue expanding its evaluation framework.
RoboColiseum：Enter RoboColiseum, take part in the evaluation and issue a challenge!
