On 10 November, Anthropic released a new generation of the Fable 5.2 model, which once again pushes the physical action capabilities of large models to new heights.
In the ratings released by Anthropic, Fable 5.2 achieved a 64.9% accuracy in RoboDojo Sim's two-arm robot simulation rating, a significant improvement over the previous generation Fable 5.1 and significantly ahead of the GPT-6 Astra.
As of November 11, Claude Fable 5.2 is ranked second on the Robodojo Sim Agent list, behind Q0-Preview, which was listed on October 30th: mission success rate of 68.87%, aggregate score of 74.2 points. Q0-Preview is also the first comprehensive score to break 70 points since the list's launch. More important than the results are the technical pathways.
Surprisingly, the baseline model behind Q0-Preview is not the critically acclaimed GPT 6 Astra, but Claude's previous generation Opus 5.5! and Anthropic's official reporting of the Opus 5.5 is only 39.4 percent accurate. So, as the General Model begins to accelerate into the physical world, deeper questions arise: where are the physical action limits of the General Model?
This article, as a technical precursor to the Q0-Preview, summarizes the exploration of Qoo Tech on Physical Agent.
Q0-Preview The official test code is open source.
It's a great way to get started.
A question of openness: What is the nature of driving a large model into embodied manipulation?
The key point of Q0-Preview is to determine the contact position and contact posture at each stage of the operation.
The simple task of grabbing Apple, for example, involves identifying which parts of the target object to grab, from which direction the claw is approaching, and in which direction the grip is being held. Transforming these judgments into the target position of the mechanical arm to further generate executable actions solves the problem.
Core program: Spatial computing is physically operational
The GMM already has excellent multi-mode perception and programming capabilities, but whether and how these capabilities can be fully exploited is a new open question for physical control.
Returning to the question we just posed, control itself predicts contact information and contact gestures, and gestures themselves can also be determined by multiple points, so multi-modal intelligences can act as a bridge in which they perceive multiple pixel points by computing the contact information at coordinates in space.
Therefore, Q0-Preview proposes to make full use of the model's spatial perception and programming capabilities, i.e. by identifying the location of the target pixel with a single/multi-faceted observation, in conjunction with the camera's internal and external pixel coordinate calculation tools for spatial computation, to position the target pixel in an image in contact with the world coordinate system.
Specifically, for the claw, the model can express the claw's target position and contact information through three spatial reference points, i.e. the middle of two ordered points determines the reference contact position, the line determines the holding direction, and the third point complements the approximate direction. Thus, the representation of position and gesture and contact information are unified into a computable spatial computation problem.
The multi-modal intelligent body determines the subtasks of the current phase based on observation and instruction, and submits the target position gesture and claw instruction. Movement planning components such as the downstream cuRobo convert it into continuous action. After execution, the multi-modal intelligent body examines and adjusts the next step based on the results of new observations, forming a claw observation, calculating a claw, executing a claw feedback loop.
Harness: Make the big model real
I'm going from a chicken to a chicken to a chicken.
On the basis of these core concepts, Q0-Preview builds a Harness set that specifies the interaction between models and observations, tools, and execution modules.
1. Clearly implement boundaries and call tools
Unified models are available for observation, robotic status and prior information, budget-conforming thinking and tool-calling, and the output format of control instructions. Tasks are advanced by short-action group, setting visual checkpoints at the necessary intermediate stage, and continuing or reprogramming depending on the execution results.
2. Accumulate general operational experience by simulating trial and error
In the development phase, through simulation trials and errors, a summary of commonly available model reference operating rules. For example, using the wrist camera of a vacuum cleaner to assist observation, reduce overshadowing; planning paths in conjunction with collision avoidance and follow-up visibility; adjusting motion speed according to the grabbing, transportation and placement phases, etc. These skills can be learned in simulation to migrate to the machine.
3. Complementing the requirements of the task and clearly fulfilling the conditions
For some tasks, Q0-Preview complements static task rules, completing conditions and necessary operational recommendations, reducing the deviation between model understanding and evaluation requirements. No real-time rewards or environmental judgment signals from any evaluator are read during execution.
4. Manage context and task memory
Maintaining initial information, recent observations and tight task memory, recording confirmed progress, current status and follow-up objectives, and controlling the costs of context growth. At the same time, the upstream and downstream execution of decision-making by clearly isolating interfaces allows multi-phase operations to be continuously connected.
Demonstration of ability
It's worth noting that Q0-Preview has performed surprisingly well in multiple dimensions, even surpassing remote expertise in some tasks.
1. Simultaneous operation of both arms
At the same stage of the action, the target is set for the left arm, respectively. Simultaneous advancement can lead to parallel operations.
2. Cooperation of the arms
Arranging the division of arms around a common goal, coordinated execution of the dependency relationship between supporting, connecting and other actions.
3. Accurate operation
Spatial computation determines location and direction through multiple viewpoints, and uses new observations to correct biases after closure.
4. Adaptation
Repositioning targets, adjusting postures or revising operational plans depending on changes in the object's state and the outcome of execution of the action.
5. Memory
Preserving previously acquired information and the steps that have been taken to complete the task of relying on historical information in the subsequent stages of decision-making.
The real challenge of physical intelligence is just beginning.
The Q0-Preview presented by Qoo Tech demonstrates the potential of the front-end baseline model in physical operations. The baseline model is bringing a new technological turning point to the physical domain: combining semantic understanding, spatial reasoning and tool calling with key constraints in robotic operations, with the opportunity to make significant progress in a concise and direct manner.
However, there are still many serious challenges to advancing the foundation model in real-world environments and missions.
The first is the efficiency and cost of reasoning. Large front-end models may consume millions of tokens and hours of reasoning time to complete seemingly simple physical operations. Such time and economic costs and the distance from the requirements of actual deployment remain marked. How to reduce redundant reasoning and tool calls while maintaining operational capability, and improve decision-making efficiency, is a problem that the physical agent must solve.
GPT 6 Astra is a thunderstorm in the field of bodybuilding, where the outside world has speculated that the training data of the GPT 6 Astra has added a huge amount of body EGO data, which has led to a leap in spatial understanding and bodybuilding. However, the practice of Q0-Preview based on Opus 5.5 has led us to think further: who is contributing significantly to the physical performance of the large model, code generation tools, calls and even three-dimensional generation?
In addition, longer delays in reasoning also limit the ability of the current Agent to adapt to dynamic environments. In the real world, objects and environments do not wait for the model to think and observe changes between executions that may cause the original plan to fail. Continuous exploration of how to respond to dynamic changes in a timely manner, combine vision with power, touch feedback, and respond to disruption, restore failure, and maintain operational reliability in long-term operation is still needed.
Although these challenges remain to be addressed, the zero-sample generalization capabilities of the foundation model have led to new technological pathways for physical intelligence. The exploration of Q0-Preview shows us that the combination of general model capabilities with knowledge and constraints in the physical realm is expected to continue to expand the mission boundaries of robots. The possibilities opened by the foundation model are foreshadowing the arrival of a new turning point in physical intelligence!
The team will be releasing a recent technical report with a new version of code that is cleaned up faster, adding technical details, showcasing more real-world experiments, and discussing further the boundaries and future direction of current approaches.
