Changing the basic unit of evolution of the RSI from task to capability is expected to move from local optimization for specific tasks to continuous evolution across tasks, across models.
Recently, Shenzhen-based Shenzhen Infinite Returns Intelligent Technology Ltd. has proposed a full-scale mission to return to the self-improvement framework The Trinity, with the ability to evolve as a basic unit, through the collaboration of the three characters Creator, Explorer and Evolver, to promote environmental generation and synergy with Harness.
The framework first classifies the task as a Task Tree, then analyzes each task based on the test trajectory, converts the Task Tree into a Capability Tree based on the ability, and eventually evolves along the Capability Tree into a Harness Tree.
After four rounds of RSI, The Trinity improved on average scores of 34.1 and 26.2 points on 28 training benchmarks and 18 unseen benchmarks, respectively, and initially demonstrated the potential to migrate to real robot missions.
This work provides a new exploratory pathway for Agent's continuous evolution from mission-driven local optimization to capability-driven cross-tasking, cross-modal.
The results of the experiment
1. Multimodal tasks: four rounds of RSI, and unseen tasks are continuously improving
In 28 cross-modal, cross-domain training Benchmarks such as ARC-AGI-3, buildarena, video mme v2, phyre, SOON, CaP-X, the average improvement was 34 points, and the average improvement was 26 points in 18 Unseen Benchmarks.
The first round brought the biggest improvement, but the three rounds that followed were still profitable and the results on unprecedented missions also improved with succession.
Of course, not every benchmark will continue to rise, with some tasks reaching the platform early, and individual tasks showing small setbacks. But overall, the system can accumulate improvements on fixed capability trees and migrate them to tasks beyond training.
2. Embodied Transfer: Can what we learn in a simulated environment be applied to real robots?
Finally, we used the evolutionary Harness in two real-life missions.
One is that the SOON:Unitree G1 needs to find the target table next to the fan in the office and go to a tactile location; the other is that the CaP-X:AlohaMini2 needs to check the cup in the back button to see if there is a square in it and report its color and location.
Both of these tasks require the intelligent body to continuously obtain environmental information during the operation and make a decision after sufficient evidence. We use Harness to evolve the capability tree to correspond with the nodes for both tasks, and we evaluate Skill by rebuilding the simulated environment and then evaluating it on real robots.
Eventually, SOON's actual mission success rate increased from 0% to 60%, and CaP-X from 0% to 60%.
One, multi-modal intelligence returns to the bottlenecks and challenges of self-improvement
Giving a task to an intelligent organism to execute, get feedback, adjust strategies, and then re-execut it is the basic idea of many current Recursive Self-Improvement (RSI) work. Establishing a feedback loop around several benchmarks can indeed lead to performance improvements by analyzing failed trajectories, continuously optimizing tips, tools, and execution processes.
But as we tried to extend this approach to multi-modal long-range interaction tasks, the relatively simple feedback loop began to face new challenges.
1. The first is the scale-building of the environment. From ARC-AGI-3, interactive games, to GUI operations, robotic navigation and operations, the observation form of different tasks, the state space, the rules of action and the conditions of success vary. It is necessary to constantly build and verify new interactive environments in order for the intelligent body to continue to be exposed to more diverse tasks.
2. The Harness generation also faces similar problems. The performance of intelligent agents is not only dependent on the model itself, but is also increasingly influenced by cue words, tool configuration and execution processes. As tasks become more complex and diverse, the speed of artificial design and optimization of Harness makes it difficult to keep up with the changing needs of tasks.
A natural thought is this: Since there are bottlenecks in creating an AI environment and optimizing Harness, can we automate both of these processes?
We have proposed a plan for the evolution of the environment in sync with the intelligent body, which does mitigate the problem of scale, but it also brings a deeper level of thinking.
Suppose a system is able to automatically generate a hundred tasks and evolve better Harness for each task. In the end, it may obtain a hundred sets of optimized solutions. However, is this one hundredth improvement a joint drive for system improvement, or is it just a hundred independent local optimizations?
There are two interrelated issues here.
1. The way in which experience is organized. If experience is always accumulated around a single task, optimization results are easily confined to specific environments and difficult to migrate to other tasks; but if all experiences are integrated into the same Harness, the demands between different tasks may interfere with each other rather than limit performance.
2. Ways to evolve improvements. Even if better strategies can be produced in each round, the system still needs a mechanism that allows improvements to be retained, inherited and replicated in subsequent tasks, and gradually adjusted and differentiated with different capability requirements, rather than optimizing from scratch each time.
Thus, an increase in the number of tasks does not mean that improvements can continue to accumulate; an increase in local performance does not necessarily lead to an increase in cross-tasking capabilities.
What is really worth thinking about is: can we find a way to organize beyond specific tasks, to interconnect the experience gained in different tasks, so that one improvement can be the basis for subsequent improvements, and continue to evolve in successive iterations?
This is the starting point for our rethinking of the RSI drive.
Two mechanisms: co-evolution and sharing
We prefer to understand the development of multi-modal intelligence as a human-led collaborative evolution of Agent and World.
World provides tasks and feedback, Agent learns how to solve problems through interaction. In the past, this cycle was largely done by humans: people chose tasks, built environments, and modified intelligent agents based on failed trajectories.
If we want this process to continue, we need at least two mechanisms.
Mechanism 1: Agency Environment Co-evolution, enabling the environment and the intelligent body to drive each other
Through the synergistic evolution of the environment, allowing the environment to generate and adapt training challenges based on the performance of the intelligent body, allowing the intelligent body to continuously optimize Harness using environmental feedback. Both are mutually driven, reducing dependence on the artificial built environment and mutating Harness, making scalable iterative improvement possible.
Mechanism 2: Shared Capability Requirements, where different tasks may require the same underlying capabilities
Tasks may vary widely in form, shape, and purpose, but may rely on the same underlying capabilities. For example, ARC-AGI-3 and Micromouse rely on accurate perception and status tracking, although different in mission form.
This means that a task may require multiple capabilities, while the same capability may also serve multiple different tasks. If it is possible to organize experiences around these common needs, the feedback from previously decentralized tasks has the opportunity to become a shared, portable learning signal.
If we can find the needs for common abilities from different tasks, we will no longer have to consider each task as a completely independent learning goal. We can build experiences around these shared abilities and combine and specialize them according to the needs of different tasks.
This is the core starting point of the Capability-driven RSI.
Third, The Trinity: From Task-driven to Capability-driven RSI
Based on previous observations, we have proposed The Trinity, which puts environmental generation and Harness evolution into the same RSI framework.
One model in the system consists of three roles: Creator responsible for building the world, Explorer responsible for exploring the world, and Evolution responsible for analyzing feedback and guiding the improvement of both.
But the most important change here is not merely combining the three roles, but shifting the learning unit of the system from Task to Capability.
Task Tree → Capability Tree: Reorganizing experiences
First, we collected different types of multi-modal tasks, constructed a Task Tree, and had the initial Harness run on those tasks with verifiable interaction trajectories.
Evolver analyzes the capabilities required for the task and the decision-making implications of the failure, depending on the task definition and actual execution trajectory.
We then rebuilt the Capability Tree based on the need for shared capabilities, no longer organizing these experiences according to the original task categories.
The same capability can be shared by different tasks, and a task can also require multiple capabilities at the same time. Therefore, paths on a tree represent a combination of capability needs rather than a relationship of belonging to a task category.
The significance of this step is that experience that was once scattered across different benchmarks now has a common organizational basis.
Capability-Driven Real2Sim2Real: Creating training environments around capabilities
With the Capability Tree, how should the environment be created?
Traditional Real2Sim2Real often requires the creation of corresponding simulation environments for specific real-world tasks. But if our goal is not to replicate tasks, but to learn the common capabilities behind the tasks, environment design is another possibility.
For one of the nodes on the Capability Tree, the Creator will extract the decision-making mechanisms that the respective tasks depend on to generate a more concise, verifiable simulation world.
For example, if several tasks require sufficient pre-operational verification, the training environment can be designed around the decision-making process without having to completely copy all the visual details, scene structure and rules of action of the original task.
The Explorer learns in these simulation environments, while the original benchmark is used to test whether the ability to learn can migrate backwards.
The key here is not to make the simulation environment more complex, but to preserve the decision-making mechanisms that need to be learned as much as possible while reducing the complexities that are not related to it and getting cleaner evolutionary signals.
3. Harness Tree: Making improvements the starting point for the next round of improvements
Harness can be broken down into different protocol modules such as Perception, Memory, Inference, Planning, Orchestration and Loop.
In The Trinity, the Explorer evolves the Harness node-by-node along the Capability Tree. Each node can either selectively inherit the modules that the parent node has learned, or modify or add modules based on the failed trajectory of the node. Therefore, when tasks require more capabilities, the system does not have to rebuild a set of Harness from scratch, but can continue to improve on an already configured basis.
The different branches do not have to force the merger into one universal optimization. They can share existing modules while retaining modifications to their specific capability needs.
After several iterations, the modules gradually form a Harness Tree. Creators also continually improve their Harness based on environmental verification failures, and at the end of each round integrate verified modules updated for the next round of environment construction.
4. Capability-Aware Router: Different tasks, Choose different Harness
Harness Tree also raises a natural question: Since different branches have already created different specialized configurations, which one should be used when faced with new tasks?
We further introduced the Capability-Aware Harness Router, a router based on the public information of the mission, a description of the capabilities of the candidate Harness, and past verified execution experience, to select the Harness that is more suited to the current mission. It does not need to know in advance the actual capability label of the mission, nor does it need to install all the modules in the same Harness.
This is where the logic of the whole approach is complete: tasks provide capability requirements, creators generate training environments around these needs, explorers have improved based on feedback evolution modules, harness trees keep up, and routers are responsible for applying the appropriate configuration to new tasks.
Four: Experimental analysis
1. Task vs. Capability: How organizing really affects what you learn
We compared different ways of organizing experiences and the benefits of Capability-Driven Real2Sim2Real.
While Sim's initial build costs are higher, the environment can be replicated in subsequent iterations, and the reasoning costs are lower. This demonstrates that less is more can lead to better cross-task migration, as opposed to simply optimizing tasks, organizing experiences according to capabilities, and generating training environments for shared capabilities.
2. Agent as a Router: After specialization, can you choose Harness?
Harness Tree allows different branches to optimize, but this specialization is meaningless if used incorrectly. Therefore, we further compared different Harness selection methods. This also explains why we don't want to end up with an increasingly large unified Harness: different tasks can depend on different module combinations, and choice itself can be part of the learning needs of intelligent agents.
Capability and Module Relationships: Can Capability Divisions Really Guide Module Design?
We first analyzed the relationship between the 29 Modules and the six Capability classes after the fourth round of the RSI, and found that different capabilities do correspond to different module preferences, and that Capability can not only be used for organizational tasks but also provide the basis for Harness's module selection and combination.
4. Customize Capability Tree: from fixed topology to structural exploration
Previous experiments have validated the effectiveness of Capability-driven RSI, but there is a question worth exploring further: How should the Capability Tree itself be constructed?
Currently, The Trinity builds the Capability Tree based on the capability requirements of the training task and develops Harness under a fixed topology. Such a design allows us to see more clearly the benefits of capacity sharing, selective inheritance and local optimization, but there are also certain limitations: once the structure of the tree is determined, it will not be adjusted with subsequent learning dynamics.
To this end, the team further opened up an exploratory experiment supporting the custom Capability Tree, which allowed researchers to customize capability nodes, adjust tree levels and branch structures, and design simulation training environments for different nodes, thus conducting RSI experiments in different ways of organizing capabilities.
This allows us to further explore some interesting questions: which capabilities are suitable for sharing and which should evolve independently? How will different tree structures affect the inheritance and characteristics of the Harness? Should the capability tree remain stable as the task type continues to expand, or should it allow nodes to continue to divide, merge and regroup?
While there are no definitive answers to these questions, customized experiments provide a starting point for exploring them.
If the current Trinity validates the feasibility of recurring improvements along the capability tree, the next step worth exploring is to make the capability tree itself an object that can be optimized and evolved.
From continuous improvements in fixed capacity structures to the co-evolution of capacity structures with Harness, this may be the direction of further development of the Capability-driven RSI.
What drives evolution and what shapes it?
Going back to the first question. If the RSI always revolves around independent tasks, the system can easily be limited to its own local goals. Increasing the number of tasks, perhaps, can accumulate more solutions, does not mean that these experiences can be interrelated, nor does it mean that they will naturally become replicable capabilities.
Expanding the scope of evolution, optimizing version selection and seed task sets, or having the decision-maker Agent participate in the evolution itself, are all attempts to improve the search and update efficiency of the RSI.
But even if the system has the ability to improve strongly enough to record and preserve all historical experiences, a more fundamental question remains: which experiences are worth sharing? which improvements need to be preserved independently? and how should the evolution of different tasks be interrelated?
Capability-driven RSI attempts to answer these questions from a different perspective: no longer with the task itself as an optimization unit, but with the shared capacity needs behind the task as the basis for driving improvement.
The Trinity follows this idea by building environments, accumulating experiences, and developing Harness organizations under a shared capacity structure. On the one hand, by connecting different tasks through shared capacity requirements, the experience can be duplicated across tasks; on the other hand, by branching Harness evolution, maintaining the optimization direction of different capabilities, allowing improvements to be inherited, adjusted, and combined.
This organizational approach not only helps mitigate the risk of over-matching and reward hacking from task-level optimization, but also provides opportunities for local performance improvements to gradually translate into more generalized capacity building.
Papers and author information
The author explains:
The project is headed by Ph.D. student Liu Weiqiang, Ph.D. student at Xi'an University of Electronics and Technology, whose main research focuses on multi-modal models, recurring self-evolution, and physical intelligence; Ph.D. student Wang Rehm, Ph.D. student at the National University of Singapore, Zhang Zhenqiang, and Chief Executive Officer and new contributor to InfRec. The authors of the correspondence are Ph.D. student Xufei, Ph.D. student at the National University of Singapore; Ph.D. student Zhang Zhenqiang, Ph.D. student at the Chinese University of Hong Kong; and Prof. Yuyang, Professor Young, President of the National University of Singapore.
The title of the paper is: The Trinity: Capability-Driven Recursive Self-Improvement for Multimodal Agents
The code link:https://github.com/InfRec-Cardinal/The-Trinity
Link to the paper:https://arxiv.org/abs/2610.36077
The main page of the project:https://trinity-rsi.github.io/
Hugging Face Daily Paper No.1：
Authors: Weidong Tang, Ruiheng Wang, Jingxuan Zhang, Hexin Wang, Haoyuan Chen, Zhengfeiyang Zhang, Yixing Ma, Zichen Jiang, Ziyu Guo, Haodong Li, Zhengbo Jiao, Zhiqing Cui, Xinxiang Yin, Shuo Liang, Zhenglin Wan, Hongzhan Lin, Wangbo Zhao, Kaipeng Zhang, Pengfei Zhou, Odin Zhang, Yang You
Unit: InfRec, XDU, TJU, NUS, UC Berkeley, CUHK, HKUST, HKU, SII。
The author's contact:wdtang0705@gmail.com
