分享好友 Global News首页 Global News分类 切换频道

A Unified Multimodal Model Learns to Inspect and Edit Images without an External Judge

2026-11-10 23:2400RoboInd Editorial TeamWeChat Official Account

A unified multimodal model can look at a chart and draw a chart, so it can't, like a human being, check it out, find the problem, and then change it?

Researchers from the South China University of Technology, the Shanghai University of Transportation and the University of Tokyo gave a positive answer. They proposed UMM-Reflection, a multi-cycle trajectory of reinforced learning training in the open-source unified model BAGEL: the same model writes diagnoses, repaints diagnoses, and does not require any external punters when reasoning.

After reinforcement, the model improved the repair success rate for initial error images from 20.6% to 64.9%, and GenEval from 0.72 (reflect SFT) to 0.84, where space positioning jumped from 0.47 to 0.89; up 11 points on a WISE never seen before in training, nearly 20 points higher than reinforcement training done directly to the generator.

A look at the core results

I can see and I can draw, but I can't change my own.

As text instruction becomes increasingly complex, images generated once often carry errors: the position relation is reversed, the number is incorrect, the color binds the object incorrectly. Traditional text graphics models do not detect these errors and cannot be corrected. Unified Multimodal Models (UMM) put visual understanding and image generation into the same network, theoretically "picture graphics models can also see the image, so it is possible to reproduce the self-correcting loop of errors common in large language models: checking, diagnosing, modifying".

Existing work progresses along roughly two routes. One route uses reinforcement learning in single generations, such as pre-writing text planning and mapping, or using the model itself to understand the image as a reward, but the model never returns to modify the already drawn image. The other route allows the model to continue to reason and edit on the middle image, but relies mainly on the supervision of synthetic trajectory simulation (SFT).

The question for researchers is: What does simulation teach the model, and what does it lack?

Second, key observation: SFTs learn to "change", but they don't.

The researchers first constructed 29,529 multi-cycle reflection paths to reflect SFTs on BAGEL. The results showed that SFTs learned a lot more than formatting: 95% of the paths adhered to the reflection protocol, and 78% of the images had at least one correct correction path when training 16 samples of initial errors within the model's capabilities.

The problem lies in reliability. Looking only at single trajectories, the SFT model can correct only 20.6% of the initial error images, with three rounds of reflection on GenEval resulting in a total of only about 2 points of improvement, and essentially stagnating after the first round. As shown in Figure 1 to the left, modifications to the SFT are often reversed in the original location: first move the belt, then move the dog, then move the belt back, and in the end are still wrong.

图片

Figure 1: Under the same hints as the random seed, the reflection trajectory of SFT and RL contrasts. SFTs have been able to write meaningful modifications, but the single trajectory is often rounded; the entire trajectory RL allows the strategy to focus on the modification path that can reach the "correct image region".

This means that the signal missing from "will change" to "will change right" is a result-driven signal: once a reflection is useful, it is necessary to wait for the triggered image to be generated, even after a few rounds to find out.

Three: In the same model, reinforcement learning is done using the entire trajectory.

Researchers call this multi-stage "check-and-diagnose-and-modify" behavior by the unified model itself native reflection. Since the diagnostic text and the modified images come from the same set of parameters (text headers are responsible for writing the diagnosis, flow headers are re-rendered according to the diagnosis), the same result reward can update both along the same trajectory simultaneously.

Stage 1: Reflect on the cold start of the SFT.

Each round, the model observes the original request, all previous graphical history and current images, output structured reflections (Thinking / ACTION / EDIT), chooses to continue editing or stop. The training data is synthesized by an external model: GPT-5.5 as the critic, Qwen-Image is responsible for generating and editing with Qwen-Image-Edit, and BAGEL itself is not involved, so as not to distill its own failed patterns back into the monitoring signal.

Stage two: Full-track reinforcement learning.

The core has two designs:

（1）Share a sample of the initial image.For each request, only one initial image is generated and separated from the computed graph and 16 complete reflection trajectories are sampled from the exact same graph.

（2）One track, one advantage.Each trajectory calculates only one result-driven advantage value while updating all of its reflective token and flow generation steps. This avoids the exponential rollout required to allocate credit in rounds and does not require additional training in value models.

图片

Figure 2: UMM-Reflection's reinforced learning process. 16 tracks share the same initial image, with each trajectory alternating to reflect and render the model itself (up to three rounds); the frozen verifier is divided into each chart and each trajectory, with group integration resulting in a trajectory-level advantage while updating the text headers and flow heads.

On the reward side, frozen validators give a rating score of 0 to 1 for each image, and some corrections can also get non-zero progress signals. The trajectory reward takes into account the final image quality, the extent of improvement in each round, and the punishment for "correcting the wrong". The validator is only used during training, and models rely entirely on themselves to make reflection and stop decisions.

Four: Experiment: Improve focus on reflection cycles and move to unseen benchmarks

All comparisons use the same BAGEL skeleton, the same pronouns, the same resolution and the same scores. The RL is only performed on 3,000 training pronouns in the six GenEval tasks, and the assessment covers all 553 GenEval official pronouns, as well as WISE, OneIG-Bench and T2I-CompBench++ that never appeared in the training.


图片Table 1: The main results under the same evaluation protocol

On GenEval, UMM-Reflection improved by 12 points compared to reflective SFTs, the most reliant of which is the spatial positioning of the modified composition (position) one improved from 0.47 to 0.89, and the color attribute binding and counting improved by 14 and 10 points respectively. It is noteworthy that the initial image accuracy of the Base, SFT and RL three is almost the same (70 x 73) and the initial generation itself does not accept the RL gradient, so the increase is from reflective secondary rather than better first image.


图片Table 2: Results of the GenEval segmentation (Article 553 Official Tip, 0x1 point)

On three benchmarks that have never been used in the training, the same checkpoint reflects that SFTs are still rising respectively: WISE +11.0, T2I-CompBench++ +4.6, OneIG +3.5. On OneIG, pure SFTs are even slightly lower than Base.

图片

Figure 3: The accuracy rate of GenEval varies with the number of reflection rounds. The three reflection rounds of SFTs add up to about +2 points and level after the first round; only the first round after the RL brings about +9 points and continues to increase until the third round.

From the point of view of expanded testing, the post-RL model's condition correction rate (the proportion of initial error images that were eventually corrected) increased from 20.6% to 64.9%. The cost was to generate the maximum of 4 images when reasoning, so the researchers also made the comparison under the same four-figure budget.

Five, the effect is shown.

The following is the UMM-Reflection results of multiple reflections on four benchmarks. Each row from left to right is followed by the initial image and the three reflections, and the model gradually corrects errors in spatial relationships, color, material, quantity, and knowledge of the world.

图片

Figure 4: Reflection trajectory on the four benchmarks: initial image → 1st, 2nd, 3rd round modification)

Six: Meltdown: It's not enough to train a generator, to train a head.

A natural question is: what would happen if the same reinforcement learning budget was spent directly on the generator? The researchers updated the RL directly with the same 1,000-step RL. It can refer to the single-generation GenEval from 71 to 76, but almost never migrates: WISE is only 54 ((Base is 55), while UMM-Reflection reaches 84 and 74 respectively.

图片Table 3: GenEval meltdown: Initial: accuracy of the first image; final: accuracy after reflection; correction rate: proportion of images that were corrected after the initial error.

Several groups of controls pointed to the same conclusion: reflecting the SFT itself yielded only two points; only the flow head or only the text head, respectively, reached only 73 and 78, and the combined optimization of the entire trajectory reached only 84. Under the same four-figure budget, selecting the best image from four samples of the direct RL model could only reach 80, forcing the SFT model to modify every round to only 72, showing that the gain did not come from "multiple drawings, but from learning which modifications were really effective".

Seven, what did RL change about the model?

To understand the source of the gain, the researchers sent all 553 trajectories to the three checkpoints of Base, SFT and RL, respectively, and trained the linear probe inside the model to read "is this chart correct". The results showed that the RL changed little in the model's internal judgement of correctness: understanding the AUC of the branch probe at all three checkpoints was between 0.80 and 0.82.

The real change is the direction of the image in this internal space. For the initial error, after three rounds of reflection, the proportion of images falling into the "correct region" increased from 34% to 62% under the RL and only from 36% to 42% under the SFT. In other words, the model would have "knowingly" its own graphs are wrong, and the RL Church would have acted accordingly: among the many modifications that SFT has been able to produce, it chose the paths that could really push the wrong image to the correct region.

图片

Figure 5: The initial error image reads the trajectory in space for the correctness of the model itself. The longitudinal axis scores the linear probe "whether validated" on the branches and generating branches respectively; the green area is the reference image validated. The left two are RL from row 0 to row 3, and the right two are SFTs.

Eight, to sum up.

This work shows that the paradigm that has been extensively verified in the post-training large-language model of "SFT cold start + complete trajectory RL" for the unified multi-module model can also migrate to native multi-phase visual self-correction: SFT enables the model to reflect on the format with meaningful modifications, while RL makes these modifications reliable. Since it does not rely on any external critics or verifiers when reasoning, the model completes its own view, diagnosis, modification, and stop in a given context.

It's worth pointing out that multiple rounds of reflection will lead to additional reasoning costs (up to 4 images generated); the RL phase rewards come from GenEval-like combination alignment verifiers, training tips cover only six GenEval tasks, and more dimensional rewards are still to be explored. The research team has open source all training and evaluation codes, RL and SFT model weights, and SFT data, hoping to provide a replicable starting point for the study of reflection and reinforcement learning on a unified model.


点赞 0
举报
收藏 0
分享 1