"Before data becomes excess, junk data becomes excess".The money comes out of your pocket, goes around and goes back into your pocket.
One of the consensus in these two years of being physically intelligent is the lack of data, but ten of them are collecting data and selling data sets, and nine of them are selling mostly with us.
In the majority of routes, remote control is basically considered the most practical route available today.
From real-life remote control, to UMI's direct recording of actions with a claw in the hand, to moving gloves and extra bones; and to the two-year-old ego route: simply not remotely controlling robots, directly recording people's actions, and then mapping people's actions to robots.
The lighter the equipment, the lower the threshold, the more people can participate in the collection, the higher the limit on data capacity.
However, this path leads to an irreversible trade-off: data is increasingly distant from the shape and dynamics of the robot itself.
During this time, we interacted with many teams and noticed a new trend:
The full-scene camera, which was originally rooted in video, VR and full-scene photography, is being moved to the race track where physical data is being collected.
Current industry discussions focus largely on narrow-sighted Ego, hand-held claw collection, and the overall route is still in the early stages of testing, with little consensus.
We do not intend to argue that this route is better, nor do we believe that it will solve the core problem of mapping.
It's a parallel alternate path, with its applicable boundaries, and a bunch of engineering shutdowns.
This article is still the old rule, and we mainly start from the evolution of the digital pathways, the way in which the landscape data is processed, and the chain of technologies such as the depth of the landscape, 3D reconstruction and simulation, to consider the current progress in digital cameras.
Welcome to the world of deep blue AI.We will continue to share the dynamics at the forefront of AIDeep blue AI1—Numerical routes evolve: human-to-robot dataIf you're going to train a robot to pick up a desk, the most ideal data comes naturally from the robot itself. The machine arm sees the cup on the table, reaches out, grabs, moves, and then puts the cup in a designated position. Throughout the process, the camera directly records what it sees, how the machine arm moves, when the claw closes, and even the position of the joints and the trajectory of the end executor.
For robot learning, this kind of data is very clean. The problem is that it's not very scalable.
DROID is a typical example.
In order to extend the number of robots to a more realistic environment, the research team deployed a unified Franka mechanical arm, camera and remote operating system in 13 institutions. In 12 months, 50 collectors obtained about 76,000 tracks, 350 hours of interactive data, covering 564 scenarios and 86 class missions.
350 hours is already a considerable amount of data in the field of robotics, but from a different perspective, it also means a whole set of robotic hardware, interagency deployments and a year of collection, with the latest data being played continuously for less than 15 days. The cost will soon become a bottleneck for physical models that want to cover more real-world scenarios, long-tail missions and human activity, relying on the robot itself to accumulate a bit of data.
In the last few years, an increasingly clear change has begun to emerge: researchers have begun to think of ways to replace robots with data collection.
Figure 1: DROID uses a unified Franka mechanical arm, dual cameras, wrist cameras, and VR remote control to extend robot-native computing to multiple institutions and real-world scenarios. It shows the advantages of robot native data and intuitively illustrates why it's not easy to expand at low cost.
Deep blue AI2—Re-evaluation 2: From holding a claw to first-person glassesThis route is not a step from jumping off a remote-operating robot to wearing a camera, but a bit of dismantling of the robot from the data collection site.
UMI, or Universal Manipulation Interface, is a typical step in the middle.
It uses the GoPro, IMU and a parallel claw of a robot-like end-executor as a portable device where the collector does not have to move the mechanical arm into the kitchen, office or dining room, but uses the device to complete the task directly, and then converts the recorded movement into the data required for the robot strategy.
In the cup-setting experiments given on the project page, UMI achieved 111 demonstrations per hour, while SpaceMouse remotely operated as 35 per hour; the device could start collecting soon after it reached a new environment. It changed not an algorithm module, but the way the data was generated: robots no longer had to appear on the scene of each demonstration.
Figure 2: UMI uses a number of interfaces as a portable hand claw. Instead of having to deploy the robot to the target environment, the collector can go directly to a home, a restaurant, etc. to complete the presentation and then turn the human movement into the data needed for the robot strategy.
DexCap has also taken the collection equipment further to the monkeys.
It uses motion capture cameras and gloves to record human palms, finger movements, and project reports with a collection throughput that is about three times that of traditional remote operations and approaches the speed of natural human movements. To EgoMimic, the device has even gone further into first-person spectacles such as Project Aria: the system builds human demonstrations from human first-person video and 3D hand-held trajectories, along with robot data for co-training strategies.
Putting these work together, we'll see a very clear evolution: robot teleoperation → portable interface → instrumented human → natural human experience.
Of course, this change has also led to new exchanges. When robots themselves collect data, the movements oversee natural alignment with the robot's embodiment, joints, claws, and endpoints can be read directly; after adult adoption, environmental coverage and collection efficiency are improved, but the robot needs to do more by visual, motion capture, gesture recovery or retargeting, re-translating it into what the robot should do.
As a result, there is a rebalancing of three things: quality oversight, real-world coverage, and scaling-up. As more and more movements can recover from human behaviour, another less prominent bottleneck has begun to emerge:
If a person only needs to wear a camera to do a task naturally, how much of that experience does the camera leave behind?
Deep blue AI3—Now, the first point of view trade-off?Suppose a person clears a shelf in a convenience store. He looks to the right of the shelf, picks up a bottle of drinks and turns to put it in the trailer behind him; then takes a few steps to the left and finds new merchandise from another shelf.
It's pretty close to what a robot might see in the future. But there's a very basic problem with a first-person camera:
Where the camera sees is a data choice in itself.
When people look down at the goods, the distant shelves may have been painted; when people turn around and drive the car, the operating position is just behind them. To clear their hands, the camera may lose a wider range of scenes; to see the whole room, the local operation may become too small. What is more troubling is that this loss occurs during collection.
Figure 3: One of the key values of full-screen recording is that the decision on where to look is left to the data collection. UC Berkeley's EyeRobot uses 360-degree video to preserve the entire visual environment and then generate camera views in different directions from the same recording. In other words, a full-screen recording can accommodate many subsequent observations rather than a permanent fixed field of view at the time of filming. © Deep Blue AI Compilation
This is the most interesting place for a full-screen camera to go after a body image.
Putting in consumer photography, landscape recording is like "before shooting, after taking"; putting in body data is even more meaningful: the same original recording, if used for operational learning in the future, can focus on cutting out hands and target objects; if used for navigation or spatial understanding, it can preserve a more complete environmental structure; and when new training targets appear, there is also the opportunity to return to the original landscape to look for evidence.
In other words, ordinary video often determines observation when it is collected, while full-screen recording attempts to preserve a visual scene that can be retrieved after collection. This is where it really differs from ordinary first-person view data.
Deep blue AI4—Full-screen video becomes a training signal, where are we now?But capturing the entire room is just the first step. A more critical question for physical intelligence is: how do you turn a "human-activated video" into a training signal that a model can actually consume?
Currently, this processing chain has been systematically described in a few projects such as Ego360, PhysBrain 1.0/1.5.
Processing chains first solve the time problem: synchronizing full-screen video with voice, then processing with voice activity detection, automatic speech recognition and word-level timing, and using natural language as a clue to the task.
What follows are the scenarios and interactions: the system organizes objects, regions, visible text, spatial relationships and state changes from the mission snippets, and forms the Ego-Element Graph, structured caption and QA data. The data no longer merely answers what's wrong with the slide, but also begins to describe the direction of the target, how the objects are relative to each other, when contact occurs, and how the state changes with movement.
Figure 4. Human first-person video needs to be further processed into machine-learning structured supervision before it can be integrated into the body model. PhysBrain 1.0 organizes the scene elements, spatial changes, action execution and depth relationships in video into physical understanding data, showing a typical processing chain that stretches from the behavior of human recorders to the extraction of physical experiences.
Back and forth is human motion and scene geometry: it is necessary to restore 3D human motion from a mission-related perspective after a shift, representing the retention of multiple body joints and arm joints, and using time-lapse camera poses to convert motion to a unified world coordinate. Scene reconstruction re-uses complete panoramic observation, estimating episode-level camera trajectories and sustained scene structures.
Figure 5: Human-as-Humanoid uses synchronous first and third perspective video to restore human movement, and then converts it into a 60-DoF motion tag that a robot controller can use directly, allowing human demonstration to be further transformed into observation-action supervision that can be used for VLA training.
This allowed the monkey to open the fridge and take the objects out of the fridge and walk back to the table. The monkey was no longer just a few short clips cut from the viewing area, but had the opportunity to retrieve the same sequence of activities.
It is here that the meaning of panoramic data has shifted from a broader perspective to a more complete perspective: for large-scale human video, the truly valuable unit of data may not be just an action trajectory, but a whole physical experience that includes environments, intentions, human movement, object state changes and long-range spatial relationships.
The model side has also begun to experiment with this form of data. PhysBrain 1.5 released in September 2026 states in its technical report and project page that its physical-aware pre-training embodied supervision comes from human interaction videos, which include egocentric, synchronous ego-exocentric and panoramic recordings; subsequent supervised fine-tuning and a blend of human demonstrations, real-robot trajectories and simulated experience.
This created a very interesting data division: first, larger-scale human interaction video learning space, interaction and state changes, then more direct robot-native data to complement and embodiment.
At this point, the value of a panoramic camera is not only the shooting tool itself, but also its ability to provide a sufficiently complete original record of this larger-scale human-physical experience.
Deep blue AI5—Technological chains supporting landscaping are under constructionIf we continue to look down on this issue, we'll find that the panoramic camera needs to go through several layers of transformation before it actually enters the body system: the camera needs to be able to be called by the system first, the RGB needs to go further into depth and geometry, and the real scene may need to be reconstructed into a computable three-dimensional asset.
There are some specific developments in these layers at present, and here is an example of the public work done by consumer-grade full-screen camera manufacturer Insta360.
At the bottom is the engineering interface. Insta360's Desktop Camera SDK already supports Windows and Ubuntu 22.04, Linux covers x86-64 and AArch64, and provides camera connectivity, parameter control, photographic video, real-time preview, and file management. In August 2026, the SDK update also added tools to analyze metadata from the end of the file, IMU, exposure time, etc. For robots or edge devices, this means that consumer-grade full-scene cameras are starting to have basic engineering conditions for integrated system sensors.
The next step is geography. Insta360 Research in CVPR 2026's DAP ((Depth Any Panoramas) studies how to recover metric depth directly from panoramic images of different scenarios. RGB records the appearance, while depth provides further distance and spatial measurements, which is a key step in moving panoramic images from a crystalline visual material to a crystalline spatial data set.
Figure 6. Insta360 Research's DAP estimates metric depth from the RGB of the landscape. This kind of work shows how landscape imaging extends from visual recording to spatial geometry with scale. © Deep Blue AI
This is a large-scale reconstruction. PanoLOG pushes the problem to a large-scale 3D Gaussian Splatting reconstruction. The panorama covers the entire 360° field of view at once, which can reduce the number of views that large scenes need to collect, while also causing optimization problems that are different from ordinary camera vision. AirSim360 complements another type of data from the opposite direction: it builds a full-scale drone simulation environment based on Unreal Engine 5 and synchronizes RGB, real-world depth, semantic and instance segmentation, with over 120,000 tons of open Omni360-scene.In April 2026, Insta360 announced a partnership with Splatica, allowing users to create 3D Gaussian Splatting scenes with short 360-degree video and export them to Unity, Unreal Engine, Isaac Sim and Omniverse.
Figure 7. Insta360 and Splatica's published workflow attempts to convert short 360-degree landscape recordings into 3D Gaussian Splatting scenes and then import them into multiple real-time and simulation platforms.
The real concern is not that a company is "cross-border robot", but that the panorama data along the "collection of images → restoration of geometry → reconstruction of space → organizational training data" chain is being supplemented a little bit.
But it's one thing to replenish, and it's another to be widely adopted by the robotics team.
Deep blue AI6—Why is it not enough to see the whole room?The advantages of a full-screen camera are intuitive, but it also has one problem that is easily overlooked: seeing the whole environment is not the same as seeing all the action.
A sampling assessment published by the Ego360 team showed that in eight separate episodes, 357 events, the task context was 100% visible, with at least one hand visible at 88.0%, the target object was 90.6%, the interaction visible at 83.4%, and the complete-event recall at 82.7%; but the full torso visible was only 4.4%.
This is the opposite of what a full-screen camera does and doesn't do.
Worn cameras can keep the surroundings almost complete, but the human body still obscures itself: critical movements may still disappear when the hands are crossed, the body is turned, or the object is taken away. Therefore, the main solution to 360° vision is spatial context coverage, with precise movements, touch and force still requiring additional sensors.
This also makes a more realistic numerical model clear: a panoramic camera can take on the context layer, recording where the movement is happening, what's going on around it, where the target is coming from, and how a local operation embeds longer tasks; wrist cameras, hand tracking, IMUs, and the joint state of the robot's endpoints, endpoints, force/torque or tactile continue to add precise action and contact information.
At the same time, panoramic data itself is also more difficult to process. The equirectangular projection of the spherical world, which introduces geometrical changes, non-uniform spatial sampling, and boundary continuity problems. Ego360 also lists stitching artifact, motion blur, self-shading, and practical problems such as pose and reconstruction degradation under weak textures and fast motion.
In other words, it's good at leaving behind the "world" in which an activity takes place - how the hand moves, how much contact power it has, what the robot ultimately has to do, and what it needs to do with other data sources.
Deep blue AI7—What about the heat wave at the end of "Painting Land for Prison"?What's a bottleneck with numbers? The expansion of native robot data is stagnant, and human activity data faces a new problem of "incomplete recording".
The full-screen camera is aimed at the latter: its advantage is that the decision to "see where" is postponed until the filming, so that the same recording can serve multiple subsequent uses.
Full-screen cameras excel in this layer of contextual space, with precise motion, touch and force still requiring wrist cameras, hand tracking, IMUs, and robotic sensors to complement.Context layer candidates in a mixed collection scheme。
The script was actually played over and over again.
The Internet's big model is to eat almost all the text on the web, only to find that high-quality data is already at the bottom of the list, and that the next two years are going to pay for the data quality.
The difference is that Internet data is wrong, and the cost is to recommend the wrong video; physical data is wrong, and the cost is to break a cup, throw it at a person.
More importantly, the question is downstream. The industry has not yet answered one of the simplest questions: how smart has a lot of data come in, how smart has the robot become? The success rate has moved several points; what data is attributed to it, no one can measure.
A full-screen camera is currently just a hotbed of "out-of-the-box", and the entire digital industry may be.
