Listening to the rain from the temple in Phnom Penh
QbitAI is public.
The training data with physical intelligence is moving from tens of thousands of hours to millions of hours.
An important source of support for this round of data expansion is ego data, i.e. human first-person video. Human operations tools, the process of completing tasks, are recorded and provide material for learning real-world operations using body models.
From the original video to the standard data set that can be used directly for model training, there is still an engineering gap in the middle. Large amounts of cleaning, marking and structured post-processing work, requiring scalable, standardized processing pipeline undertaking, are not yet complete.
This also exposes a structural contradiction in the development of physical intelligence:Much R&D is focused on innovation in model architecture and training paradigms, supporting these innovative data engineering infrastructures, but construction progress is relatively lagging behind.
As the data scale expands, the efficiency of cleaning, marking and post-processing links is increasingly directly affecting the speed of model generation.
How can we get millions of hours of data into training more efficiently while controlling the cost of processing and retrieval?
It's not.Baidu World's 2026 Forum for Physical IntelligenceAround model R&D and real-world applications, Baidu shared cutting-edge explorations and practices with industry partners. How data supports continuous model evolution and how AI infrastructure improves R&D efficiency are key topics.
Baidu Intelligent Cloud relies on the Baidu platform and continues to build a full-fledged AI infrastructure geared towards physical intelligence. It covers the R&D process from data processing to model training and deployment, and related services extend to the robotic body and application scenarios.
Vice President of Baidu GroupHo Chi Minh CityThe long-term competitiveness of physical intelligence, the co-evolution of data from real-world scenarios, models and AI infrastructure, and the engineering systems that systematically organize these capabilities.
According to Baidu, Baidu Intelligent Cloud currently provides full-service AI Infra services to more than 70 enterprises in the Furniture Intelligent Industry Chain and ranks first in China's Furniture Intelligent Cloud market with a market share of 29.6%.
Back to the corporate R&D process, how does Baidu help teams reduce the repetitive processing of data, shorten training waiting, and make the same inputs support more wheeled model iterations?
From model to real-world, how Baidu Intelligent Cloud's full-featured capabilities support hardware development
A body model, from training to deployment, needs to be continuously integrated with data, robotics and the operating environment. The full range of capabilities offered by the Baidu Intelligent Cloud revolves around these four aspects.
First of all, on the model side, Baudelaire is responsible for training, reasoning and computational modeling.
The architecture and training methods for body models are still evolving. When the R&D team tries new models, it requires both convenient modification of the code and control of the time and computational expense of each round of experiments.
Bacon's approach is to provide acceleration kits on a relatively flexible open-source framework, through the use of model optimization training for VLAs, world action models, and more.
As an example ofπ0.5, according to Baidu, in specific tests of single 8 cards, the optimised OpenPI PyTorch implementation reached 2.37 times the training throughput of the JAX reference implementation. This means that the model can complete more training calculations in the same time.
Computational scheduling goes through these processes. Data pre-processing and model training, with different demands on CPUs and GPUs, are integrated into a unified management, allocating the resources needed for different tasks, and reducing the waste caused by hardware configuration not matching to task needs.
For the R&D team, these optimizations ultimately relate to: how much data they have at hand, how much data they can process, and how many rounds of model rotation they can support.
On the data side, the collection, marking, management and evaluation should be connected.
There are many sources of physical data, with human first-person video recording the operation process, and remote-operating robot data containing feedback when performing the task itself. Data from different sources need to be processed accordingly in order to be used in model training.
For example, in a human first-person video, after a piece of material has been retrieved, it is necessary to cut into pieces, add semantic tags, and mark the movements of the hands according to the training requirements.
According to senior technology experts in the furniture industryHe won.Over the past year, data size growth has led to improved modeling capabilities and has been repeatedly verified. But from collection to availability, and then to real training gains, data is screened at a layered level, and the efficiency of the data pipeline directly affects the conversion rate.
Baidu separates data processing from data asset management: through algorithms, workflow compilation and computational scheduling, data from different sources is processed; while preserving the original material and processing results, recording versions and usage relationships, so that the data can still be replicated and traced in the process of constantly updating processing methods.
In the processing segment, Baidu provides hand reconstruction, camera trajectory recovery, motion cutting and marking, and other algorithms, executed through workflow organization, and prioritizes data quality checks to identify materials that are not suitable for subsequent processing. These processes can be embedded in parametric templates, and teams can adjust parameters when processing new data sets, and replicate processes that have already been modified.
Specifically, in terms of processing efficiency, Baidu disclosed that in tests for 10 episodes of EgoLife videos, Baidu optimised the location and direction of the camera from the video. With the MegaSaM program, the computing power consumption required to process the video per hour decreased from 11.36 GPU hours to 6.48 GPU hours, reducing by about 43%.
As video data continues to accumulate, this type of optimization can also reduce the computational consumption of bulk processing.
Data management helps researchers and developers record data sources and processed versions to provide a basis for subsequent reuse and effect tracking.
Evaluation results will also affect the next round of collection. Which tasks are underperforming, which operations need to add samples, need to be combined with model effect judgments to make subsequent data inputs more targeted.
On the body side, the model needs to be compatible with robot interaction and operating conditions.
When a robot enters a service scene, it often requires instructions from the receiver. Whether the voice can be understood accurately and the interaction process smoothly will affect the user experience.
In this regard, Baidu Intelligent Cloud provides voice interaction, security and end-to-end cloud collaboration support. For example, in collaboration with robotic computing platform enterprise Givea Robot, Baidu Intelligent Cloud provides voice interaction programs for AI accompanying devices that enable devices to understand user expressions and responses.
End-to-end cloud collaboration focuses on how end-to-end robotic devices work with cloud resources. Which tasks are handled on the end-to-end, which use cloud capabilities, and which require a combination of response time and on-board computing resources to arrange.
Therefore, the deployment of the model must be taken into account in conjunction with the hardware and software conditions of the system, and the security requirements must also be incorporated into the system design.
On the scene side, continue to test whether robots are able to complete tasks steadily, and whether the application experience can be replicated.
A model that has been validated in one environment and replaced by another robot or another workplace still needs to test performance. After entering the real business, companies also have to look at task success rates, man-made takeovers, and ongoing running costs.
Baidu Intelligent Cloud supports enterprises to continue to complete verification and explore replication to more scenarios through training sites and intermediate testing platforms. At the same time, it works with industry scenario resources and industry partners to help enterprises find the right business environment for application verification.
These validations will also provide a new basis for research and development. Field exposures can be used to adjust subsequent data acquisition and model training; effective methods that have been validated have the opportunity to continue to be used in future deployments.
Modeling updates require data support, deployment to adapt to physical conditions, and scenario verification to provide feedback for the next round of research and development.
How can we achieve more efficient R&D with the same investment?
The data from the training will be used repeatedly. Today the video cleaning was completed, and tomorrow it may be necessary to add a quality rating; this round of training will be followed by an updated marking method.
Each adjustment involves a question: which of the previously processed results can still be used?
The first thing that needs to be reduced is these re-processes.
The same sets of videos are often used in multiple rounds of experiments. At each adjustment of the model or training program, a portion of the labeling, screening and other work that has been done before is still applicable. To continue using these results, they need to be preserved in accordance with the original material and clearly documented how they are processed.
In the data governance pipeline that is already in place, Bacon is being usedLance broadband organizational dataEach row corresponds to a sample, and the video and its labels, ratings and other information are kept in different columns, to be easily supplemented and updated separately.
For example, the team replaced the video quality rating model to recalculate the rating, retaining the original video and the still-applicable action tag. In the next round of training, the new rating will be used to screen the video, along with the already labeled. Thus, adjustments in an experiment do not have to be processed from scratch with the entire batch of data.
This table-level reading and writing method is currently being extended to more pipelines.
To determine which results can be used, it is also necessary to know how they are produced. The platform associates the data version with the processing process, the training results, and records which batches of data were used during the training round, how it was processed, and the corresponding training performance.
Researchers and developers can therefore check whether the results already obtained meet the requirements of new experiments and select replicable parts; when the model performance changes, they can also look back at data and processing processes to find reasons. As experiments progress, the continuous accumulation of labels, ratings and processing records on raw materials can become the basis for further research and development.
For processing tasks that still need to be executed, the next step is to reduce computational consumption of unit data.
For example, the collection of ego data, which requires people to operate the camera, move and turn around, changes the image. In order to restore the spatial relationship when shooting, it is necessary to project the camera's position, direction and moving trajectory from the sequence of images.The camera position is restored.。
This type of processing requires a combination of CPU and GPU. If the CPU is busy recruiting resources and cannot get the data ready, the GPU may also have to wait, extending the processing time of the entire task.
Bacon's camera position restoration programDroid-WAccording to Baidu, multiple processing tasks running simultaneously on an 8-card server by default start 720 CPU threads, while the integrated system has only 180 logic cores. Too many threads instead slow down the processing efficiency of the integrated system.
Therefore, the team limited the number of CPU threads for each task and adjusted the resource allocation. At the same time, the team prepared the next batch of data in advance when computing the current data in the GPU, cached the replicable intermediate results, and migrated the appropriate computation to the GPU, further reducing the wait and repeat operations.
According to the results of the tests provided by Baidu, after targeted optimization,Droid-W and MegaSaM decreased the GPU time required to process video per hour by 31% and 43% respectively.
Similar video processing tasks can be completed faster, reducing GPU time.
Once the data is ready, the training itself can be quicker.
In the practice of optimizingπ0.5 of OpenPI,China University participates in RLinf research and development in a super-teamFirst, the PyTorch implementation was completed with training accuracy alignment with the JAX version. Based on this, one-by-one analysis of data loading, model computing and multi-card communication takes time.
This LIBERO training has only two camera images, while the model interface has three locations reserved. The original process will complement a blank image. Although the image will eventually be shielded, the front still needs to be transmitted, to do visual coding, and unnecessary inputs also consume time and computing power.
Baidu allows the number of inputs to be adjusted according to the actual data: if there are two paths of images, only two paths are processed, saving the ineffective transmission and calculation of blank images.
Another optimization is taking place inWhen Doca collaboratedMultiple GPUs trained a model together, requiring constant data exchange, non-communication, computing shutdowns, etc. The team therefore adjusted the parametric grouping and communication mode to allow the GPU to forward the data required for the next phase of the current computation in advance, reducing the wait.
These adjustments also yield a portion of the display that can preserve more intermediate results for subsequent use, reduce repetition and further shorten training time.
Keeping the core training configuration consistent in a single 8-card, NVLink-free test environment published by Baidu.π0.5 completes an epoch in the LIBERO training set, reducing the training time from 61.4 minutes to 26 minutes, increasing the training throughput to 2.36 times the original.
At the same time, the team also checked the training performance. According to the OpenPI Training Optimization Technical article published by 100 Centurion, in an 8-car, 1000-step control test of π0.5, the optimized version is highly consistent with the training loss curve of the baseline, with an average loss of only 0.01 per second.
For teams that need to repeatedly adjust models, validate ideas, and shorten the time required to complete the same round of training, they can see results earlier and arrange follow-up experiments.
This set of OpenPI optimization has also been absorbed into the default operating environment of the Baidu platform, which allows customers to use it in corresponding configurations, so that optimization results can be incorporated into everyday research and development.
Which customers have access to these capabilities in the R&D process?
After these optimizations enter the client's R&D processes, the solution is a series of experiments. The same sets of data are used repeatedly and the processing methods are adjusted as the model is updated.
In a client practice disclosed by Baidu, Baidu helped a furniture company to stream video storage, processing, and results management into a data pipeline, allowing teams to use existing results, update parts that need to be reworked, and track the data used in each training round.
The company has accumulated dozens of TB of human first-person video, hoping to recover the operation of the originator and organize it into training data that can be used by robots. The original video is collected and uploaded in batches, managed together with information such as mission descriptions, and further processed by the Baidu workflow organization.
The process consists of eight nodes, three of which require a GPU model. The hand movement involves three-dimensional reconstruction, motion computation and trajectory processing, and ultimately data extraction as required by training.
The process will also be adjusted as research and development progresses. For example, after replacing the rebuilding model, the team will need to compare the new results to the old ones and re-enact the affected follow-up steps. Original video and other still valid processing results can continue to be used.
The results of each step, such as gestures and hand tracks, are associated with the original video, while recording the models and parameters used.
After adjusting the processing method, the customer team can check the new results accordingly, run only the affected steps, and then use the updated data for training, reducing the entire batch of data from scratch.
In addition to data preparation, Baidu Intelligent Cloud also works with customers around the world.Model training, rational deployment and voice interactionIt's been a long time coming.
In a collaboration with Baidu, Asteroid is focusing on the development of new technologies.Fast-WAM world action modelIt's the most effective way of thinking.
These models are used to generate follow-up movements based on the images and task instructions seen by the robot. Fast-WAM is used in training to learn the relationship between the movements and environmental changes, and to directly output the movements during actual execution to reduce computational costs.
△Fast-WAM demonstrates the task of folding a towel on the Galaxy R1 Lite robot on a starboard mapAccording to the results of the tests provided by Baidu, Fast-WAM has a delay of 87 milliseconds under the 10-step reasoning setting. This optimizes the response speed when the model is deployed, helping the R&D team shorten the wait from the model calculation to the output results.
The collaboration between KISS and the Baidu Intelligent Cloud further involves the stability of the training process.
The Spirit series, developed by Microsoft Intelligent, has physical models that require processing of multiple types of data, such as images, language, and motion trajectories, and that consume research and development time through continuous training in iteration.
In this collaborative case, Baidu disclosed that its platform can improve the duration of effective cluster training to a maximum of 10 years.98.8%Reduce interruptions and waiting in the training process; improve the efficiency of training and reasoning reported in text, respectively, in conjunction with large model training and acceleration kits30% and 60%。
These optimizations focus both on computational speed and on whether training can continue to run, allowing the input computation to be used more fully for model iteration.
Cooperation with guinea pig robots will extend support to voice communication between robots and humans.
According to Sun Xiang, director of solutions for the Baidu Intelligent Cloud Pan-Technology industry, Baidu Intelligent Cloud combines Cube 3 and Cube 5 computing chips with cube robots to create multi-modal interaction frameworks and end-to-end voice interactions for AI hardware in different locations.
Multimodal interaction frameworks combine voice recognition, large models, and voice synthesis, combining capabilities such as emotional recognition, long-term memory, and visual understanding. Developers can access culture or third-party models, and can call on content resources such as music, education, and more for AI toys and children's accompanying robots.
The end-to-end speech program receives audio directly, and combines audio, tone, and other information to understand user expressions, allowing responses to combine the speaker's emotions and tone.
In addition, at this forum, Baidu Intelligent Cloud has also signed cooperation agreements with Shenzhen, Submarine Technologies, and Purdue Technologies to further expand industrial cooperation.
For enterprises, each round of R&D input needs to be the basis for the next generation as much as possible. Good processed data can continue to be used, processed processes can be repeatedly recalled, and field findings can provide the basis for further training.
This also echoes Lee Hsien Loong's judgments about smart organizations:Hidden knowledge, design process closureThe methods that engineers accumulate in a single experiment are gradually absorbed into tools and processes that can be used together by the team, allowing research and development experience to accumulate as the project progresses.
Smart organizations need smart clouds.Baidu Intelligent Cloud relies on platforms such as Baidu to connect data processing, model training and deployment verification, supporting enterprises to use these accumulations for further research and development, allowing the same inputs to support more iterations.
