DVS: Dynamic Virtual-Real Simulation Platform for Mobile Robotic Tasks

Zijie Zheng1*, Zeshun Li1*, Yunpeng Wang1*, Qinghongbing Xie1, Long Zeng1†.
1Tsinghua University *Equal contribution †Corresponding author
Paper arXiv
DVS Platform Overview

Overview of DVS Platform

DVS offers a variety of large-scale indoor scene types and dynamic element plugins on the left, enabling users to construct dynamic environments. In the middle, the platform supports various data types that can be generated, such as RGB, depth, and semantic labels. On the right, the data created using this platform can be applied to train robots for tasks such as navigation, trajectory prediction, and grasping. Through a virtual-real fusion feedback mechanism, the platform allows bidirectional mapping of the states of real and virtual agents, enriching the research scenarios.

Abstract

With the development of Embodied AI, robotic research has increasingly focused on complex tasks. Existing simulation platforms, however, are often limited to idealized environments, simple task scenarios and lack data interoperability. This restricts task decomposition and multi-task learning. Additionally, current Simulation Platforms face challenges in dynamic pedestrian modeling, scene editability, and synchronization between virtual and real assets. These limitations hinder real-world robot deployment and feedback. To address these challenges, we propose DVS (Dynamic Virtual-Real Simulation Platform), a platform for dynamic virtual-real synchronization in mobile robotic tasks. DVS integrates a random pedestrian behavior modeling plugin and large-scale, customizable indoor scenes for generating annotated training datasets. It features a optical motion capture system, synchronizing object poses and coordinates between virtual and real worlds to support dynamic task benchmarking. Experimental validation shows that DVS supports tasks such as pedestrian trajectory prediction, robot path planning, and robotic arm grasping, with potential for both simulation and real-world deployment. In this way, DVS represents more than just a versatile robotic platform; it paves the way for research in human intervention in robot execution tasks and real-time feedback algorithms in virtual-real fusion environments.

Video

Use Cases of DVS Platform

Explore different applications of our platform in robotic tasks

RGB Perception

The RGB perception from the first view of robot

Segment Perception

The Segment perception from the first view of robot

Social Navigation

Robot avoids pedestrians to navigate

VR Interation

Edit the virtual scen via VR headset

Training Data Collection

Parallel virtual-real fusion data

Real Robot Deployed

Real2Sim2Real training and real time deployment and monitoring

These demonstrations showcase different applications of the DVS platform in various robotic tasks. The platform enables realistic simulation and training for robots in dynamic environments.

Experiment

Virtual-Real Fusion Data VLA Finetune

Virtual-Real Fusion Finetune VLA Model

Performance comparison of virtual and virtua-real fusion data

Navigation Experiment

Indoor Navigation

Comparison of navigation performance in different scenes

Trajectory Prediction Experiment

Trajectory Prediction Experiment

Performance of trajectory prediction models in dynamic environments

Our experiments validate the effectiveness of DVS in various robotic tasks, demonstrating improved performance in real-world deployments.

BibTeX


      @misc{zheng2025demonstratingdvsdynamicvirtualreal,
        title={Demonstrating DVS: Dynamic Virtual-Real Simulation Platform for Mobile Robotic Tasks}, 
        author={Zijie Zheng and Zeshun Li and Yunpeng Wang and Qinghongbing Xie and Long Zeng},
        year={2025},
        eprint={2504.18944},
        archivePrefix={arXiv},
        primaryClass={cs.RO},
        url={https://arxiv.org/abs/2504.18944}, 
  }