| CARVIEW |
PR2: A Physics- and Photo-realistic Humanoid Testbed with Pilot Study in Competition
Xiaokun Leng3, Lining Sun4,5, Jingwen Zhang1✉️ , Zhicheng He3✉️ , Yao Su1✉️ ,
✉️ indicates corresponding author
Abstract
This paper presents the development of a Physics-realistic and Photo-realistic humanoid robot testbed, PR2, to facilitate collaborative research between Embodied Artificial Intelligence (Embodied AI) and robotics. PR2 offers high-quality scene rendering and robot dynamic simulation, enabling (i) the creation of diverse scenes using various digital assets, (ii) the integration of advanced perception or foundation models, and (iii) the implementation of planning and control algorithms for dynamic humanoid robot behaviors based on environmental feedback. The beta version of PR2 has been deployed for the simulation track of a nationwide full-size humanoid robot competition for college students, attracting 137 teams and over 400 participants within four months. This competition covered traditional tasks in bipedal walking, as well as novel challenges in loco-manipulation and language-instruction-based object search, marking a first for public college robotics competitions. A retrospective analysis of the competition suggests that future events should emphasize the integration of locomotion with manipulation and perception. By making the PR2 testbed publicly, we aim to further advance education and training in humanoid robotics.
Competition Trials
To comprehensively address humanoid locomotion, manipulation, and the integration of AI techniques while maintaining a moderate level of difficulty, we designed six distinct tasks; see Tab. for a detailed description.
,
and
indicate
score assignments related to humanoid robot’s locomotion, manipulation, and perception, respectively.
Try It Out
System Architecture
PR2 is built on top of NVIDIA Isaac Sim , which leverages advanced GPU-enabled graphics and physics simulation with Nvidia PhysX 5, enabling users to investigate and test robotic skills in a physics-realistic and photo-realistic simulation environment. By extending the core components of NVIDIA Isaac Sim, PR2 consists of three main modules : (i) The task module and (ii) the physics simulation module support the import of various digital assets to create diverse task environments and simulate physical effects such as forces and system dynamics. Additionally, (iii) built-in controller module includes model predictive and whole-body controllers and high-level planners are provided for beginners but can be replaced by more advanced users as needed
Acknowledgments
We thank the engineering team from Leju Robotics for technical support, Ms. Zhen Chen from BIGAI for refining the figures, and all colleagues from the BIGAI TongVerse project for fruitful discussions and help on simulation developments.