D(R,O)\small \mathcal{D(R,O)}D(R,O) Grasp: A Unified Representation of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping (original) (raw)

1National University of Singapore, 2Shanghai Jiao Tong University

* denotes equal contribution

ICRA 2025 Best Paper Award on Robot Manipulation and Locomotion

Best Robotics Paper Award @ CoRL 2024 MAPoDeL Workshop

mathcalD(R,O)\mathcal{D(R,O)}mathcalD(R,O) === **$\mathcal{D}$istances$($$\mathcal{R}$**obot, **$\mathcal{O}$**bject$)$

Abstract

Dexterous grasping is a fundamental yet challenging skill in robotic manipulation, requiring precise interaction between robotic hands and objects. In this paper, we present mathcalD(R,O)\mathcal{D(R,O)}mathcalD(R,O) Grasp, a novel framework that models the interaction between the robotic hand in its grasping pose and the object, enabling broad generalization across various robot hands and object geometries. Our model takes the robot hand’s description and object point cloud as inputs and efficiently predicts kinematically valid and stable grasps, demonstrating strong adapt ability to diverse robot embodiments and object geometries. Extensive experiments conducted in both simulated and real world environments validate the effectiveness of our approach, with significant improvements in success rate, grasp diversity, and inference speed across multiple robotic hands. Our method achieves an average success rate of 87.53% in simulation in less than one second, tested on three different dexterous robotic hands, and also performs successfully in real-world experiments using LeapHand. mathcalD(R,O)\mathcal{D(R,O)}mathcalD(R,O) Grasp provides a robust solution for dexterous grasping in complex and varied environments.

Pipeline Overview

Overview of mathcalD(R,O)\mathcal{D(R,O)}mathcalD(R,O) Grasp: We first pretrain the robot encoder with the proposed configuration-invariant pretraining method. Then, we predict the mathcalD(R,O)\mathcal{D(R,O)}mathcalD(R,O) representation between the robot and object point cloud. Finally, we extract joint values from the mathcalD(R,O)\mathcal{D(R,O)}mathcalD(R,O) representation.

Simulation Grasps

Partial Observation Grasps

Real-world Demos (1x)

Your browser does not support the video tag.

Apple

Bag

Brush

Cookie Box

Cube

Cup

Dinosaur

Duck

Tea Box

Toilet Cleaner

More Real-world Results

Your browser does not support the video tag.

BibTeX

@article{wei2024dro,
    title={D(R,O) Grasp: A Unified Representation of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping},
    author={Wei, Zhenyu and Xu, Zhixuan and Guo, Jingxiang and Hou, Yiwen and Gao, Chongkai and Cai, Zhehao and Luo, Jiayu and Shao, Lin},
    journal={arXiv preprint arXiv:2410.01702},
    year={2024}
}