| CARVIEW |
AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation
Abstract
This paper introduces AdaptiGraph, a learning-based dynamics modeling approach that enables robots to predict, adapt to, and control a wide array of challenging deformable materials with unknown physical properties. AdaptiGraph leverages the highly flexible graph-based neural dynamics (GBND) framework, which represents material bits as particles and employs a graph neural network (GNN) to predict particle motion. Its key innovation is a unified physical property-conditioned GBND model capable of predicting the motions of diverse materials with varying physical properties without retraining. Upon encountering new materials during online deployment, AdaptiGraph utilizes a physical property optimization process for a few-shot adaptation of the model, enhancing its fit to the observed interaction data. The adapted models can precisely simulate the dynamics and predict the motion of various deformable materials, such as ropes, granular media, rigid boxes, and cloth, while adapting to different physical properties, including stiffness, granular size, and center of pressure. On prediction and manipulation tasks involving a diverse set of real-world deformable objects, our method exhibits superior prediction accuracy and task proficiency over non-material-conditioned and non-adaptive models.
Video
Method Overview
In “AdaptiGraph”, we learn a graph-based neural dynamics model that adapts to objects with varying dynamics. We employ physical property conditioning, and perform inverse optimization to estimate the physical properties of unseen objects through interaction.
Material-Adaptive Dynamics
Our method can capture the dynamics of various object types such as rigid boxes, ropes, granular objects, and cloths. The red dashed boxes highlight the improved dynamics prediction accuracy of our method compared to the no adaptation baseline.
The baseline assumes that the center of pressure is at the geometric center of the box, and predicts the wrong rotation direction.
Our method identifies that the yarn has low stiffness, and predicts more bending effects.
The baseline assumes that the center of pressure is at the geometric center of the box, and predicts the wrong rotation direction.
Our method identifies that the polymer rope has high stiffness, and predictes less bending effects.
Our method identifies that the coffee beans have small granularity, and predicts more stacking effects.
The baseline predicts insufficient stretching and shearing effects for the low-stiffness modal fabric cloth.
Our method identifies that the chocolates have large granularity, and predicts less stacking effects.
The baseline predicts too much stretching effects for the high-stiffness cotton cloth.
Interactive Visualization
Before push
After push
Dynamics prediction
Adaptation Results
For unseen instances at test time, we perform online inverse optimization to refine their physical property variables based on the physical behaviors observed during interactions. Our estimation rankings align with human impressions of these objects.
Planning
Finally, our method enhances model-based planning for manipulation of different materials. The estimated physical property variable allows us to find more efficient actions, resulting in less steps to successfully achieve the target configuration and with lower final errors.
Box pushing - w/o Adaptation
Box pushing - Ours
Rope straightening - w/o Adaptation
Rope straightening - Ours
Granular gathering - w/o Adaptation
Granular gathering - Ours
Cloth relocating - w/o Adaptation
Cloth relocating - Ours
Example Simulation Data
Granular - Low granularity
Granular - High granularity
Rope - Low stiffness
Rope - High stiffness
Cloth - Low stiffness
Cloth - High stiffness
BibTeX
@inproceedings{zhang2024adaptigraph,
title={AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation},
author={Zhang, Kaifeng and Li, Baoyu and Hauser, Kris and Li, Yunzhu},
booktitle={Proceedings of Robotics: Science and Systems (RSS)},
year={2024}
}