Learning to Design and Use Tools for Robotic Manipulation

Sample Efficiency

Push

Catch Balls

Scoop

Fetch Cube

Lift Cup

3D Scoop

Learning curves for our framework, prior methods, and baselines. Across all tasks, our framework achieves improved performance and sample efficiency. Shaded areas indicate standard error across 6 random seeds on all methods, except the Scoop (3D) task where we use 3 seeds due to computational constraints.

Design-Control Tradeoff

Control cost / design cost ratio with different α.

α = 1.0

α = 0.3

α = 0.7

α = 0.0

Qualitative examples of tools generated by setting our tradeoff parameter α to different values. We can see that as α increases, the tools created by the designer policy have shorter links at the left and right sides to decrease material usage. With low α values, large tools prevent the control policy from having to move the tool far.

Generalization

(a) Initialization ranges and zero-shot performance when cutting out 60% of the area of the entire possible training region.

(b) Returns for policies trained with varying relative cutout region area.

(c) Fine-tuning performance compared to learning from scratch across 4 target goals

Interpolation results on the pushing task. In (a), we plot the success (light blue) and failure (dark blue) goal regions. Areas within the dotted yellow borders denote unseen cutout regions (interpolation). The region within the teal border (but outside cutout regions) shows the training region. The area outside the teal border is unseen during training (extrapolation). (b) and (c) show return curves (averaged over 3 runs). Shaded regions denote standard error. In (b), we observe that the performance for policies trained with small cutouts is close to that of the setting trained on all goals. In (c) we show that even for poses that are far away from the initial training region, our policies are able to learn to solve the task within a handful of gradient steps, and is much more effective than learning from scratch.

Visualizations of tool designs outputted by a single learned designer policy for push, fetch cube, lift cup, and 3D scoop as the initial/goal position varies.

Supplementary Video