Text-Aware Diffusion for Policy Learning

We showcase text-conditioned goal-reaching tasks learned via TADPoLe, for both Dog and Humanoid environments, and compare it against other text-to-reward approaches. TADPoLe is able to successfully learn a variety of behaviors, from standing upright to doing splits to kneeling.

We further explore the ability of TADPoLe to learn continuous locomotion behaviors conditioned on natural language specifications. Such tasks are often difficult to learn purely from static external description, as there is no canonical pose or goal frame that if reached, would denote successful achievement of the task. We propose Video-TADPoLe, which leverages large-scale pretrained text-to-video generative models, as a promising direction forward, and demonstrate how it outperforms using a language-video alignment model (ViCLIP) instead.

We investigate how well TADPoLe can be applied to learn robotic manipulation tasks through dense text-conditioned feedback. We do so by replacing the manually-designed ground-truth dense reward for each Meta-World task with TADPoLe's text-conditioned reward. We perform thorough comparisons between TADPoLe and VLM-RM by evaluating them on a diverse set of selected Meta-World tasks, in which we observe that TADPoLe provides meaningful zero-shot dense supervision that enables success across a variety of robotic manipulation tasks through text-conditioning.