Codebase for paper: RoCo: Dialectic Multi-Robot Collaboration with Large Language Models

Mandi Zhao, Shreeya Jain, Shuran Song

Arxiv | Project Website

conda create -n roco python=3.8 
conda activate roco

pip install mujoco==2.3.0
pip install dm_control==1.0.8 

If you have M1 Macbook like me and would like to visualize the task scenes locally:

Download the macos-compatible .dmg file from MuJoCo release page, inside it should have a MuJoCo.app file that you can drag into your /Application folder, so it becomes just like other apps in your Mac. You could then open up the app and drag xml files in it. Find more informationa in the official documentation.

pip install -r requirements.txt

This is required for prompting GPTs or Claude LLMs. You don't necessarily need both of them. Put your key string somewhere safely in your local repo, and provide a file path (something like ./roco/openai_key.json) and load them in the scripts. Example code snippet:

import openai  
openai.api_key = YOUR_OPENAI_KEY
import anthropic
client = anthropic.Client(api_key=YOUR_CLAUDE_KEY)
streamed = client.completion_stream(...)  

$ conda activate roco
(roco) $ python run_dialog.py --task pack -llm gpt-4

Please direct to Mandi Zhao. If you are interested in contributing or collaborating, please feel free to reach out! I'm more than happy to chat and brainstorm together.

@misc{mandi2023roco,
      title={RoCo: Dialectic Multi-Robot Collaboration with Large Language Models}, 
      author={Zhao Mandi and Shreeya Jain and Shuran Song},
      year={2023},
      eprint={2307.04738},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}