Xinhao Liu*, Jintong Li*, Yicheng Jiang, Niranjan Sujay, Zhicheng Ynag, Juexiao Zhang, John Abanes, Jing Zhang, Chen Feng†

Checkout a mosaic demo of our dataset:

The project should be compatible with latest Pytorch and CUDA versions. The code is tested with Python 3.11, PyTorch 2.5.0, and CUDA 12.1. To install the dependencies, run:

conda env create -f environment.yml
conda activate citywalker

Please see dataset/README.md for details on how to prepare the dataset.

To train the model, run:

python train.py --config config/citywalk_2000hr.yaml

We provide our pretrained model in the releases tab.

To fine-tune the model, run:

python fine_tune.py --config config/finetune.yaml --checkpoint <path_to_checkpoint>

To test the model, run:

python test.py --config config/finetune.yaml --checkpoint <path_to_checkpoint>

@inproceedings{liu2025citywalker,
  title={Citywalker: Learning embodied urban navigation from web-scale videos},
  author={Liu, Xinhao and Li, Jintong and Jiang, Yicheng and Sujay, Niranjan and Yang, Zhicheng and Zhang, Juexiao and Abanes, John and Zhang, Jing and Feng, Chen},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  pages={6875--6885},
  year={2025}
}

The work was supported by NSF grants 2238968, 2121391, 2322242 and 2345139; and in part through the NYU IT High Performance Computing resources, services, and staff expertise. We thank Xingyu Liu and Zixuan Hu for their help in data collection.

We also thank the authors of the following repositories for their open-source implementations:

• ViNT: A Foundation Model for Visual Navigation, CoRL 2023
• NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration, ICRA 2024

Here is a list for highly relevant work for interested readers:

• MetaUrban: An Embodied AI Simulation Platform for Urban Micromobility, ICLR 2025
• NaVILA: Legged Robot Vision-Language-Action Model for Navigation, RSS 2025
• Learning to Drive Anyware with Model-Based Reannotation, arXiv 2025
• From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learning, arXiv 2025