CARVIEW

MOTORHOMES

Select Language

HTTP/2 301 server: GitHub.com content-type: text/html location: https://haofeixu.github.io/unimatch/ access-control-allow-origin: * expires: Mon, 29 Dec 2025 23:03:41 GMT cache-control: max-age=600 x-proxy-cache: MISS x-github-request-id: EF35:2DDCFF:95C0F6:A81989:69530674 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 22:53:41 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210035-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1767048821.102521,VS0,VE201 vary: Accept-Encoding x-fastly-request-id: 2708c540d01a413f202b16aafb99871326b77b03 content-length: 162 HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Mon, 08 Dec 2025 12:29:34 GMT access-control-allow-origin: * etag: W/"6936c4ae-423f" expires: Mon, 29 Dec 2025 23:03:41 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: F83B:2DDCFF:95C0FA:A8198B:69530675 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 22:53:41 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210035-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1767048821.316899,VS0,VE210 vary: Accept-Encoding x-fastly-request-id: 69142e57e84d752f29d371469d154768c61c8950 content-length: 4150 UniMatch

Unifying Flow, Stereo and Depth Estimation

Haofei Xu^1,2 Jing Zhang³ Jianfei Cai⁴ Hamid Rezatofighi⁴ Fisher Yu¹
Dacheng Tao³ Andreas Geiger^2,5

TPAMI 2023

¹ETH Zurich ²University of Tübingen ³The University of Sydney ⁴Monash University ⁵MPI for Intelligent Systems, Tübingen

Paper

Slides

Video Code

Colab

Demo

A unified model for three motion and 3D perception tasks.

Results on unseen videos.

Highlights

A unified dense correspondence matching formulation and model for three tasks.
Our unified model naturally enables cross-task transfer (flow → stereo, flow → depth) since the model architecture and parameters are shared across tasks.
State-of-the-art or competitive performance on 10 popular flow, stereo and depth datasets, while being simpler and more effcient in terms of model design and inference speed.

Overview

Strong features + parameter-free matching layers ⇒ a unified model for flow/stereo/depth.
An additional self-attention layer to propagate the high-quality predictions to unmatched regions.

Cross-Task Transfer

Flow to depth transfer. We use an optical flow model pretrained on Chairs and Things datasets to directly predict depth on the ScanNet dataset, without any finetuning. The performance can be further improved by finetuning for the depth task.

When finetuning with a pretrained flow model as initialization, we not only enjoy faster training speed for stereo and depth, but also achieve better performance.

Results

Our GMFlow with only one refinement outperforms RAFT with 31 refinements on Sintel dataset.

We achieve the 1st places on Sintel (clean), Middlebury (rms metric) and Argoverse benchmarks.

Our GMFlow better captures fast-moving small object than RAFT.

Our GMStereo produces sharper object structures than RAFT-Stereo and CREStereo.

BibTeX

@article{xu2023unifying,
      title={Unifying Flow, Stereo and Depth Estimation},
      author={Xu, Haofei and Zhang, Jing and Cai, Jianfei and Rezatofighi, Hamid and Yu, Fisher and Tao, Dacheng and Geiger, Andreas},
      journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
      year={2023}
    }

This work is a substantial extension of our previous conference paper GMFlow (CVPR 2022, Oral), please consider citing GMFlow as well if you found this work useful in your research.

@inproceedings{xu2022gmflow,
      title={GMFlow: Learning Optical Flow via Global Matching},
      author={Xu, Haofei and Zhang, Jing and Cai, Jianfei and Rezatofighi, Hamid and Tao, Dacheng},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
      pages={8121-8130},
      year={2022}
    }

awesome webpage template

Original Source | Taken Source