This is the official implementation of paper OIFlow: Occlusion-Inpainting Optical Flow Estimation by Unsupervised Learning, IEEE Transactions on Image Processing, 2021

Occlusion is an inevitable and critical problem in unsupervised optical flow learning. Existing methods either treat occlusions equally as non-occluded regions or simply remove them to avoid incorrectness. However, the occlusion regions can provide effective information for optical flow learning. In this paper, we present OIFlow, an occlusion-inpainting framework to make full use of occlusion regions. Specifically, a new appearance-flow network is proposed to inpaint occluded flows based on the image content. Moreover, a boundary dilated warp is proposed to deal with occlusions caused by displacement beyond the image border. We conduct experiments on multiple leading flow benchmark datasets such as Flying Chairs, KITTI and MPISintel, which demonstrate that the performance is significantly improved by our proposed occlusion handling framework.

(a) reference image. (b) ground truth flow. (c)(d) results without / with the optical flow refinement. (e) detected occlusion map. (f) a zoom-in window with mask (dark region) overlaid on the image. Here an appearance flow is learned on the image domain, which is further used to inpaint the occluded regions by the non-occluded ones. (g) the appearance flow of (f) is applied to guide the propagation in the flow field. (h) The propagated results in the zoom-in region.

If you think this work is helpful, please cite

    @article{liu2021oiflow,
      title={OIFlow: occlusion-inpainting optical flow estimation by unsupervised learning},
      author={Liu, Shuaicheng and Luo, Kunming and Ye, Nianjin and Wang, Chuan and Wang, Jue and Zeng, Bing},
      journal={IEEE Transactions on Image Processing},
      volume={30},
      pages={6420--6433},
      year={2021},
      publisher={IEEE}
    }

Part of our codes are adapted from IRR-PWC and UnFlow, we thank the authors for their contributions.