This repo contains FlowNet2[1] for TensorFlow. It includes FlowNetC, S, CS, CSS, CSS-ft-sd, SD, and 2.

pip install enum
pip install pypng
pip install matplotlib
pip install image
pip install scipy
pip install numpy
pip install tensorflow

Linux: sudo apt-get install python-tk

You must have CUDA installed: make all

To download the weights for all models (4.4GB), run the download.sh script in the checkpoints directory. All test scripts rely on these checkpoints to work properly.

python -m src.flownet2.test --input_a data/samples/0img0.ppm --input_b data/samples/0img1.ppm --out ./

Available models:

• flownet2
• flownet_s
• flownet_c
• flownet_cs
• flownet_css (can edit test.py to use css-ft-sd weights)
• flownet_sd

If installation is successful, you should predict the following flow from samples/0img0.ppm:

If you would like to train any of the networks from scratch (replace flownet2 with the appropriate model):

python -m src.flownet2.train

For stacked networks, previous network weights will be loaded and fixed. For example, if training CS, the C weights are loaded and fixed and the S weights are randomly initialized.

TODO

Benchmarks are for a forward pass with each model of two 512x384 images. All benchmarks were tested with a K80 GPU and Intel Xeon CPU E5-2682 v4 @ 2.30GHz. Code was executed with TensorFlow-1.2.1 and python 2.7.12 on Ubuntu 16.04. Resulting times were averaged over 10 runs. The first run is always slower as it sets up the Tensorflow Session.

[1] E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, T. Brox FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks, IEEE Conference in Computer Vision and Pattern Recognition (CVPR), 2017.