Keyword [FlowNet] [Correlation Layer]

Dosovitskiy A, Fischer P, Ilg E, et al. Flownet: Learning optical flow with convolutional networks[C]//Proceedings of the IEEE international conference on computer vision. 2015: 2758-2766.

1. Overview

In this paper, it proposes
1) FlownetSimple
2) FlowNetCorr (Correlation Layer)
3) Flying Chairs Dataset

And experiments on Sintel and KITTI.

2. Architecture

2.1. FlowNet

2.2. Correlation Layer

1) Top Branch. patch size $k\times k$, global stride $s_1$
2) Bottem Branch. patch size $d\times d$, stride in patch $s_2$

In this paper, $k=0$, $d=20$, $s_1=1$, $s_2=2$.
1) Top Branch. vector $1\times 1 \times c$, total $h \times w$ vectors.
2) Bottem Branch. patch $20 \times 20$ with stride 2.

For each point of A, there will be $21 \times 21$ values. ($21 = 2 * (d / s_2) + 1$)
So output $h \times w \ times 441$.

Replace bilinear upsampling
1) compute image boundaries
2) replace smoothness coefficient

3. Flying Chairs

1) Randomly sample 2D affine transformation parameters for the background and the chairs
2) use parameters to generate the second image, GT optical flow and occlusion regions.

4. Data Augmentation

1) translation, rotation, scaling and Gaussian noise
2) brightness, contrast, gamma and color

(ICCV 2015) FlowNet:Learning optical flow with convolutional networks