Reading time v1: 20190728.
This first article counted as electronic reading notes, so I hope to set a standard: notes will include, Abstract Duplication, Approach, Contributions / Goodness and Weakness.
- Abstract Duplication
optical flow is a critical component of video editing applications, e.g. for tasks such as object tracking, segmentation, and selection. In this paper, we propose an optical flow algorithm called simpleflow whose running times increase sublinearly in the number of pixels. Central to our approach is a probabilistic representation of the motion flow that is computed using only local evidence and without resorting to global optimization. To estimate the flow in image regions where the motion is smooth, we use a sparse set of samples only, thereby avoiding the expensive computation inherent in traditional dense algorithms. We show that our results can be used as is for a variety of video editing tasks. For applications where accuracy is paramount, we use our result to bootstrap a global optimization. This significantly reduces the running times of such methods without sacrificing accuracy. We also demonstrate that the simpleflow algorithm can process hd and 4k footage in reasonable times.
The central idea: A fast run time of registration algorithm -Simple Flow linear complexity of Asia. In the pyramid model architecture, from the ( L + 1'd) to the layer ( L) upsampling process layer, the patch according to the span where local motion flow, determining the motion of a pixel in the patch, if it is dense flow, in ( L ) level, the motion stream for each pixel in the patch will be accurately calculated; if the smooth flow, the ( L ) level, the first calculation of the optical flow on the angle of the patch, and light of other ways to get an interpolation position stream, using only smooth flow of sparse samples. Region in which the presence of smooth and slow, since the interpolation calculation speed is faster than the optical flow of each pixel, so that computational complexity is sub-linear, when the scene is at the most dense flow is also linear, it can accept.
- Approcah
Notation： we consider two successive frames ft and ft+1. we use (x, y) for pixel positions and (u, v) for flow vectors, that is, we seek to estimate u and v at each pixel such that the scene point at (x, y) in ft is visible at (x + u, y + v) in ft+1. although strictly speaking, u and v depend on (x, y); for the sake of clarity, we will use the notation (u, v) instead of (u(x, y),v(x, y)) when possible. we use ft(x, y) to denote the rgb color of the (x, y) pixel in ft.
On the same assumptions assumption constant-color and the flow is locally smooth. Equation 1 is used to represent the energy difference between before and after the two:
（1）
2 is used to calculate the formula ( X 0 , Y 0 ) at the optimum optical flow ( U 0 , V 0 ) such that the minimum energy loss:
（2）
In the above formula, items corresponding to [Sigma filter cartridge, i.e., each pixel value is added, is not conducive to distinguish the boundary information, it is introduced or joint- Cross- Bilateral Filtering:
（3）
- Multiscale Flow Estimation
作者对Pyramid的讲述很清晰，直接引用：“We construct an image pyramid for each image frame in which each level is twice coarser than the previous one，at the coarsest level of the pyramid, we estimate the flow using the scheme described[ in section 2论文中的]. we now explain how to compute the flow at level l assuming the flow at l +1 is known.”
The remaining part of the detailed reading papers, it is clear! Especially for the determination of the sparse part:
“for each layer, we estimate a flow irregularity map where the flow is smooth and where it varies more. at each pixel (x0;y0), we compute the irregularity value as：
during upscaling, if this value is above a threshold t, we run the full pipeline on the corresponding upscaled pixels. otherwise, we compute the flow at the corners of the patch, and find the flows at other pixel using bilinear interpolation.but when more accuracy is desirable, one could trade-off estimating the flow at more points and fitting a higher-order function to them.”
- Contributions/Goodness:
1, linear interpolation using the region of slow motion to reduce the time, speed thief, registration algorithm using fast motion area calculation, sub-pixel estimation may also be implemented Oh
2, high accuracy, if desired, the paper can be used as other algorithms algorithm a step guide, see the specific caption of Figure 8
"we presented a simple method for optical flow with running times that grow sublinearly with video resolution. a key property of our approach is that we do not resort to global optimization to propagate local information across the image. instead, we average local probability distributions computed from standard color differences.The local aspect of our scheme is also the key component that enables sublinear computation "
End!
The first to write study notes, the purpose of the paper after reading the review have convenient place, but it feels to write more slowly, in the future the process of writing, the author of the original English text excerpt is still based, coupled with their own understand, thus reducing the time, but also the right of expression to the idea, sometimes translated into Chinese is not like the look.
And the king of mutual encouragement!