Visual newcomer, pure white notes
Subscription number: Mo Li Sansheng
Disclaimer: This blog post is only a code test tutorial, thank you for your support;
Basic project information:
Deep Video Super-Resolution using HR Optical Flow Estimation
https://arxiv.org/abs/2001.02129
GitHub project link:
https://github.com/LongguangWang/SOF-VSR/tree/master/TIP
This project is 200M, if some students have difficulty downloading, you can go to my code cloud library to download
https://gitee.com/xiaohuoer1995/SOF-VSR
Summary:
Video Super Resolution (SR) aims to generate a series of high resolution (HR) frames from the corresponding objects of low resolution (LR), and the details of these frames seem reasonable and consistent in time.
The key challenge of video SR is to effectively utilize the time dependence between consecutive frames.
Existing deep learning-based methods usually estimate the optical flow between LR frames to provide time dependence.
However, the resolution conflict between the LR optical flow and the HR output hinders the restoration of details.
In this article, we propose an end-to-end video SR network to super-resolve optical flow and images.
The optical flow SR from the LR frame provides precise time dependence and ultimately improves the video SR performance.
Specifically, we first proposed the Optical Flow Reconstruction Network (OFRnet) to infer HR optical flow in a coarse-to-fine manner.
Then, the time dependence of HR optical flow encoding is used for motion compensation.
Finally, input the compensated LR input to the super-resolution network (SRnet) to generate the SR result.
A large number of experiments have proved the effectiveness of HR optical flow to improve SR performance.
The comparison results on the Vid4 and DAVIS-10 data sets show that our network has reached the most advanced performance.
to sum up:
In this paper, we propose an end-to-end deep network video SR . Our SOF-VSR first super-resolution optical flow provides accurate time dependence.
Then perform motion compensation according to HR optical flow.
Finally, SRnet is used to infer the SR result from the compensated LR frame.
A large number of experimental results show that our SOF-VSR network can restore accurate time details and improve the accuracy and consistency of SR.
The comparison with existing video SR methods also proves the most advanced performance of our SOF-VSR network.
Video super-division reconstruction, data set download:
The test data set, data and model download items are all included; the link below can download more related data;
Data download link backup:
https://davischallenge.org/davis2017/code.html
The test steps are as follows:
The troubleshooting process is as follows:
Run the rebuild command:
python demo_Vid4.py --degradation BI --scale 4 --gpu_mode True
The small errors that may be encountered are as follows:
Error one:
FileNotFoundError: [Errno 2] No such file or directory: 'data/test/Vid4/walk/lr_x4_BI'
The reason: because there is no execution Run data/test/generate_LR_images.mTherefore, the lack of data LR
Error two:
FileNotFoundError: [Errno 2] No such file or directory: 'results/Vid4'
Reason: The code is default, you need to create a new directory manually TIP/results/Vid4
The generated result directory is as follows:
Evaluation:
The quality of the generated reconstructed image seems to be worrying;
did the author fail to release the best model?