AI Super Resolution
Super-resolution technology refers to the operation of converting low-resolution images or videos into high-resolution images or videos through algorithms.
Super-resolution can be divided into two types: single image super-resolution (Single Image Super Resolution, SISR) and video super-resolution (Video Super Resolution, VSR).
The super-resolution function in OpenCV is concentrated in the contrib module, so we first need to install the module, which can be installed using the following code:
pip install opencv-contrib-python==4.4.0.44 -i https://pypi.tuna.tsinghua.edu.cn/simple/
If the opencv-python library exists before, then installing the opencv-contrib-python library may cause errors when running the code; the solution is to uninstall all opencv libraries first, and then install the opencv-contrib-python library.
The OpenCV vision library currently only supports four different image super-resolution models, which can achieve 2x, 3x, 4x, and even 8x image super-resolution. The specific description of these models is as follows:
- EDSR (CVPR2017): The model size is ~38.5MB, and Batch Size=16 was trained for 3 days. The advantage is high accuracy, but the disadvantage is that the model is large and slow. It provides x2, x3, and x4 training models.
- ESPCN (CVPR2016): The model size is ~100KB, and Batch Size=32 trained about 100 iterations. The advantages are small size, fast speed, and good performance. The disadvantage is that the visual performance is not as good as the newer and more robust model. Provide x2, x3, x4 training models.
- FSRCNN (ECCV2016): The model size is ~40KB, and Batch Size=1 trained about 30 iterations. The advantage is fast, small, and accurate, and the disadvantage is that the accuracy is not high enough.
- LapSRN (CVPR2017): The model size is 1-5MB, and Batch Size=32 trained about 50 iterations. The advantage is that the model can do multi-scale super-resolution with one forward pass. It can now support 2x, 4x, 8x and [2x , 4x] and [2x, 4x, 8x] super-resolution, the disadvantage is that the speed is slower than ESPCN and FSRCNN, and the accuracy is worse than EDSR, providing x2, x4, x8 training models.
The download address of the above four models is: https://github.com/opencv/opencv_contrib/tree/master/modules/dnn_superres
The implementation code is as follows:
import cv2
from cv2 import dnn_superres
img = cv2.imread(r'F:/SuperResolution/test_image_720p.jpg') # 读取720p的图像
trained_model_path = r'F:/SuperResolution/TF-ESPCN/export/ESPCN_x2.pb' # 训练好的ESPCN_x2模型的存储路径
sr = dnn_superres.DnnSuperResImpl_create() # 实例化对象
sr.readModel(trained_model_path) # 读取ESPCN_x2模型
sr.setModel('espcn', 2) # 设置超分图像放大比例（与训练模型的超分倍数一致），放大图像
result = sr.upsample(img) # 上采样，超分
cv2.imwrite(r'F:/SuperResolution/result_image_2k.jpg', result) # 保存2k图像