Huawei has just released P30 "telescope" the phone can be tens of meters to the names on outdoor photo in the Eifel, indeed admirable, but its price is prohibitive. So, do not buy Huawei cell phone, high-level SLR to shoot less than full details of high-definition picture yet?
The camera is not enough algorithm compact, has a super ability to take pictures of the phone can not do without the blessing of the algorithm. This article describes the image super-resolution projects can help you padded short board camera lens.
Huawei P30 conference shown on the Eiffel Tower HD long-distance photos.
Today, a Reddit users posted their Keras based image super-resolution projects, after allowing for a larger picture is still clear. First look at the results.
After significantly enlarged photographs of butterflies (moths?) Still no distortion, the back of the villi clearly visible.
Authors said the project aims to improve the quality of low-resolution images, making it a new look. Using this tool can be super image scaling, but also easily in the RDN and GAN to experiment on.
The residual Keras project comprises a dense network of different implementations, they may be used for efficient single super-resolution image (ISR). The author also provides a variety of documentation to help train the model, including how to use these networks against the loss of training components.
Examples of projects
The amplification factor (upscaling factor) used in Example 2, i.e., triple the number of pixels. We can see in the example of an image generated in sample_weights weight, they are stored in git lfs. To download these weights, you need to copy the repo, and then run git lfs pull.
Left to the original low-resolution image, the output of the intermediate graph of the network, the right is the baseline model using GIMP bicubic scaling obtained amplified result.
The following are different methods of action in contrast to the effect of image noise, these methods are: use bicubic scaling the baseline model, using the pixel level content loss function training RDN network, and the use of VGG . 19 content compressed data set and the loss functions retraining RDN network. The repo contains heavy weight of these models.
Bicubic up-scaling (base model) output sample.
The output pixel level using the sample contents of the loss function trained network RDN.
The output example uses VGG content and training components against loss of RDN network.
What super-resolution projects
Show in front of the super-resolution effect is made out of different models according to the project implementation. Super-resolution reconstructed image detail desired deletion according to the existing image information, it is usually by means of a convolutional neural network decimated image information, and then extend these convolution resolution desired information to the image obtained by the transpose.
In this project, a lot of new features and modules, such as the use of VGG and GAN achieve real enlarged image. The project implementation is RDN with RRDN network, while also providing pre-training and weights Colab tutorials. Whether for training or infer, based on this information we can get started quickly.
In addition, the project now can be published to PyPI, so installation is also just type pip command.
In short, the project achieve super-resolution three networks, and the use of Keras version VGG-19 as a feature extraction module. Correlation is shown below the super-resolution three networks:
-
Residual Dense Network for Image Super-Resolution(Zhang et al. 2018, arXiv:1802.08797)
-
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks(Wang et al. 2018, arXiv:1809.00219)
-
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network(SRGANS, Ledig et al. 2017, arXiv:1609.04802)
If we want to generate that kind of high-definition view of the above, the project also provides a range of resources:
-
Documentation: https: //idealo.github.io/image-super-resolution/
-
Code: https: //github.com/idealo/image-super-resolution/
-
Colab infer Code: https: //colab.research.google.com/github/idealo/image-super-resolution/blob/master/notebooks/ISR_Prediction_Tutorial.ipynb
-
Colab training Code: https: //colab.research.google.com/github/idealo/image-super-resolution/blob/master/notebooks/ISR_Traininig_Tutorial.ipynb
How to use super-resolution project
You can choose between two ways to install super-resolution image (ISR) package.
PyPI installation from the ISR (recommended):
pip install ISR
Mounting the ISR GitHub source:
git clone https://github.com/idealo/image-super-resolution
cd image-super-resolution
python setup.py install
prediction
If we need to extend the low-resolution image, by means of a simple two-step ISR can perform super-resolution. First, load the image and do some pre-processing:
import numpy as npfrom PIL import Image
img = Image.open('data/input/test_images/sample_image.jpg')
lr_img = np.array(img)/255.
lr_img = np.expand_dims(lr_img, axis=0)
Load forecasting model and execution:
from ISR.models import RDN
rdn = RDN(arch_params={'C':6, 'D':20, 'G':64, 'G0':64, 'x':2})
rdn.model.load_weights('weights/rdn-C6-D20-G64-G064-x2_enhanced-e219.hdf5')
sr_img = rdn.model.predict(lr_img)[0]
sr_img = sr_img.clip(0, 1) * 255
sr_img = np.uint8(sr_img)
Image.fromarray(sr_img)
training
If you need to use your data set retraining super-resolution model, then we only need to modify it to read parameters. First, create the model as follows:
from ISR.models import RRDN
from ISR.models import Discriminator
from ISR.models import Cut_VGG19
lr_train_patch_size = 40
layers_to_extract = [5, 9]
scale = 2
hr_train_patch_size = lr_train_patch_size * scale
rrdn = RRDN(arch_params={'C':4, 'D':3, 'G':64, 'G0':64, 'T':10, 'x':scale}, patch_size=lr_train_patch_size)
f_ext = Cut_VGG19(patch_size=hr_train_patch_size, layers_to_extract=layers_to_extract)
discr = Discriminator(patch_size=hr_train_patch_size, kernel_size=3)
Trainer create a variety of objects, configuration and training delivered to the object:
from ISR.train import Trainer
loss_weights = {
'generator': 0.0,
'feat_extr': 0.0833,
'discriminator': 0.01,
}
trainer = Trainer(
generator=rrdn,
discriminator=discr,
feature_extractor=f_ext,
lr_train_dir='low_res/training/images',
hr_train_dir='high_res/training/images',
lr_valid_dir='low_res/validation/images',
hr_valid_dir='high_res/validation/images',
loss_weights=loss_weights,
dataname='image_dataset',
logs_dir='./logs',
weights_dir='./weights',
weights_generator=None,
weights_discriminator=None,
n_validation=40,
lr_decay_frequency=30,
lr_decay_factor=0.5,
T=0.01,
)
Start training:
trainer.train(
epochs=80,
steps_per_epoch=500,
batch_size=16,
)
Network architecture and hyperparameter
In fact, if we need to re-training, you also need to know what specific parameters have said. This section describes the architecture and the corresponding parameters of the super-super-resolution network.
RDN network architecture
The main parameters RDN network architecture as follows:
-
D: dense residual block (RDB) Number
-
C: Number of convolutional inner layer stacked RDB
-
G: number of feature within each convolution FIG layer RDB
FIG. Source: https: //arxiv.org/abs/1802.08797
RRDN network architecture
The main parameters RRDN architecture are as follows:
-
T: the number of residuals in the residual dense block (the RRDB)
-
Each RRDB number of internal residual dense block (RDB) is: D
-
C: Number of convolutional inner layer stacked RDB
-
G: number of feature within each convolution FIG layer RDB
FIG. Source: https: //arxiv.org/abs/1809.00219
