Like it and look again, get into a habit, search on the WeChat public account [JackCui-AI] to follow this programmer who loves to send technical dry goods. This article GitHub Jack-Cherish/PythonPark has been included, the first-line interview complete test sites, materials and my series of articles.
I. Introduction
Do you have any good-looking old photos of elders in your family?
Or black and white, or blur, or even wrinkles.
Old photos carry every meaning in life, which are memories of generations.
Today, Jack Cui teaches two algorithms to help you fix unclear, wrinkled old photos, and turn black and white photos into color photos.
Blur photos , one-click HD:
Wrinkle photos , one-click recovery:
Black and white photos , one-click coloring:
In this era of artificial intelligence, algorithms can do it for you !
Second, the algorithm
Blurred and wrinkled photo repair is the latest CVPR article from Microsoft in 2020:
「Bringing Old Photo Back to Life」
Coloring black and white photos is a classic coloring algorithm (2018):
「DeOldify」
1、Bringing Old Photo Back to Life
The author of this article is from City University of Hong Kong, Microsoft Research Asia, Microsoft Cloud + AI, University of Science and Technology of China.
The author uses a Variational Automatic Encoder (VAE) to transform the image into a hidden space, and performs image restoration operations in the hidden space.
Don't understand, speak human words!
For example, if a chess master observes the board for 5 seconds, he can remember the positions of all the pieces, which ordinary people cannot do.
But the placement of the pieces must be the actual game (that is, the internal rules of the existence of the pieces), and random placement of the pieces is not acceptable.
Chess masters are not better than us in memory, but are experienced and very good at recognizing chess patterns, so as to efficiently remember chess games.
The Auto Encoder (VAE) is the "Chess Master", it can find the inner rules (hidden space) of the picture, and then express it in its own way.
The author uses two VAEs:
The first VAE is used to encode the synthesized old photos (blurred, worn) into hidden space.
The second VAE is used to encode the corresponding clean old photos.
Then, learn the mapping from the defaced old photos to the clean photos in the hidden space.
In this way, an old photo restoration algorithm is implemented.
This is a bit like learning to control the clear and worn feature of the picture. By controlling this feature, the purpose of repairing damaged photos can be achieved.
The repair effect of the paper:
The algorithm is open source, and there is a pre-trained model, which can directly test the effect.
project address:
https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life
The project depends on "Synchronized-BatchNorm-PyTorch", just follow the tutorial configuration.
The first step, clone project:
git clone https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life
The second step, enter the project directory, clone depends on the project:
cd Face_Enhancement/models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../../
cd Global/detection_models
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../
The third step is to download the pre-trained model.
cd Face_Detection/
wget http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
bzip2 -d shape_predictor_68_face_landmarks.dat.bz2
cd ../Then download the trained models of Global and Face_Enhancement respectively, unzip them , and put them in the corresponding directories:
Global：
https://facevc.blob.core.windows.net/zhanbo/old_photo/pretrain/Global/checkpoints.zip
Face_Enhancement：
https://facevc.blob.core.windows.net/zhanbo/old_photo/pretrain/Face_Enhancement/checkpoints.zip
The download speed is slow or troublesome, you can download the project I have packaged and use it directly!
Model, code, material , project download address (extraction code: jack):
Netdisk link: https://pan.baidu.com/s/1jVjd8dS0j0AnWeFI-7l-eA
Instructions:
Image restoration without cracks, that is, the image is not clear, you can use the following commands:
python run.py --input_folder [test_image_folder_path] \
--output_folder [output_path] \
--GPU 0Put the pictures you want to repair in the [test_image_folder_path] directory (specify yourself), and the generated pictures will be put in the [output_path] directory.
For cracked pictures, an additional parameter is required, the command is as follows:
python run.py --input_folder [test_image_folder_path] \
--output_folder [output_path] \
--GPU 0 \
--with_scratchIt should be noted that the specified path needs to use an absolute path .
Running effect (click to view):
2、DeOldify
"DeOldify" is an image coloring algorithm.
This algorithm was used to restore the image of old Beijing a hundred years ago that has been on the hot search.
Everything is readily available and easy to use.
DeOldify is an application that confronts generation networks.
The principle is to use NoGAN technology, which combines the advantages of GAN training, such as excellent coloring effects, and also eliminates some side effects, such as unstable coloring and flickering.
The algorithm has been out for a long time, there should be many tutorials on the principle of the algorithm, so I won’t repeat it here, let’s see how to use it directly.
project address:
https://github.com/jantic/DeOldify
Need to configure the same as above, install some libraries, and then download the model weight file.
The project engineering page specifies which files to download and what to install.
After configuration, write the following code in the project directory:
#NOTE: This must be the first call in order to work properly!
from deoldify import device
from deoldify.device_id import DeviceId
#choices: CPU, GPU0...GPU7
device.set(device=DeviceId.GPU0)
import torch
if not torch.cuda.is_available():
print('GPU not available.')
import fastai
from deoldify.visualize import *
import warnings
warnings.filterwarnings("ignore", category=UserWarning, message=".*?Your .*? set is empty.*?")
colorizer = get_image_colorizer(artistic=True)
colorizer.plot_transformed_image("test_images/1.png", render_factor=10, compare=True)test_images/1.png is the picture to be colored. Run the program to color black and white photos.
Still, the model, code, and materials have been packaged, and the project download address (extraction code: jack):
Netdisk link: https://pan.baidu.com/s/17sma_a1ICJMY07KLnDpiww
Open Jupyter and run the run.ipynb file directly. The generated results are saved in the result_images folder.
In addition to processing pictures, coloring the video is no problem.
Three, summary
Are there any old black and white photos in your home?
Restoring their old photos for the elders is a surprise and a heart. Time passes and never goes back, the past can only be reminiscent~
If the elders are so amazing, remember to come back to forward and like!
I’m Jack Cui, a programmer who loves to send technical dry goods. See you next time~
The article is continuously updated, and you can search on the WeChat public account [JackCui-AI] to read it for the first time. This article has been included on GitHub Jack-Cherish/PythonPark . There are complete interview test sites for major companies. Welcome to Star.