I don’t have enough pixels, so I can edit the picture later?
Searching for low-pixel retouching in Zhihu, there are too many help posts, and there are so many dazzling tutorials from PS skills, plug-in artifacts to various retouching App tutorials. The point is that the effect is not known.
However, recently the Duke University research team has developed an AI retouching black technology PULSE, which can solve all low-pixel troubles. It is said that it can magnify the original resolution of the image by 64 times, and any slag image quality can be transformed into a high-definition, lifelike image in seconds, and even a mosaic face image, pores, wrinkles, and hair can be clearly restored.
1
Mosaic becomes HD portrait in seconds
PULSE is a new type of super-resolution algorithm, which samples photos through latent space exploration, and can enlarge the low resolution (LR) of 16x16 pixels to the high resolution (HR) of 1024x1024 pixels. It is increased by 64 times in a few seconds, while the traditional method can only zoom up to 8 times.
Let's first look at a set of examples. The most difficult LR headshots in the retouching world can be converted into high-definition and delicate images in seconds through PULSE.
More importantly, PULSE can locate key features of the face and generate a set of similar details with higher resolution. In the picture, even though the avatar is painted with mosaics, PULSE can also "imagine" facial details such as eyebrows, eyelashes, hair, face shape, etc., to form a high-definition, lifelike portrait.
However, the portrait produced by excessive blur is just a new virtual face, in fact it does not exist. Because of this, this technology cannot be used for identification. For example, the out-of-focus and indistinguishable pictures taken by the surveillance camera cannot be restored to real portraits through PULSE.
Cynthia Rudin, a computer scientist in a Duke University research team, said that "never before have such ultra-high resolution images been produced, which can produce new faces that don't exist, and they look real."
At the same time, she added that the technology adopted in this research can be widely used in the fields of medicine, microscope, astronomy, and satellite imagery. In addition, the research team has published the paper to arVix, a preprinted paper library, and was included in the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2020).
2
"Reduce losses", go beyond conventional retouching methods
For an LR image, the traditional way of matching the HR resolution part to the LR image to obtain super high resolution (SR) often results in poor sensitivity, unevenness, and picture distortion in the HR image.
In this study, the Duke University research team developed a new idea and proposed a new super-resolution algorithm PULSE. Instead of traversing the LR image to slowly add details, it found the LR corresponding to the HR. SR image is obtained by means of "loss".
Original LR (first row), PULSE output HR (middle row), LR corresponding to HR (last row)
PULSE uses a Generative Adversarial Network (GAN), which is a training model that, as the name suggests, uses an adversarial game for target training. The main structure includes a generator (Generator) and a discriminator (Discriminator). In the same set of photo training, one is responsible for training the received image and output, and the other is responsible for receiving the output and checking whether it is realistic enough.
The following are the test results compared with the original picture:
In the figure, the first line is the original image, the second line is the LR corresponding to the HR obtained by "reducing the loss", and the third line is the HR obtained through PULSE. It can be seen that although there is a slight difference from the original image, The degree of reduction is already very high.
The paper shows that in order to test the advantages of PULSE in SR, the Duke University research team used 4 different image scaling methods to conduct a comparative study. In this study, 1440 images in the CelebA HQ dataset were used to test LR facial images, especially the details of the eyes, lips, and hair, with scale factors of x8 and x64.
PULSE shows obvious advantages, especially at X64 resolution, the blurred portrait is completely restored, especially in the details of the eyes and lips, other methods can hardly achieve this effect.
In addition, for the test results, the researchers used the common MOS test method for perceptual super-resolution, and invited five raters to score the image results from 1-5. The results showed that the HR source HD image resolution score was 3.74, and the PULSE Reached 3.60, only 0.14 difference, it can be said that it has almost reached the level of real high-quality images.
However, the researchers also admit that PULSE is not perfect. The high-resolution images it produces are different from the professional original images. But as technology and tools improve, this technology will be improved little by little.
Now the research team has released PULSE to the Github open source platform and harvested 569 stars. Friends who have troubles about editing pictures can install and experience~ (Github address: https://github.com/adamian98/pulse)
Reference link:
http://pulse.cs.duke.edu/
https://www.gizmodo.co.uk/2020/06/researchers-have-created-a-tool-that-can-perfectly-depixelate-faces/
https://www.rt.com/news/492091-ai-tech-undo-pixelation/
Author | Bei Shuang
Transfer from: Leifeng.com ( leiphone-sz )
Paper address: https://urlify.cn/ABJRFf
Sharing or watching is my greatest support