Hello everyone, I’m Xiao Zhang, long time no see~
This tweet is related to computer vision. It takes less than 20 lines of Python code to convert a picture from natural wind to hand-drawn style . During this period, no pre-processing or post-processing is performed on the picture; only two common libraries are used in the code. Numpy is responsible for the core calculations, and Pillow is responsible for image reading and writing
Before starting the text, let’s take a look at the initial results. The following is a comparison of before and after conversion of a single image.
Figure one
Figure II
Figure three
In order to increase the fun, apply this code to a video later, add a background music, and a fresh "hand-drawn style video" is released.
Python hand-painted video production!
"Hand-painted style" realization steps
Before explaining, you need to understand the three main characteristics of hand-drawn images:
- The picture must be a grayscale image, single-channel;
- The edges of the lines are heavily painted black, and the same or similar pixel values tend to be white after conversion;
- With the support of the light source effect, the gray scale change can simulate the distance effect of human vision
Read the picture and convert it to an array
Because the pixel calculation will be used later, for convenience, the read image is converted into an array in advance
a = np.asarray(Image.open("Annie1.jpg").convert('L')).astype('float')
Calculate the x, y, z axis gradient values and normalize them
A hand-drawn picture characteristics just mentioned, is painted more focused picture region edge , the edge portion of the positioning image, the most effective way is to calculate the gradient, a gray scale image to simulate the effect of distance, depth denotes a preset depth, z-axis gradient default Is 1
depth = 10. # (0-100)
grad = np.gradient(a) # 取图像灰度的梯度值
grad_x, grad_y = grad # 分别取横纵图像梯度值
grad_x = grad_x * depth / 100.
grad_y = grad_y * depth / 100.
Complete the normalization operation on the gradient value
A = np.sqrt(grad_x ** 2 + grad_y ** 2 + 1.)
uni_x = grad_x / A
uni_y = grad_y / A
uni_z = 1. / A
Add light source effect
In addition to calculating the gradient value of the hand-painted wind picture, the influence of the light source needs to be considered; according to the angle of the light source, the gradient value on the x, y, and z axes will be affected to different degrees. Add a simulated light source and place it in the oblique Above, it forms two angles with x and y respectively
And these two angles are known through experiments, and then the final new pixel value is calculated according to the sine and cosine function
vec_el = np.pi / 2.2 # 光源的俯视角度,弧度值
vec_az = np.pi / 4. # 光源的方位角度,弧度值
dx = np.cos(vec_el) * np.cos(vec_az) # 光源对 x轴的影响
dy = np.cos(vec_el) * np.sin(vec_az) # 光源对 y轴的影响
dz = np.sin(vec_el) # 光源对z 轴的影响
b = 255 * (dx * uni_x + dy * uni_y + dz * uni_z) # 光源归一化,8 255
b = b.clip(0, 255)# 对像素值低于0，高于255部分做截断处理
Export the picture and save it
im.save("Annie_shouhui.jpg")
The following is all the code involved in this step
from PIL import Image
import numpy as np
a = np.asarray(Image.open("Annie1.jpg").convert('L')).astype('float')
depth = 10. # (0-100)
grad = np.gradient(a) # 取图像灰度的梯度值
grad_x, grad_y = grad # 分别取横纵图像梯度值
grad_x = grad_x * depth / 100.
grad_y = grad_y * depth / 100.
A = np.sqrt(grad_x ** 2 + grad_y ** 2 + 1.)
uni_x = grad_x / A
uni_y = grad_y / A
uni_z = 1. / A
vec_el = np.pi / 2.2 # 光源的俯视角度,弧度值
vec_az = np.pi / 4. # 光源的方位角度,弧度值
dx = np.cos(vec_el) * np.cos(vec_az) # 光源对 x轴的影响
dy = np.cos(vec_el) * np.sin(vec_az) # 光源对 y轴的影响
dz = np.sin(vec_el) # 光源对z 轴的影响
b = 255 * (dx * uni_x + dy * uni_y + dz * uni_z) # 光源归一化
b = b.clip(0, 255)
im = Image.fromarray(b.astype('uint8')) # 重构图像
im.save("Annie_shouhui.jpg")
Make a hand-painted video
Although the effect of the image conversion is good, the image is static after all. As a visual animal, it would be great if it can be made dynamic. After knowing the above method, you only need to add another frame splitting to the video. Operation, you can make a hand-painted video effect
you-get download video
Here I used the you-get command to find a video on station B and downloaded it.
you-get --format=dash-flv -o ./ https://www.bilibili.com/video/BV1tT4y1j7a9?from=search&8014393453748720686
After downloading, use OpenCV2 to cut the frame of the video, cut the frame and convert the picture at the same time, and write it out to the local video file
vc = cv2.VideoCapture(video_path)
c = 0
if vc.isOpened():
rval,frame = vc.read()
height,width = frame.shape[0],frame.shape[1]
print(height, width)
else:
rval = False
height,width = 960,1200
# jpg_list = [os.path.join('Pic_Directory/',i) for i in os.listdir('Pic_Directory') if i.endswith('.jpg')]
fps = 24 # 视频帧率
video_path1 = './text.mp4'
video_writer = cv2.VideoWriter(video_path1,cv2.VideoWriter_fourcc(*'mp4v'),fps,(width,height))
while rval:
rval,frame = vc.read()# 读取视频帧
img = coonvert_jpg(Image.fromarray(frame))
frame_converted = np.array(img)
# 转化为三通道
image = np.expand_dims(frame_converted,axis = 2)
result_arr = np.concatenate((image,image,image),axis = -1)
video_writer.write(result_arr)
print('Sucessfully Conveted---------{}'.format(c))
c = c + 1
if c >= 3000:
break
video_writer.release()
When extracting a picture sequence, you need to pay attention, because the converted picture is a single-channel, it is impossible to directly generate a video sequence with OpenCV. It is necessary to add a step to convert a single-channel to a three-channel!
# 转化为三通道
image = np.expand_dims(frame_converted,axis = 2)
result_arr = np.concatenate((image,image,image),axis = -1)
If you want to make the generated video more feel, you can add a back view music, with the help of editing software, Python can be used, it is recommended to use editing software here, because Python customization to increase audio effects is not ideal, and real-time feedback is required when adding music , And Python cannot meet this requirement temporarily
Data source code acquisition
For the source code involved in the article to obtain the data, follow the WeChat official account : Xiao Zhang Python , and the back-end reply keyword: 210322 !
summary
This article mainly introduces how to use Python to convert a picture into a hand-painted style. The amount of code is small but the knowledge domain is related to mathematics and physics, so it is not easy to understand. The purpose of this article is only to introduce you to the hand-painted style conversion of pictures. One way, of course, if you have an interested partner, you can delve into it
Alright, the above is the entire content of this article, and finally thank you all for reading, see you in the next issue~