How to Funkify an image using Python (NumPy and OpenCV)
Author|Luke Tambakis
Compile|Flin
Source|medium
In this blog, I'll explain how to make a Python script to "funkify" an image using Python code. The program is fast enough to even process live video (no GPU required)! like this:
If you're not interested in boring code explanations and just want to try it yourself, the easiest way is to use the FunkyCam repository. It'll show you how to install and run it in a few lines of code: https://github.com/LTambam/FunkyCam
How to run
The main function of the code is to take an image as input and return a funkified image as output:
def funkify(self, img):
edges = self.edge_mask(img)
blur = cv2.GaussianBlur(img, (self.color_blur_val, self.color_blur_val),
sigmaX=0, sigmaY=0)
indices = self.pick_color(blur.reshape((-1, 3)),
self.lightness, self.n_colors)
recolored = np.uint8(self.colors[indices].reshape(blur.shape))
cartoon = cv2.bitwise_and(recolored, recolored, mask=edges)
return cartoonNow, I'm going to take this diagram and break it down line by line.
Sample image
Step 1: Increase Edge Width
The project is based on this blog (https://github.com/Sudarshana2000/cartoonization). I want to repurpose this code so it can run live on a webcam. But their method proved impossible, but some of their code was still brilliant. Specifically the function used to find the edges of an image and increase the width of the edges.
The purpose of thickening the edges is to make it look like the ink lines in a cartoon or anime. Getting the edge is done by the first line in the code.
edges = self.edge_mask(img)The edge_mask() function calls the following code.
def edge_mask(self, img):
# get the edges of the image
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray_blur = cv2.GaussianBlur(gray,
(self.edge_blur_val, self.edge_blur_val),
-1)
edges = cv2.adaptiveThreshold(gray_blur, 255,
cv2.ADAPTIVE_THRESH_MEAN_C,
cv2.THRESH_BINARY,
self.block_size,
2)
return edgesThe first and second lines preprocess the image to prepare it for the adaptive threshold function. The image is first converted from color to grayscale and then blurred using a Gaussian kernel so that the adaptive threshold function produces a less noisy output.
The third line is the main part. Adaptive thresholding is a binarization function, which means it classifies each pixel in the image as black (0) or white (1). There are many binarization algorithms. Adaptive thresholding is unique in that it classifies each pixel based on the intensity of neighboring pixels rather than the entire image. This makes it perform better in different lighting conditions.
Adaptive threshold and global threshold comparison
Using this feature, the edges of the image can be quickly found, which makes it ideal for the version we run in real time. This is the function applied to our example image.
Step 2: Recolor the image
The next step is to recolor the image. One of the things that makes something look cartoonish is that there are fewer colors in the image. For example, real images will have thousands of different colors and tones due to lighting, but cartoon images only have a few colors. This can be done using toon shaders, which is usually done in video games.
In my code based on this project (https://github.com/Sudarshana2000/cartoonization), they use K-means to cluster pixels into a certain number of colors. Each pixel is then changed to the average color of its group. While this approach works well, it is too slow for real-time use. Besides, it's kind of boring.
Instead, colors are selected manually, and then all pixels are overlaid with those colors based on the color to which they are closest in brightness. This eliminates the need to look up colors and actually makes the image look as cool as you want. Luminance (a quantitative measure of brightness) was chosen because it preserves the lighting effects of the original image.
The code for recoloring in the main function consists of the following three lines:
blur = cv2.GaussianBlur(img, (self.color_blur_val, self.color_blur_val),
sigmaX=0, sigmaY=0)
indices = self.pick_color(blur.reshape((-1, 3)),
self.luminance, self.n_colors)
recolored = np.uint8(self.colors[indices].reshape(blur.shape))The first line just blurs the image, as in edge widening. Its purpose is to make the colors in the final image look smoother.
The second line is the most important part. The pick_color() function is a function used to calculate the color of each pixel in an image. It runs the following code:
def pick_color(self, img, color_lums, n_colors):
# reassign pixel colors based on luminance
# get luminance of pixels
lum_mult = [0.114, 0.587, 0.299]
img_lum = np.sum(np.multiply(img, lum_mult), axis=1)
# create list of conditions for each color
condlist = []
choicelist = []
for i in range(n_colors):
choicelist.append(i)
if i < n_colors-1:
condlist.append(img_lum < (color_lums[i]+color_lums[i+1])/2)
else:
condlist.append(img_lum > (color_lums[i]+color_lums[i-1])/2)
# get index of new color for each pixel
inds = np.select(condlist, choicelist)
return indsFirst, the function calculates the brightness of each pixel based on its RGB value using the formula from this post.
Then, a list of criteria was created to determine which color to select based on brightness. By creating this list of conditions, the function can adapt to any number of colors the user wants to use, rather than hard-coding a fixed number. The np.select() function is used to actually select from this list. It's actually a series of "if statements" like this:
# e.g. for 3 colors
if pix_lum < (color_lums[0] + color_lums[1])/2:
pix_ind = 0
else if pix_lum < (color_lums[1] + color_lums[2])/2:
pix_ind = 1
else:
pix_ind = 2The third line recolors the image based on the index we got from the pick_color() function. It does this simply by slicing the image array and converting it to uint_8 so that it can be displayed correctly.
The output of this process is shown below.
Step 3: Merge
The last part of the main feature is to combine the recolored image with rough edges.
cartoon = cv2.bitwise_and(recolored, recolored, mask=edges)Final Results:
Looks cool, right?
It runs so fast that you can use it with a webcam in real time, without even using GPU processing. You can also convert videos as follows:
☆ END ☆
If you see this, it means you like this article, please forward it and like it. Search "uncle_pn" on WeChat. Welcome to add the editor's WeChat "woshicver". A high-quality blog post will be updated in the circle of friends every day.
↓Scan the QR code to add the editor↓
