☞ ░Go to LaoYuanPython blog░ https://blog.csdn.net/LaoYuanPython
I. Introduction
In " OpenCV-Python image addition operation cv2.add function detailed explanation " and " OpenCV-Python image subtraction operation cv2.subtract function detailed explanation and the difference comparison with matrix subtraction " detailed introduction of image addition operation and subtraction operation, there are addition Subtraction is multiplication and division. This article introduces image multiplication.
There are three types of image multiplication. For details , please refer to " Understanding and Mastering the Three Multiplication Operations in OpenCV ". We only focus on the last one, that is, the multiplication provided by the cv2.multiply function. For two image matrices A and B:
the calculation method of multiplication in this way is as follows:
Second, the syntax of image multiplication cv2.multiply
Calling syntax:
multiply(src1, src2, dst=None, scale=None, dtype=None)
Parameter Description:
The grammatical content of multiplication introduced in the OpenCV manual is interpreted as follows:
- src1: the image array as the multiplicand
- src2: The image array used as the multiplier, the size and type are the same as src1
- dst: Optional parameter, the variable for saving the output result, the default value is None, if it is not None, the output image is saved to the corresponding actual parameter of dst, and its size and channel number are the same as the input image. The depth of the image (that is, the image pixel Digits) is determined by the dtype parameter or the input image
- scale: Optional scaling factor of the result image, that is, multiply the scale on the basis of src1*src2
- mask: Image mask, optional parameter, 8-bit single-channel grayscale image, used to specify the elements of the output image array to be changed, that is, the output image pixels are output only when the corresponding position element of the mask is not 0, otherwise All channel components of the pixel at this position are set to 0
- dtype: optional parameter, the depth of the output image array, that is, the number of bits of a single pixel value of the image (if RGB is represented by three bytes, it is 24 bits).
- Return value: the result image of the multiplication
Three, image multiplication cv2.multiply usage scenarios
Regarding the two input data of src1 and src2, the descriptions of multiplication and addition and subtraction in the OpenCV help document are different. Scalar can be used in addition and subtraction, but in the multiplication, the two must have the same size and type. Use scalar. We refer to the mode of addition and subtraction for verification and explanation.
The source image used in this part of the processing case is a collection of multiple classic images processed by OpenCV. The image is as follows:
because the image is relatively large and it is not good to take a screenshot, it will be forced to be adjusted to 1000*750 when reading the image later.
3.1, when src2 is a scalar
To observe by multiplying an image and a scalar, the code is as follows:
import numpy as np
import cv2
def main():
img = cv2.resize(cv2.imread(r'F:\pic\imgs.jpg'),(1000,750))
imgMultiply = cv2.multiply(img,1.2)
cv2.imshow('img', img)
cv2.imshow('imgMultiply', imgMultiply)
cv2.waitKey(0)
main()
We observe the result of the multiplication through program debugging, set a breakpoint before displaying the picture, and watch the relevant variable data, as shown in the figure:
From the matrix channel value in the yellow part above, it can be seen that after directly multiplying with a constant scalar, the original The last 2 channel values of the pixels of the image matrix are all 0 in the result image. Continue to run and display the image as follows:
you can see that the image is blue, which is consistent with the data of the first blue channel of the three channels above.
3.2, src2 is a quadruple
In the previous introduction of addition and subtraction, it is explained that OpenCV converts scalar operations to four-tuple operations, so here we change the scalar in the above case to four-tuples and then verify. code show as below:
def main():
img = cv2.resize(cv2.imread(r'F:\pic\imgs.jpg'),(1000,750))
imgMultiply = cv2.multiply(img,(1.5,1.5,1.5,1.5))
cv2.imshow('img', img)
cv2.imshow('imgMultiply', imgMultiply)
cv2.waitKey(0)
main()
After tracking and observation, all the pixel channel components are multiplied by 1.5, and the saturation operation mode over 255 is set to 255. The displayed image is as follows:
Compared with the original image above, the brightness is obviously enhanced. In fact, when src2 is set to a quadruple with the same element value, the effect is equivalent to the case where the four elements are set to 1 and the image is set to the element value. We adjust the above code as follows:
def main():
img = cv2.resize(cv2.imread(r'F:\pic\imgs.jpg'),(1000,750))
imgMultiply = cv2.multiply(img,(1,1,1,1),scale=0.5)
cv2.imshow('img', img)
cv2.imshow('imgMultiply', imgMultiply)
cv2.waitKey(0)
main()
Note : There is an optional parameter dst before the zoom factor, so it needs to use the keyword parameter form.
The corresponding result image is as follows:
You can see that the brightness of the image is obviously dimmed, but the contrast does not change.
Combining the verification conditions of 3.1 and 3.2 (the verification environment of the old monkey is OpenCV-Python4.3.0.36, windows version), src2 can be a four-tuple, when we want to use multiply to adjust the image brightness, we can use src2 as a four Tuple, all element values of the quadruple are the brightness factors to be adjusted, and it can also be implemented by using a quadruple with all 1 elements to superimpose the scale adjustment factor. But these two forms are not explained in the OpenCV help document, so although it is feasible now, it is not recommended to use them. If you want to directly adjust the brightness of the image, it is recommended to directly perform arithmetic multiplication of the matrix and the scalar, but you need to pay attention to the saturation operation mode.
3.3. Multiplying the color image array and the mask image array
Use the following code to multiply the color image and the mask image:
def main():
img = cv2.resize(cv2.imread(r'F:\pic\imgs.jpg'),(1000,750))
mask = np.ones(img.shape[:2],np.uint8)
mask[100:200,100:200] = 0
imgMultiply = cv2.multiply(img,mask,scale=0.5)
cv2.imshow('img', img)
cv2.imshow('imgMultiply', imgMultiply)
cv2.waitKey(0)
main()
Error during execution:
"C:\Program Files\Python38\python.exe" F:/study/python/project/cvtest/cvtest.py
Traceback (most recent call last):
File "F:/study/python/project/cvtest/cvtest.py", line 100, in <module>
main()
File "F:/study/python/project/cvtest/cvtest.py", line 94, in main
imgMultiply = cv2.multiply(img,mask,scale=0.5)
cv2.error: OpenCV(4.3.0) C:\projects\opencv-python\opencv\modules\core\src\arithm.cpp:669: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in function 'cv::arithm_op'
It can be seen that this operation cannot be done.
3.4. Multiplying the color image array and the color image array
We use the following code to construct the multiplication operation of two color image arrays. The second image is a matrix image based on the same size and number of channels of the original image, and its element values are only 0 and 1:
def main():
img = cv2.resize(cv2.imread(r'F:\pic\imgs.jpg'),(1000,750))
mask = np.ones(img.shape,np.uint8)
mask[500:650,10:300] = 0
imgMultiply = cv2.multiply(img,mask,scale=0.8)
cv2.imshow('img', img)
cv2.imshow('imgMultiply', imgMultiply)
cv2.waitKey(0)
main()
Screenshot of the result image after execution:
You can see that the lower left corner of the picture is replaced by black. If you want to use white instead, adjust the channel with mask set to 0 to a larger value so that the saturation value after multiplication is 255. Such as:
def main():
img = cv2.resize(cv2.imread(r'F:\pic\imgs.jpg'),(1000,750))
mask = np.ones(img.shape,np.uint8)
mask[500:650,10:300] = 100
imgMultiply = cv2.multiply(img,mask,scale=0.8)
cv2.imshow('img', img)
cv2.imshow('imgMultiply', imgMultiply)
cv2.waitKey(0)
main()
Screenshot of the execution result:
You can see that by multiplying such two images, you can decide which parts of the result image are kept and which parts are cleared.
3.5, color image saturation operation multiplication
If two images are multiplied or even an image is multiplied by itself, because the pixel channel value of the image itself is generally greater than 0 or a number greater than 1, and the image operation is a saturation operation, it will cause either of the two images to be black. In the resulting image, it is all black, and most of the other part of the channel value will greatly increase or even reach saturation.
The code for multiplying the original image above is as follows:
def main():
img = cv2.resize(cv2.imread(r'F:\pic\imgs.jpg'),(1000,750))
imgMultiply = cv2.multiply(img,img)
cv2.imshow('img', img)
cv2.imshow('imgMultiply', imgMultiply)
cv2.waitKey(0)
main()
The resulting image after self-multiplication is as follows:
So this kind of real multiplication of two images is basically meaningless in low-level image processing, and the old ape has not studied in the middle and high-level image processing, it is not clear whether there is a special purpose.
3.6. Overflow processing of color image product
3.6.1, overflow solution
The single channel value of the image is represented by an unsigned byte (numpy.uint8), and its value range is 0-255. If the image pixel channel value exceeds the range, the saturation calculation mechanism of OpenCV will set the channel value less than 0 Set to 255 if it is greater than 255. However, this mechanism may cause a large range of image pixel channel values to be set to 255 or 0 in some cases. For example, from 3.4 above, it can be seen that the product of the real two images causes a large range of whitening due to saturated elements.
In order to solve this problem, the following methods can be used:
- When multiplying, save the image product result to the channel value uint16 or float32 matrix (because float32 can save decimals, so the old monkey recommends float32), there may be two ways, one is to perform the multiplication of the original input image matrix Convert it to a float32 matrix before the operation. The second is that the original input image matrix remains unchanged, and the product is saved to the float32 image matrix. However, the old monkey has not found the second way to achieve in OpenCV-Python, only the verification passed the first Convert the input image to float32;
- The product matrix of the multiplication is normalized. For normalization, please refer to " The Principle of Normalization Function Normalize() in Opencv ".
3.6.2 Algorithm considerations for normalization processing
When using the NORM_MINMAX mode of the normalize function of OpenCV-Python to return the matrix element values to the range of [0,255], the algorithm used by OpenCV is:
The idea of this algorithm is to map the maximum element value in the original matrix to the upper limit of the interval such as 255, and map the smallest element value in the original matrix to the lower limit of the interval such as 0, and other intermediate values are mapped to the difference between this value and the smallest element value in the matrix. The ratio of the difference between the maximum and minimum element values in the original matrix multiplied by the product of the mapping interval plus the lower limit of the mapping interval, that is, the linear transformation of the mapping process is based on the difference between the original element value and the minimum value of the original matrix as the linear transformation. Instead of directly taking the element value as a linear transformation.
Algorithmically, this method is more scientific, but Lao Yuan believes that the situation of image processing is special. If an image is composed of dark images, each channel value of all pixels is relatively large, which will lead to the image When the product is normalized to [0-255], the difference will be small, and the single channel of the image is between [0-255], so the product of two images, especially the product of self-multiplication, theoretically takes 0 as the lower limit Considering that normalization may be more reasonable in certain scenarios, of course, in other normalization scenarios (such as image addition), the NORM_MINMAX mode adopted by OpenCV may be more reasonable.
3.6.3, image multiplication and normalization processing case
For the normalization of OpenCV-Python and the normalization considered by the old monkey, let's compare the normalization effect of the image imgs.jpg after the image is multiplied. The corresponding code is as follows:
def main():
img = cv2.imread(r'F:\pic\imgs.jpg')
img32 = img.astype(np.float32)
imgMultiply = cv2.multiply(img32, img32)
imgNormalizeOpenCV = cv2.normalize(imgMultiply,None,0,255,cv2.NORM_MINMAX) #opencv归一化处理
#老猿的归一化处理
maxv = np.max(imgMultiply)
imgNormalizeLaoyuan = (imgMultiply / maxv) * 255
cv2.imshow('imgNormalizeOpenCV', imgNormalizeOpenCV.astype(np.uint8))
cv2.imshow('imgNormalizeLaoyuan', imgNormalizeLaoyuan.astype(np.uint8))
cv2.waitKey(0)
main()
Let's vertically superimpose the original image and the two normalized images together to compare and see the effect of image self-multiplication and normalization. The first line of the picture below is the original image, the second line is normalized by OpenCV, and the third line is normalized by the old ape.
It can be seen from the above picture comparison that the contrast of the image increases after the self-multiplication normalization process. This is because the difference is enlarged after the self-multiplication process, and the normalization process will not change this trend. At the same time, it can be seen that the two normalized processing images have similar effects. The old ape did not analyze the data carefully. It is almost the same. This is because the image of imgs.jpg exists in both black and white, which causes the actual results of the two algorithms to be processed. basically the same.
4. Summary:
This article introduces in detail the calling syntax of the OpenCV-Python image multiplication operation cv2.multiply function, and analyzes several usage scenarios of OpenCV multiplication and the normalization of image overflow. Through these analyses, we can know that there are three main types of OpenCV image multiplication operations. effect:
- The image and scalar multiplication can adjust the brightness of the image;
- The multiplication of the image and the mask can control the range of the output image;
- Image self-multiplication can adjust the contrast of the image.
For more introduction to OpenCV-Python, please refer to the column "OpenCV-Python Graphics and Image Processing"
Blog address : https://blog.csdn.net/laoyuanpython/category_9979286.html
Paid column about the old ape
Lao Yuan’s paid column "Developing Graphical Interface Python Applications Using PyQt" ( https://blog.csdn.net/laoyuanpython/category_9607725.html ) specifically introduces the basic tutorials for Python-based PyQt graphical interface development, and the paid column "moviepy audio and video development Column" ( https://blog.csdn.net/laoyuanpython/category_10232926.html ) introduces in detail the related methods of moviepy audio and video editing and synthesis and the use of related methods to process related editing and synthesis scenes. Both columns are suitable for certain Novice readers who have basic Python but no relevant knowledge will learn.
Paid column article directory : "moviepy audio and video development column article directory" ( https://blog.csdn.net/LaoYuanPython/article/details/107574583 ), "Use PyQt to develop graphical interface Python application column directory" ( https:// blog.csdn.net/LaoYuanPython/article/details/107580932 ).
For those who lack Python foundation, you can learn Python from scratch through Lao Yuan’s free column "Column: Python Basic Tutorial Directory" ( https://blog.csdn.net/laoyuanpython/category_9831699.html ).
If you are interested and willing to support the readers of Old Ape, welcome to buy paid columns.
