Deep Learning: Summary and Examples of OpenCV Basic Methods

---

1. Introduction to Machine Vision

​ The machine vision we are talking about now generally refers to computer vision. In simple terms, it is to study how to make machines understand things, and use cameras and computers instead of human eyes to identify, track and measure objects. Vision, and further graphics processing, make computer processing an image that is more suitable for human observation or transmission to instrument detection. At present, there is no big difference between the two.

​ First of all, in many literatures, computer vision and machine vision are not distinguished, but in fact these two terms are both different and related. Computer vision is a combination of image processing, pattern recognition, and artificial intelligence technology, focusing on computer analysis of one or more images. Machine vision focuses on the engineering of computer vision technology, which can automatically acquire and analyze specific images to control corresponding behaviors.

1.1 Application of machine vision

Object recognition, human body recognition, vehicle detection, OCR (Optical Character Recognition, optical character recognition)

2. Introduction to OpenCV

OpenCV is a cross-platform computer vision C++ class composition based on the Apache2.0 license (open source). It also provides interfaces for languages ​​such as Python, Ruby, and MATLAB, and implements many general algorithms for image recognition and computer vision.

OpenCV is written in C++ language, it has C++, Python , Java , and MATLAB interfaces, and supports Windows, Linux, Android, and Mac OS, OpenCV is mainly inclined to real-time vision applications, and utilizes MMX and SSE instructions when available, and now also Provide support for C#, Ch, Ruby, GO. [2]

3. OpenCV environment installation

3.1 Installation steps

First, pip install virtualenv installs the package to create a virtual environment

Create a virtual environment such as virtualenv venv in the folder you specify

Enter the cmd environment in venv/Scripts and enter activate to activate the virtual environment

Execute the following statement in the activated virtual environment:

pip install opencv-python==3.4.11.45 -i https://pypi.douban.com/simple

pip install opencv-contrib-python==3.4.11.45 -i https://pypi.douban.com/simple

pip install jupyter -i https://pypi.douban.com/simple

pip install matplotlib -i https://pypi.douban.com/simple

At this point, the necessary environment is installed

[Install the lower version of OpenCV because some algorithms in the higher version have applied for patents]

Or go to the OpenCV official website to download the corresponding .exe file for execution.

3.2 Installation test

Also enter the cmd environment in the virtual environment folder

Enter activate to activate the virtual environment

Enter ipython to enter the python environment

import cv2
cv2.__version__
# 获得打印结果 是安装时选择的版本号 
# 此处我的电脑安装版本为 3.4.11

4. Create a display window

4.1 Method introduction

• namedWindow()
• imshow()
• destroyAllwindows()
• resizeWindow()
• waitKey()

4.2 Simple case

import cv2
# 创建窗口
# cv2.WINDOW_AUTOSIZE 大小自动 不允许开发者修改
# cv2.namedWindow('window',cv2.WINDOW_AUTOSIZE)
cv2.namedWindow('window',cv2.WINDOW_NORMAL)
# 更改窗口大小
cv2.resizeWindow('window',800,600)
# 展示窗口
cv2.imshow('window',0)
# 等待按键
# waitKey会返回按键的ascii的值
# 0表示接受任意按键 如果给其他的整数，则表示等待按键的时间。
key = cv2.waitKey(0)
if key & 0xFF == ord('q'):
    cv2.destroyAllWindows()
# 若接收到按键q 则销毁所有窗口

Simple solid black window that closes after hitting the keyboard key 'q'

5. Window display image

5.1 Method introduction

• imread(path) - read the file at the specified path

5.2 Simple case

# 导包
import cv2
import matplotlib.pyplot as plt
import numpy as np

# 读取图片
img = cv2.imread('img/dog.jpg')
# 显示图片
plt.imshow(img)

The channel that OpenCV reads the picture is not the default RGB, but BGR, so there is a color difference between the picture displayed by matplotlib.pyplot and the original picture.

Therefore, after using OpenCV to read the picture, it is also necessary to use OpenCV to display the picture, and modify it to the following code:

# 导包
import cv2
import matplotlib.pyplot as plt
import numpy as np

# 读取图片
img = cv2.imread('img/dog.jpg')
# 显示图片
plt.imshow(img)
cv2.imshow('img',img)
key = cv2.waitKey(0)
if key & 0xFF == ord('q'):
    cv2.destroyAllWindows()

The picture is shown in normal color:

Encapsulate the image display as a function for later calling.

import cv2
def cv_show(name, img):
    cv2.imshow(name,img)
    key = cv2.waitKey(0)
    if key & 0xFF == ord('q'):
        cv2.destroyAllWindows()

6. Save pictures in the window

6.1 Method introduction

• imwrite(path, img) ——write image img to the specified path

6.2 Simple case

import cv2
cv2.namedWindow('img',cv2.WINDOW_NORMAL)
cv2.resizeWindow('img',320,240)
img = cv2.imread('img/dog.jpg')

while True:
    cv2.imshow('img',img)
    key = cv2.waitKey(0)
    if key & 0xFF == ord('q'):
        print('程序中止')
        break
    if key & 0xFF == ord('s'):
        cv2.imwrite("./dog.png",img)
        print('dog.png已保存')
    else:
        print(key)
cv2.destroyAllWindows()

After hitting the keyboard 's' we saved the dog.png image in the file system

And the program window is closed after hitting the keyboard 'q'.

7. Video capture (local video + calling camera equipment)

7.1 Method introduction

• VideoCapture(0) - Capture the camera, using numbers to represent different devices, such as 0 or 1
• VideoCapture(video) - capture local video resources

7.2 Simple case

#尝试打开mp4文件
import cv2
vc = cv2.VideoCapture('video/0.mp4')
if vc.isOpened():
    # 每次读取一帧 视频则是由连续的帧组成
    open,frame = vc.read()
else:
    open = False

If open is True, it proves that the video file is successfully opened, and frame is the image format to be read.

# -*- coding: <encoding name> -*- : # -*- coding: utf-8 -*-

import cv2
cv2.namedWindow('video',cv2.WINDOW_NORMAL)
cv2.resizeWindow('video',640, 480)

# 对cap的赋值语句只能够保存一行

# 读取摄像头设备
#cap = cv2.VideoCapture(0)

# 播放video/0.mp4文件
cap = cv2.VideoCapture('video/0.mp4')

# 循环读取
while True:
    ret, frame = cap.read()
    if not ret:
        break
    cv2.imshow('video',frame)
    # 此处不能使用0作为等待参数 因为0会导致无限等待！
    key = cv2.waitKey(1)
    if key & 0xFF == ord('q'):
        break;
    if key & 0xFF == ord('s'):
        cv2.imencode('.png', frame)[1].tofile('img/截图.png') 
        print('截图已保存')
# 记得释放资源
cap.release() 
cv2.destroyAllWindows()

If you hit the keyboard 's', it will take a screenshot of the frame of the video playback at the moment or the picture captured by the camera

The file save address is under the img folder of the current directory, and the picture name is Screenshot.png

8. Record video

8.1 Method introduction

• VideoWriter: parameter one is output file address parameter two is multimedia file format parameter three is frame rate parameter four is resolution
• write encodes and writes to cache
• release cache content is written to disk, and the disk is released

8.2 Simple case

import cv2
# 调用摄像头设备
cap = cv2.VideoCapture(0)
# *mp4v是解包操作 = ['m','p','4','v']
# 选定文件格式mp4
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
vw = cv2.VideoWriter('video/output.mp4',fourcc,20,(640,480))
# 事件循环
while cap.isOpened():
    # 读取摄像头内容
    ret, frame = cap.read()
    # 如果未能读取成功 则证明视频录制未在进行工作
    if not ret:
        print('can not recive frame, Exiting...')
        break
    # 将读取的画面写入缓存区中
    vw.write(frame)
    # 将摄像头抓拍内容显示屏幕中
    cv2.imshow('frame',frame)
    # 按键q则停止录制
    if cv2.waitKey(1) == ord('q'):
        break
# 记得释放资源 否则内容无法有效保存至内存中
cap.release() 
vw.release()
cv2.destroyAllWindows()

The output.mp4 file is saved under the video path, which is also the video just recorded.

9. Mouse Behavior

9.1 Method introduction

• setMouseCallback(winname,callback,userdata)

• Set the mouse event (window name, callback function, callback function parameters)

• callback(event,x,y,flags,uesrdata)

• Event has a fixed selection and is the type of mouse event.

• X and Y are the coordinates of the mouse, in pixels

• flags are used to combine keys

• uesrdata corresponds to the parameter with the same name in setMouseCallback

The type of mouse event Event:

constant	variable name	mouse behavior
1	EVENT_MOUSEMOVE	mouse movement
2	EVENT_LBUTTONDOWN	Click the left mouse button
3	EVENT_RBUTTONDOWN	Right click
4	EVENT_MBUTTONDOWN	middle mouse button click
5	EVENT_LBUTTONUP	release the left mouse button
6	EVENT_RBUTTONUP	release the right mouse button
7	EVENT_MBUTTONUP	release middle mouse button
8	EVENT_LBUTTONDBLCLK	Double click the left mouse button
9	EVENT_RBUTTONDBLCLK	Double click the right mouse button
10	EVENT_MBUTTONDBLCLK	Double click the middle mouse button
11	EVENT_MOUSEHWHEEL	scroll mouse left and right

The type of combination button flags:

constant	variable name	key event
1	EVENT_FLAG_LBUTTON	press the left button
2	EVENT_FLAG_RBUTTON	right click
4	EVENT_FLAG_MBUTTON	press middle button
8	EVENT_FLAG_CRTLKEY	press the ctrl key
16	EVENT_FLAG_SHIFTKEY	press the shift key
32	EVENT_FLAG_ALTKEY	press the alt key

9.2 Simple case

# 导包
import cv2
import numpy as np


# 生成全黑的图片 用作初始背景
img = np.zeros((360,640,3),np.uint8)


# 创建名为‘mouse’的窗口
cv2.namedWindow('mouse',cv2.WINDOW_NORMAL)
cv2.resizeWindow('mouse', 640, 360)

# 设置鼠标回调函数
cv2.setMouseCallback('mouse',mouse_callback,'2')

# 鼠标事件 
def mouse_callback(event, x, y, flags, userdata):
    # 如果鼠标发生事件4所代表的行为 则执行下列函数
    if event == 4:
        cv2.destroyAllWindows()
        # 读取img文件路径下的dog.jpg
        img = cv2.imread('img/dog.jpg')
        # 重新创建名为dog的窗口
        cv2.namedWindow('dog',cv2.WINDOW_NORMAL)
        cv2.resizeWindow('dog', 640, 360)
         # 事件循环 展示dog图片
        while True:
            cv2.imshow('dog',img)
            key = cv2.waitKey(1)
            #'q'则退出
            if key & 0xFF == ord('q'):
                break
        cv2.destroyAllWindows()
    
# 显示窗口和背景
while True:
    cv2.imshow('mouse',img)
    key = cv2.waitKey(1)
    #'q'则退出
    if key & 0xFF == ord('q'):
        break
cv2.destroyAllWindows()

1. Run the program

2. Trigger mouse events

3. Trigger the key event window to close

10. Draggable control TrackBar

10.1 Method introduction

• createTrackbar(trackbarname,winname,value,count,onChange)
• Create a TrackBar control value as the default value of the trackbar count as the maximum value of the bar, the minimum value is 0
• getTrackbarPos(trackbarname, winname) Get the current value of TrackBar

10.2 Simple case

## trackbar
import cv2
import numpy as np

# 创建窗口 
cv2.namedWindow('trackbar', cv2.WINDOW_NORMAL)
cv2.resizeWindow('mouse',640,480)

# 定义回调函数
def callback(value):
    print(value)

# 创建trackbar
cv2.createTrackbar('R','trackbar',0,255,callback)
cv2.createTrackbar('G','trackbar',0,255,callback)
cv2.createTrackbar('B','trackbar',0,255,callback)

# 创建一个背景图片
img  = np.zeros((480,640,3),np.uint8)

# 事件循环
while True:
    # 获取当前trackbar的值
    r = cv2.getTrackbarPos('R','trackbar')
    g = cv2.getTrackbarPos('G','trackbar')
    b = cv2.getTrackbarPos('B','trackbar')
    
    # 改变背景图片颜色
    # 全部行列(所有像素点)均修改为getTrackbarPos()的值
    img[:] = [b, g, r]
    # OpenCV的色彩空间为BGR
    cv2.imshow('trackbar',img)
    
    key = cv2.waitKey(1)
    if key & 0xFF == ord('q'):
        break
cv2.destroyAllWindows()

1. Run the program

2. Drag the control TrackBar to observe the real-time color change

10.3 Supplement: Color Spaces

• The human eye used to recognize colors and the common color space is RGB
• But OpenCV uses BGR by default; the difference between BGR and RGB color spaces is that the pictures are arranged in different order on the color channels.