ArUco mark

First of all, what is the aruco mark?

Aruco markers are binary square fiducial markers that can be used for camera pose estimation. Its main advantages are that detection is simple, fast, and robust. An ArUco marker is a square marker consisting of a wide black border and an internal binary matrix that determines its identifier (id). The black border of the aruco marker facilitates its fast detection in images, and the internal binary code is used to identify the marker and provide error detection and correction. The size of the aruco mark determines the size of the internal matrix, for example, a mark with a size of 4x4 consists of 16 binary numbers.

In layman's terms, the aruco mark is actually a kind of code, which is similar to the QR code in our daily life, but due to the different coding methods, the way of storing information, capacity, etc. are different, so in There are also differences at the application level. Since a single aruco mark can provide sufficient correspondence, such as four distinct corners and internal binary codes, aruco marks are widely used to increase the amount of information when mapping from a two-dimensional world to a three-dimensional world, so that it is easy to find two The projection relationship between the three-dimensional world and the three-dimensional world, so as to realize pose estimation, camera correction and other applications.

The ArUco module in OpenCV includes the creation and detection of aruco markers, as well as related APIs for using aruco markers for applications such as pose estimation and camera correction, and also provides marker boards, etc. In this note, we mainly sort out the creation and detection of aruco marks.

First of all, when we create an aruco mark, we need to specify a dictionary first. This dictionary indicates the size, encoding, etc. of the created aruco mark. We use APIgetPredefinedDictionary() to declare the dictionary we use. In OpenCV, a variety of predefined dictionaries are provided. We can check which predefined dictionaries are available through PREDEFINED_DICTIONARY_NAME. And the dictionary name indicates the number and size of the aruco marks of the dictionary, for example, DICT_7X7_50 represents a dictionary containing 50 kinds of 7x7 bit marks.

ArUco Markup Generator

Online aruco marker generator: http://aruco.dgut.top/

(Alternate): https://chev.me/arucogen/

Generating ArUco markers in OpenCV

opencv-python generates aruco markers

After determining the dictionary we need, we can drawMarker()draw the aruco mark through the API, and the meanings of its parameters are as follows:

import cv2
import numpy as np
# 生成aruco标记
# 加载预定义的字典
dictionary = cv2.aruco.Dictionary_get(cv2.aruco.DICT_6X6_250)

# 生成标记
markerImage = np.zeros((200, 200), dtype=np.uint8)
markerImage = cv2.aruco.drawMarker(dictionary, 22, 200, markerImage, 1)
cv2.imwrite("marker22.png", markerImage)

The aruco module of opencv has a total of 25 predefined tag dictionaries. All Aruco marks in each dictionary contain the same number of blocks or bits (e.g. 4×4, 5×5, 6×6 or 7×7), and the number of Aruco marks in each dictionary is fixed (e.g. 50, 100 , 250 or 1000).

cv2.aruco.Dictionary_get()The function loads cv2.aruco.DICT_6X6_250a dictionary of 250 tokens, where each token is a 6×6 bit binary pattern

cv2.aruco.drawMarker(dictionary, 22, 200, markerImage, 1)The second parameter in 22is the tag id of aruco (0~249), the third parameter determines the size of the generated tag, in the above example, it will generate a pixel 200×200image, and the fourth parameter indicates that the aruco tag will be stored object (above markerImage), and finally, the fifth parameter is the border width parameter, which determines how many bits (blocks) should be added as a border to the resulting binary pattern.

After execution, such tags will be generated: the tag ids are22

expand supported tag dictionaries


展开查看的内容;
DICT_4X4_50 
Python: cv.aruco.DICT_4X4_50
DICT_4X4_100 
Python: cv.aruco.DICT_4X4_100
DICT_4X4_250 
Python: cv.aruco.DICT_4X4_250
DICT_4X4_1000 
Python: cv.aruco.DICT_4X4_1000
DICT_5X5_50 
Python: cv.aruco.DICT_5X5_50
DICT_5X5_100 
Python: cv.aruco.DICT_5X5_100
DICT_5X5_250 
Python: cv.aruco.DICT_5X5_250
DICT_5X5_1000 
Python: cv.aruco.DICT_5X5_1000
DICT_6X6_50 
Python: cv.aruco.DICT_6X6_50
DICT_6X6_100 
Python: cv.aruco.DICT_6X6_100
DICT_6X6_250 
Python: cv.aruco.DICT_6X6_250
DICT_6X6_1000 
Python: cv.aruco.DICT_6X6_1000
DICT_7X7_50 
Python: cv.aruco.DICT_7X7_50
DICT_7X7_100 
Python: cv.aruco.DICT_7X7_100
DICT_7X7_250 
Python: cv.aruco.DICT_7X7_250
DICT_7X7_1000 
Python: cv.aruco.DICT_7X7_1000
DICT_ARUCO_ORIGINAL 
Python: cv.aruco.DICT_ARUCO_ORIGINAL
DICT_APRILTAG_16h5 
Python: cv.aruco.DICT_APRILTAG_16h5
4x4 bits, minimum hamming distance between any two codes = 5, 30 codes

Generate aruco marks in batches

import cv2
import numpy as np
# 生成aruco标记
# 加载预定义的字典
dictionary = cv2.aruco.Dictionary_get(cv2.aruco.DICT_6X6_250)

# 生成标记
markerImage = np.zeros((200, 200), dtype=np.uint8)
for i in range(30):
    markerImage = cv2.aruco.drawMarker(dictionary, i, 200, markerImage, 1);

    firename='armark/'+str(i)+'.png'
    cv2.imwrite(firename, markerImage);

A series of 6*6 will be generated under the armark folderaruco标记

Detection and localization of Aruco markers

static detection

Image detection Aruco markers in environment with 7 markers in environment

import numpy as np
import time
import cv2
import cv2.aruco as aruco
#读取图片
frame=cv2.imread('IMG_3739.jpg')
#调整图片大小
frame=cv2.resize(frame,None,fx=0.2,fy=0.2,interpolation=cv2.INTER_CUBIC)
#灰度话
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
#设置预定义的字典
aruco_dict = aruco.Dictionary_get(aruco.DICT_6X6_250)
#使用默认值初始化检测器参数
parameters =  aruco.DetectorParameters_create()
#使用aruco.detectMarkers()函数可以检测到marker，返回ID和标志板的4个角点坐标
corners, ids, rejectedImgPoints = aruco.detectMarkers(gray,aruco_dict,parameters=parameters)
#画出标志位置
aruco.drawDetectedMarkers(frame, corners,ids)


cv2.imshow("frame",frame)
cv2.waitKey(0)
cv2.destroyAllWindows()

For each successful detection of a marker, the four corners of the marker will be detected in the order from top left, top right, bottom right and bottom left. In C++, store these 4 detected corners as a point vector, and store multiple markers in the image together in a point vector container. In Python, they are stored as Numpy arrays.

detectMarkers()Function to detect and determine the position of the marker corners.

The first argument imageis an image of the scene with markers.

The second parameter dictionaryis a dictionary used to generate tokens. Successfully detected markers will be stored in markerCorners and their IDs in markerIds. A previously initialized DetectorParameters object is passed as a parameter.

The third parameter parameters: DetectionParametersthe object of the class, which includes all parameters that can be customized during the detection process;

Return parameter corners: list of corners of the detected aruco markers, for each marker its four corners are returned in their original order (clockwise starting from the upper right corner), the first corner is the upper right corner, then the right Bottom, bottom left and top left.

Return ids: the id of each detected tag, it should be noted that the third parameter and the fourth parameter have the same size;

Return parameter rejectedImgPoints: list of discarded candidate markers, i.e. squares that were detected but did not provide a valid encoding. Each candidate token is also defined by its four corners, and its format is the same as the third parameter, which can be omitted if there is no special requirement.

corners, ids, rejectedImgPoints = aruco.detectMarkers(gray,aruco_dict,parameters=parameters)

When we detect the aruco tag, in order to facilitate observation, we need to perform visualization operations and mark the tag: use drawDetectedMarkers（）this API to draw the detected aruco tag, and the meanings of its parameters are as follows:

Parameter image: is the input/output image that will draw the marker (usually the image where the marker is detected)

Parameter corners: list of corners detected by aruco markers

Parameter ids: Each detected marker corresponds to the id in the dictionary to which it belongs, optional (if not provided) the ID will not be drawn.

Parameter borderColor: The color of the border of the marker, and the rest of the colors (text color and first corner color) will be calculated based on this color to improve the visualization effect.

no return value

aruco.drawDetectedMarkers(image, corners,ids,borderColor)

Effect demonstration:

dynamic detection

Use the camera to perform a real-time dynamic monitoring of aruco markers and estimate the pose. The internal parameters of the camera need to be calibrated in advance. For how to calibrate, please see my other article

import numpy as np
import time
import cv2
import cv2.aruco as aruco



# mtx = np.array([
#         [2946.48,       0, 1980.53],
#         [      0, 2945.41, 1129.25],
#         [      0,       0,       1],
#         ])
# #我的手机拍棋盘的时候图片大小是 4000 x 2250
# #ip摄像头拍视频的时候设置的是 1920 x 1080，长宽比是一样的，
# #ip摄像头设置分辨率的时候注意一下
#
#
# dist = np.array( [0.226317, -1.21478, 0.00170689, -0.000334551, 1.9892] )


#相机纠正参数

# dist=np.array(([[-0.51328742,  0.33232725 , 0.01683581 ,-0.00078608, -0.1159959]]))
#
# mtx=np.array([[464.73554153, 0.00000000e+00 ,323.989155],
#  [  0.,         476.72971528 ,210.92028],
#  [  0.,           0.,           1.        ]])
dist=np.array(([[-0.58650416 , 0.59103816, -0.00443272 , 0.00357844 ,-0.27203275]]))
newcameramtx=np.array([[189.076828   ,  0.    ,     361.20126638]
 ,[  0 ,2.01627296e+04 ,4.52759577e+02]
 ,[0, 0, 1]])
mtx=np.array([[398.12724231  , 0.      ,   304.35638757],
 [  0.       ,  345.38259888, 282.49861858],
 [  0.,           0.,           1.        ]])



cap = cv2.VideoCapture(0)


font = cv2.FONT_HERSHEY_SIMPLEX #font for displaying text (below)

#num = 0
while True:
    ret, frame = cap.read()
    h1, w1 = frame.shape[:2]
    # 读取摄像头画面
    # 纠正畸变
    newcameramtx, roi = cv2.getOptimalNewCameraMatrix(mtx, dist, (h1, w1), 0, (h1, w1))
    dst1 = cv2.undistort(frame, mtx, dist, None, newcameramtx)
    x, y, w1, h1 = roi
    dst1 = dst1[y:y + h1, x:x + w1]
    frame=dst1


    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    aruco_dict = aruco.Dictionary_get(aruco.DICT_6X6_250)
    parameters =  aruco.DetectorParameters_create()
    dst1 = cv2.undistort(frame, mtx, dist, None, newcameramtx)
    '''
    detectMarkers(...)
        detectMarkers(image, dictionary[, corners[, ids[, parameters[, rejectedI
        mgPoints]]]]) -> corners, ids, rejectedImgPoints
    '''

    #使用aruco.detectMarkers()函数可以检测到marker，返回ID和标志板的4个角点坐标
    corners, ids, rejectedImgPoints = aruco.detectMarkers(gray,aruco_dict,parameters=parameters)

#    如果找不打id
    if ids is not None:

        rvec, tvec, _ = aruco.estimatePoseSingleMarkers(corners, 0.05, mtx, dist)
        # 估计每个标记的姿态并返回值rvet和tvec ---不同
        # from camera coeficcients
        (rvec-tvec).any() # get rid of that nasty numpy value array error

#        aruco.drawAxis(frame, mtx, dist, rvec, tvec, 0.1) #绘制轴
#        aruco.drawDetectedMarkers(frame, corners) #在标记周围画一个正方形

        for i in range(rvec.shape[0]):
            aruco.drawAxis(frame, mtx, dist, rvec[i, :, :], tvec[i, :, :], 0.03)
            aruco.drawDetectedMarkers(frame, corners)
        ###### DRAW ID #####
        cv2.putText(frame, "Id: " + str(ids), (0,64), font, 1, (0,255,0),2,cv2.LINE_AA)


    else:
        ##### DRAW "NO IDS" #####
        cv2.putText(frame, "No Ids", (0,64), font, 1, (0,255,0),2,cv2.LINE_AA)


    # 显示结果框架
    cv2.imshow("frame",frame)

    key = cv2.waitKey(1)

    if key == 27:         # 按esc键退出
        print('esc break...')
        cap.release()
        cv2.destroyAllWindows()
        break

    if key == ord(' '):   # 按空格键保存
#        num = num + 1
#        filename = "frames_%s.jpg" % num  # 保存一张图像
        filename = str(time.time())[:10] + ".jpg"
        cv2.imwrite(filename, frame)