Paper frame image extraction, straighten and fill detect whether -python

Foreword

Front also share some way some image processing opencv, then I introduce to extract a contract, or paper documents with opencv border today and remove the background, the image straighten practice, and also based on this idea, introduced to check whether fill or an idea signature. Man of few words said, look at the renderings realized (the picture is boring when I scribbled paper, escape ~), the specific code I would put on my github https://github.com/Wangzg123/fileclipper, you can go to download

First, let's examine tell us about the steps to achieve
1, the image is converted to a fixed height of the picture (this step is to adapt the threshold edge detection, the following will be introduced)
2, extracted edge with edge detection algorithm opencv of
3 to find closure and calculating the area of their profile, to extract the maximum area
4, the four vertices of a quadrilateral fitting the maximum area of
5, the edges of a rectangular sheet fitted by change in perspective of opencv

First, edge detection

This step by Canny edge extraction function opencv will come out (specific features, but you own review), it is worth mentioning that before the canny to prevent some noise to be blurred by the Gaussian noise, and also expanded the edge of the image easier to close

import cv2
import numpy as np


# 固定尺寸
def resizeImg(image, height=900):
    h, w = image.shape[:2]
    pro = height / h
    size = (int(w * pro), int(height))
    img = cv2.resize(image, size)
    return img


# 边缘检测
def getCanny(image):
    # 高斯模糊
    binary = cv2.GaussianBlur(image, (3, 3), 2, 2)
    # 边缘检测
    binary = cv2.Canny(binary, 60, 240, apertureSize=3)
    # 膨胀操作，尽量使边缘闭合
    kernel = np.ones((3, 3), np.uint8)
    binary = cv2.dilate(binary, kernel, iterations=1)
    return binary

path = r'C:\Users\wzg\Desktop\test.jpg'
outpath = r'C:\Users\wzg\Desktop\getCanny.jpg'
img = cv2.imread(path)
img = resizeImg(img)
print('shape =', img.shape)
binary_img = getCanny(img)
cv2.imwrite(outpath, binary_img)

# output:    shape = (900, 420, 3)

Second, identify the edges of the paper

By findContours fit all the contours, and then find the maximum of the profile that is the paper's edge (because of filming, our goal is generally the biggest graphic)

# 代码承接上文
# 求出面积最大的轮廓
def findMaxContour(image):
    # 寻找边缘
    _, contours, _ = cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
    # 计算面积
    max_area = 0.0
    max_contour = []
    for contour in contours:
        currentArea = cv2.contourArea(contour)
        if currentArea > max_area:
            max_area = currentArea
            max_contour = contour
    return max_contour, max_area


path = r'C:\Users\wzg\Desktop\test.jpg'
outpath = r'C:\Users\wzg\Desktop\findMaxContour.jpg'
img = cv2.imread(path)
img = resizeImg(img)
binary_img = getCanny(img)
max_contour, max_area = findMaxContour(binary_img)
cv2.drawContours(img, max_contour, -1, (0, 0, 255), 3)
cv2.imwrite(outpath, img)

Third, find out the four vertices of a quadrilateral

Fitting the four vertexes of a quadrangle maximum area

# 代码承接上文
# 多边形拟合凸包的四个顶点
def getBoxPoint(contour):
    # 多边形拟合凸包
    hull = cv2.convexHull(contour)
    epsilon = 0.02 * cv2.arcLength(contour, True)
    approx = cv2.approxPolyDP(hull, epsilon, True)
    approx = approx.reshape((len(approx), 2))
    return approx

path = r'C:\Users\wzg\Desktop\test.jpg'
outpath = r'C:\Users\wzg\Desktop\getBoxPoint.jpg'
img = cv2.imread(path)
img = resizeImg(img)
binary_img = getCanny(img)
max_contour, max_area = findMaxContour(binary_img)
boxes = getBoxPoint(max_contour)
for box in boxes:
   cv2.circle(img, tuple(box), 5, (0, 0, 255), 2)
print(boxes)
cv2.imwrite(outpath, img)

Fourth, perspective transformation

Thus far, we are looking for the four vertices on the image after resize, we must also be mapped back to the original point based on these four points. And the image we obtain is a trapezoidal shape, a rectangular shape when we have to change by change in perspective.

# 代码承接上文
# 适配原四边形点集
def adaPoint(box, pro):
    box_pro = box
    if pro != 1.0:
        box_pro = box/pro
    box_pro = np.trunc(box_pro)
    return box_pro


# 四边形顶点排序，[top-left, top-right, bottom-right, bottom-left]
def orderPoints(pts):
    rect = np.zeros((4, 2), dtype="float32")
    s = pts.sum(axis=1)
    rect[0] = pts[np.argmin(s)]
    rect[2] = pts[np.argmax(s)]
    diff = np.diff(pts, axis=1)
    rect[1] = pts[np.argmin(diff)]
    rect[3] = pts[np.argmax(diff)]
    return rect


# 计算长宽
def pointDistance(a, b):
    return int(np.sqrt(np.sum(np.square(a - b))))


# 透视变换
def warpImage(image, box):
    w, h = pointDistance(box[0], box[1]), \
           pointDistance(box[1], box[2])
    dst_rect = np.array([[0, 0],
                         [w - 1, 0],
                         [w - 1, h - 1],
                         [0, h - 1]], dtype='float32')
    M = cv2.getPerspectiveTransform(box, dst_rect)
    warped = cv2.warpPerspective(image, M, (w, h))
    return warped

path = r'C:\Users\wzg\Desktop\test.jpg'
outpath = r'C:\Users\wzg\Desktop\result.jpg'
image = cv2.imread(path)
ratio = 900 / image.shape[0]
img = resizeImg(image)
binary_img = getCanny(img)
max_contour, max_area = findMaxContour(binary_img)
boxes = getBoxPoint(max_contour)
boxes = adaPoint(boxes, ratio)
boxes = orderPoints(boxes)
# 透视变化
warped = warpImage(image, boxes)
cv2.imwrite(outpath, warped)

V. Other

Thus far, our paper edge detection, straightened has been completed, you can know that you can achieve the above functions by opencv, I have here a solution to fill in some of the check whether the required number of options thinking. As the following two chart, we want to detect whether red box office signed his name.

Solutions like this:
1, the two contracts is extracted through my above algorithm the paper being quadrilateral
2, (such as the top left and right corners of the rectangle marked in the position of a blank contract by setting a preset point detection area contract )
3, cut out the marked area (the same pixel) and do binary processing (matrix into only 0 and 1)
4, because we wrote and did not write the contract pixel values are different, so such a check is greater than threshold, then signed his name, and vice versa
to solve the effect can be represented as follows (I have not posted the code, escape ~)

to sum up

The above description of how to extract the edges of the paper (in fact, regardless of the white paper can be as long as two objects can be) by the method of the opencv, then straighten the perspective changes, and finally put Solutions detecting whether a signature to fill. This article specific code can be downloaded on my github https://github.com/Wangzg123/fileclipper, I am limited, welcome the god of active discussion or give me advice (private letter to me)