LBP
LBP refers to the local binary pattern, English full name: Local Binary Pattern, is an operator used to describe the local characteristics of the image, LBP features have significant advantages such as gray invariance and rotation invariance. It was proposed by T. Ojala, M.Pietikäinen, and D. Harwood in 1994. Due to the simple calculation and better effect of LBP features, LBP features have been widely used in many fields of computer vision. LBP features are more famous The application is used in face recognition and target detection. In the computer vision open source library Opencv, there is an interface for using LBP features for face recognition. There is also a method of using LBP features to train target detection classifiers. Opencv realizes the calculation of LBP features. , but does not provide a separate interface for calculating LBP features.
The original LBP operator is defined in the neighborhood of pixels 3 by 3, with the center pixel of the neighborhood as the threshold, the gray value of the adjacent 8 pixels is compared with the pixel value of the neighborhood center, if the surrounding pixels are greater than the center pixel value, Then the position of the pixel is marked as 1, otherwise it is 0. In this way, the 8 points in the 3 3 neighborhood can be compared to generate an 8-bit binary number, and these 8-bit binary numbers are arranged in turn to form a binary number. This binary number is the LBP value of the central pixel. There are 256 values. The LBP value of the center pixel reflects the texture information of the area around the pixel.
Remarks: The image for calculating LBP features must be a grayscale image. If it is a color image, it needs to be converted into a grayscale image first.
def LBP(srcimg):
'''
:param src:图像
:return:
'''
src = cv2.cvtColor(srcimg, cv2.COLOR_RGB2GRAY)
height = src.shape[0]
width = src.shape[1]
# dst = np.zeros([height, width], dtype=np.uint8)
dst = src.copy()
lbp_value = np.zeros((1,8), dtype=np.uint8)
neighbours = np.zeros((1,8), dtype=np.uint8)
for x in range(1, width-1):
for y in range(1, height-1):
neighbours[0, 0] = src[y - 1, x - 1]
neighbours[0, 1] = src[y - 1, x]
neighbours[0, 2] = src[y - 1, x + 1]
neighbours[0, 3] = src[y, x - 1]
neighbours[0, 4] = src[y, x + 1]
neighbours[0, 5] = src[y + 1, x - 1]
neighbours[0, 6] = src[y + 1, x]
neighbours[0, 7] = src[y + 1, x + 1]
center = src[y, x]
for i in range(8):
if neighbours[0, i] > center:
lbp_value[0, i] = 1
else:
lbp_value[0, i] = 0
lbp = lbp_value[0, 0] * 1 + lbp_value[0, 1] * 2 + lbp_value[0, 2] * 4 + lbp_value[0, 3] * 8 \
+ lbp_value[0, 4] * 16 + lbp_value[0, 5] * 32 + lbp_value[0, 6] * 64 + lbp_value[0, 0] * 128
dst[y, x] = lbp
return dst
LTP（2007）
The authors make three main contributions: (i) propose a simple and effective preprocessing chain that removes most of the effects of illumination changes while still preserving the essential appearance details required for recognition; ( ii) introduce local three-dimensional meta-pattern (LTP), which is a generalization of the Local Binary Pattern (LBP) local texture descriptor, is more discriminative in homogeneous regions and less sensitive to noise; (iii) uses similarity based on local distance transform The metric replaces the local history map to further improve the performance of LBP/LTP based face recognition.
LTP extends LBP to ternary codes, i.e. local ternary patterns, where the gray quantization in the region of width ±t around the ic is zero, the gray quantization above this region is +1, and the gray quantization below this region is −1, that is, the indicator s(u) is replaced by a three-valued function:
replace the binary LBP code with the ternary LTP code. Here, t is a user-specified threshold (so LTP codes are more robust to noise, but are no longer strictly invariant to grayscale transformations). The LTP encoding process is shown in FIG. 2 . Here the threshold t is set to 5, so the tolerance interval is [49, 59].
The figure below shows an example image of a person from the Extended Yale-B frontal database. These columns give images for subsets 1 to 5, respectively. (a) Input image. (b) The corresponding illumination normalized image in the preprocessing chain.
References:
Tan X , Triggs B . Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions[J]. Amfg, 2007, 4778(6):1635-1650.
CLBP (2010)
The Complete Local Binary Patterns (CLBP) proposes three local texture description operators to express texture: window gray level difference description operator (CLBP-Sign, CLBP_S), window gradient difference description operator (CLBP- Magnitude, CLBP_M) and the center pixel description operator (CLBP-Center, CLBP_C). Its calculation method is as follows:
Among them, 
N is the number of sub-windows divided by the image. It can be seen from the formula: CLBP_SP, R is LBP in the traditional sense; CLBP_MP, R describes the gradient difference information of the local window by comparing the gray difference amplitude of two pixels with the mean value of the global gray difference amplitude , as the complementary information of CLBP_SP, R; CLBP_CP, R is the gray level information of the response center pixel. Compared with the traditional LBP and its variants, the CLBP_SMC composed of the three description operators can describe the texture more finely, and the recognition accuracy of the texture has been greatly improved.
The analysis of the CLBP algorithm shows that when the gray gradient distribution of the image is uneven due to uneven illumination, that is, compared with most areas of the image, when a small part of the area changes sharply or gently, CLBP_MP, R will show its limitations. [Not sensitive to light, the texture information of the smooth part of the image will be missed]
参考文献：
GUO Z, ZHAN GL, ZHANG D. A completed modeling of local binary pattern operator for texture classification [J]. IEEE Transactions on Image Processing, 2010, 19(6):1657-1663
ICLBP（2014）
On the basis of the CLBP algorithm, the CLBP algorithm is further improved, and a gray gradient difference feature description operator ICLBP_TP, R is proposed, which uses the gray difference amplitude of two pixels and the gray difference amplitude of the window The calculation method is as follows:
Among them, 
the four texture description operators that the ICLBP algorithm will generate are: ICLBP_SP, R, ICLBP_MP, R, ICLBP_CP, R, ICLBP_TP, R, among them, ICLBP_SP, R by comparing two The gray value of the pixel describes the gray difference characteristics of the local window, which is consistent with CLBP_SP, R; ICLBP_MP, R describes the local window by comparing the gray difference amplitude of two pixels with the average value of the gray difference amplitude of the window. Gray gradient difference feature; ICLBP_CP, R describes the gray value difference feature of the global image by comparing the window gray mean value with the
image value; ICLBP_TP, R compares the window gray level difference amplitude mean with the image global gray value The mean size of the degree difference magnitude describes the gradient difference characteristics of the global image.
Compared with the original CLBP algorithm, the improved ICLBP algorithm has better fineness and robustness in describing image texture, and the image edge is clearer, and the texture is clearer and finer
. References: Liu Hao, Yang Yongquan, Guo Xiancao, Improved LBP algorithm for texture feature extraction[J]. Computer Engineering and Applications, 2014,50(6):182-185,245. LIU Hao,
YANG Yongquan, GUO Xiancao, et al. Improved LBP used for texture feature extraction [J]. Computer Engineering and Applications, 2014,50(6):182-185,245.
FLBP（2008）
The existing LBP first subtracts the gray value of the neighborhood pixel and the gray value of the central pixel. If the result is positive, the local binary code of the pixel is set to 1, and if the result is negative, it is set to 0. The threshold of this algorithm is fixed, and it only depends on the difference in the gray value of the central pixel and the neighboring pixels. Therefore, this algorithm may obtain the same local binary code in different local spatial structure modes, as shown in the figure below.
In order to enhance the discrimination of local edge features by basic LBP, fuzzy mathematics is combined with basic LBP. When converting binary codes, it is first necessary to measure the gray difference between the central pixel and neighboring pixels, and then calculate the degree of membership of the neighboring pixels relative to the central pixel, and determine the local binary code value based on the degree of membership. As shown in Figure 2, when the membership degree is higher than 0.5, the local binary code is determined to be 1, otherwise it is determined to be 0.
The fuzzy local binary mode can greatly improve the problem that it is basically impossible to distinguish the local structural features, and the optimized LBP formula is written:
where, the local binary code S′ is defined as
μA(u)* is the membership degree of the neighborhood pixel relative to the central pixel, and the formalization can be written as
In the formula: Gmax and Gmin respectively represent the maximum and minimum gray value in the area of radius R.
FLBP improves the robustness to noise, but it is computationally intensive and difficult to extend to multi-scale analysis.
Publications:
Iakovidis DK,Keramidas EG,Maroulis D. Fuzzy Local Binary Patterns for Ultrasound Texture Characterization(J). Image Analysis and Recognition,2008:750-759.
ALBP（2010）
Because LBP cannot fully represent the local spatial features of the image, and cannot express the directional information of the texture image, Guo Zhenhua et al. proposed in 2010 to use directional statistics for the rotation-invariant texture classification, using the mean and standard deviation to define each The local difference distribution of a pixel and its surrounding pixels in each direction. In order to reduce the statistical error of local differences, an adaptive LBP method (ALBP) using least square estimation (LSE) is developed. Moreover, the ALBP method using LSE can bring additional information to the classification and improve the classification accuracy.
Adaptive LBP is a rotation-invariant texture classification method that includes orientation statistics. The distribution of local differences between each pixel and
its surrounding pixels is shaped by its mean and standard deviation. Give the gray value of a central pixel point g. , the gray value of its P neighboring points is gp, the value of P is from 0 to P-1, and the distribution of the difference between gc and gp varies with P. Figure 3.7 is an example where two texture images have similar LBP distributions, but their local difference |gc-gp| is different.
The ALBP method can also be extended to a multi-scale method.
References:
Guo ZH, Zhang L, Zhang D, Zhang S. Rotation invariant texture classification using adaptive LBP with directional statistical features. In: Proceedings of the 17th IEEE International Conference on Image Processing. Hong Kong, China: IEEE, 2010. 285 −288Song
Kechen,Yan Yunhui,Chen Wenhui,Zhang Xu.Research and Prospect of Local Binary Model Method[J].Acta Automatica Sinica,2013,39(06):730-744.
NTLBP（2012）
The NTLBP operator proposed by Fathi A et al. uses a circular majority voting filter and a suitable rotation invariant marking method to obtain more regular uniform and non-uniform patterns, which have better recognition ability and stronger noise robustness sex.
The authors design a new labeling scheme for the LBP operator, which can describe patterns in a rotation-invariant manner and is more compatible with the pattern distribution in real images. This method tries to use all non-uniform patterns with four bit transitions (v=4). Because for the P-bit LBP mode, this group of modes has at least two "1"s (00100010 for P=8), and at most P-2 "1"s (11011101 for P=8) using 1 minus "1" " as the index for each pattern in this class. Thus, in this set of non-uniform patterns, as with the uniform patterns, the authors categorized all existing patterns in the P-3 storage bin according to the number of '1' bits in the pattern. Also, for other non-uniform patterns, the authors use one label for each group with the same bit shift (v) value, rather than specifying one label for all patterns.
Furthermore, to reduce the noise sensitivity problem, the authors try to combine the proposed LBP operator with standard smoothing and denoising averaging filtering methods. To this end, the authors introduce a new spatially averaged filter in the LBP domain, called the recurrent majority vote filter. In this filter, to reduce the effect of noise, the value of each bit is modified with the majority value of its left and right neighbors. Like average filtering, this majority voting filter can smooth the obtained binary code and improve the noise immunity of the proposed LBP. Since majority voting is the traditional way to implement fault-tolerant systems, the author named the method noise tolerant LBP (NTLBP), and its formula is as follows: The core idea
of this method is not only to effectively and reasonably use the more regular non-uniform original information in the local texture mode , and is more robust to noise. To this end, an integration of recurrent majority voting filtering and a new labeling scheme is employed. Experiments on different databases show that this method has the highest recognition ability and classification accuracy among the existing LBP operators, and better anti-noise ability for texture analysis.
References:
Fathi A, Naghsh-Nilchi A R. Noise tolerant local binary pattern operator for efficient texture analysis［J］. Pattern Recognition Letters, 2012, 33(9): 1093-1100. Research on LBP texture feature extraction based on anti-
noise
NRLBP、ENRLBP（2013）
Jiang et al. proposed noise-resistant LBP (NRLBP) and extended noise-resistant LBP (ENRLBP) on the basis of FLBP. Compared with LBP, LTP and many other variants, the proposed NRLBP and ENRLBP are more resistant to noise . In various applications, the proposed NRLBP and ENRLBP exhibit superior performance to the LBP\LTP variants.
NRLBP can generate m different uniform patterns at pixel positions with ambiguity, then each uniform pattern simply takes 1/m as the weight of the final histogram statistics. If the uncertain mode C(x) cannot
produce a uniform mode, increase the bin representing the non-uniform mode by 1, and realize it by establishing a lookup table, so that its calculation amount is significantly lower than that of FLBP, and it is only slightly higher than the traditional LBP method. Quantities are still complex.
References:
Ren, Jianfeng, Jiang, et al. Noise-Resistant Local Binary Pattern With an Embedded Error-Correction; Mechanism［J］. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, 2013, 22 (10) : 4049-4060.
Research on LBP texture feature extraction based on anti-noise
CRLBP（2013）
Zhao et al. proposed a Complete Robust Local Binary Pattern (CRLBP), in which the value of each central pixel in a 3*3 local region is replaced by its average local gray level. Compared with the central gray value, the average local gray level is more robust to noise and illumination changes. In order to make CRLBP more robust and stable, weighted local grayscale (WLG) is also introduced to replace the original central pixel value, which is more robust to noise.
References:
Zhao Y, Jia W, Hu RX, et al. Completed robust local binary pattern for texture classification［J］. Neurocomputing, 2013, 106 (6): 68-76
Research on Anti-noise Based LBP Texture Feature Extraction
HYDROGEN（2014）
Based on the traditional LBP method, Liu et al. designed a new BRINT descriptor that is robust to noise. First, the circular neighborhood system of the central pixel is used to represent a neighborhood pixel in the new neighborhood system with the average value of Q consecutive neighborhood pixels, and finally a new circular neighborhood system is obtained; then the original The central pixel value is thresholded to perform binary quantization on the new neighborhood system, and finally obtain a binary pattern.
BRINT is a theoretically and computationally simple, noise-resistant texture classification method. The method is efficient, robust to additive noise, illumination and rotation invariance, avoids disadvantages associated with uniform patterns, and is capable of encoding a large number of scales. The proposed method produces consistently good classification results on all datasets, and outperforms state-of-the-art methods most significantly in high-noise situations.
References:
Liu L, Yang B, Fieguth P, et al. BRINT: A binaryrotation invariant and noise tolerant texture descriptor［C］// IEEE International Conference on Image Processing. IEEE, 2014. Research on LBP texture feature extraction based on anti-
noise
co-ALBP（2014）
CoALBP (Co-Occurrence of adjacent Local Binary Pattern) is an adjacent local binary pattern, which is improved from the original LBP operator model. Because the LBP operator is very robust to illumination, the original LBP operator is widely used in texture recognition, face recognition, etc., but the original LBP is binarized by comparing the intermediate pixels with the surrounding pixels, and then the histogram statistics, which leads to a lack of spatial information between operators. As shown in Figure 2.10, it shows the difference between LBP and CoALBP features. Compared with LBP feature operators, CoALBP considers the spatial information of operators.
References:
Qi X, Xiao R, Li CG, et al. Pairwise rotation invariant co-occurrence local binary pattern[J]. IEEE transactions on pattern analysis and machine intelligence, 2014, 36(11): 2199-2213. Network-
based Camera face recognition system design