Chapter 5 Texture Analysis

Abstract: Chapter 5 Texture Analysis Notes Main content: texture representation (gray level co-occurrence matrix, word bag model), texture synthesis

5.1 Introduction to textures

Texture is a ubiquitous and easily identifiable phenomenon that is difficult to define.
Textures will contain recurring patterns , that is, the same small image will be repeated many times in some way , and at the same time, these small image blocks may have some deformation due to changes in viewing angle and so on.
Texture is formed by some elements repeating in a certain way. These elements are called texture primitives . Textures can be represented and analyzed by first finding the texture primitives and then summarizing how the texture primitives recur .
Texture reflects the spatial distribution of grayscale or color in the image , and does not care about the specific value of grayscale or color, and texture is greatly affected by scale . 如图5-2shown.
Texture Generation : Textures can be generated by albedo variations between different surfaces in a scene .
- Such as: the texture formed by the pattern on the clothes / the texture of the bark produced by the change of the surface shape / composed of many small elements, such as the texture formed by many leaves
Texture function : through the texture, the information of the scene can be inferred, and the object can be identified and the shape of the object can be analyzed .
Texture Analysis : Contains two important questions:
- How to represent texture
- How to combine textures
Figure 5-2

5.2 Texture Representation

5.2.1 Gray level co-occurrence matrix

Gray Level Cooccurence Matrix (GLCM) is a method to describe texture by studying the spatial correlation characteristics of gray scale.

Since the texture is formed by the repeated appearance of the gray level distribution in the spatial position , there will be a certain relationship between two pixels separated by a certain distance in the image space, that is, the spatial correlation characteristics of the gray level in the image . The grayscale histogram is the result of statistics on the grayscale of each pixel on the image, and the grayscale co-occurrence matrix is obtained by statistics on the grayscale values of two pixels that maintain a certain distance on the image. of.

Gray level co-occurrence matrix (suitable for describing tiny textures )

The gray level co-occurrence matrix P[i, j] is a two-dimensional correlation matrix . A displacement vector d=(dx, dy) is stipulated, and the gray co-occurrence matrix can be obtained by calculating the number of all pixel pairs separated by d and having gray levels i and j .

For the same image, given different displacement vectors , different gray scale co-occurrence matrices can be obtained .

The gray level co-occurrence matrix represents the distribution information of the gray level in the image in space .

Since the number of pixel pairs with gray level [i,j] is not necessarily equal to the number of pixel pairs with gray level [j,i], the gray level co-occurrence matrix is

The size of the gray level co-occurrence matrix is only related to the maximum gray level and has nothing to do with the image size , so it is easy to understand and calculate.

Disadvantages: Since the gray-level co-occurrence matrix does not contain shape information, it is not suitable for describing textures with large-area primitives.

Normalized grayscale co-occurrence matrix

Calculation: Divide the gray-level co-occurrence matrix by the total number of pixel pairs satisfying the displacement vector, and the normalized gray-level co-occurrence matrix can be obtained .

Normalized gray level co-occurrence matrix :

The entropy of the texture corresponding to the randomness of the gray level distribution can be calculated

The uniformity feature entropy value is a measure of the randomness of the texture content: the larger the entropy value , the stronger the randomness

Calculate the energy feature of the texture: The energy feature reflects the uniformity or smoothness of the texture. The smaller the energy, the more uniform or smooth the texture;

The contrast feature is a measure that reflects the grayscale difference between two pixels in a texture point pair . If there are more point pairs with a large grayscale difference , the contrast will be larger and the texture will be rougher . Otherwise, the texture will be softer.

Uniformity: Reflects the uniformity of the texture

5.2.2 Bag of words model

Texture is formed by some elements repeating in a certain way. These elements are called texture primitives .

Chapter 3 shows that filters can be viewed as pattern detectors , so that filters of various patterns can be used to detect texture primitives of various patterns .
Textures can be represented and analyzed by first finding the texture primitives and then summarizing how the texture primitives recur .
But the texture primitive pattern is nearly infinite and hard to detect . So describe the texture indirectly in another way :
- Find the sub-elements of the texture primitive (generally various points and various edges. The number of sub-elements is much smaller than the number of texture primitives , and can be detected with various filters)
- Summarizes the appearance of sub-elements and indirectly describes the texture
Process :
- 1. Select a series of filters (different in size, direction, and scale)
  
  Each filter represents a pattern, including various points and edges in various ways.
  
  2. Use a filter to filter the image , and correct the result of the filter .
  
  Usually half wave (half wave) method is used for correction, that is, for a filter Fi, it is operated with the filtering result of the image to obtain two corrected results max(Fi*I) and max(0,-(fi * I)).
  
  Reason for correction: Avoid averaging the responses of dark foreground and light background and light foreground and dark background when performing subsequent operations such as averaging .
  
  3. Some form of summary of the corrected responses . ( summary vector )
  eg: find the maximum value, find the average, etc.
  - These summaries can capture the information of neighboring elements at different scales to obtain an overall description of the texture ,
  - Each pixel can then be described using a summary vector whose dimension is the number of filters used multiplied by 2.
  - The summary vector for each pixel represents the combination of various filter responses at that pixel location .
    - eg: There are only two filters, horizontal and vertical, then the summary vector of each pixel is a four-dimensional vector (each filter will have two response maps after correction).
      
      [1,1,0,0] means there is a strong horizontal edge and no vertical edge at this pixel location.
  - Represents the entire texture image : Computes a histogram of the summed vectors of the pixels it contains .
  - The summary vector is continuous and cannot be directly counted to compute a histogram.
    
    In actual calculation, the summary vector representation is continuous , that is, the responses of the filters are all continuous values. So after getting the summary vector for each pixel, we cannot simply count these vectors to calculate the histogram . (Even if you discretize the summary vector, you cannot directly compute the histogram).
    - Reason: The dimension of the summary vector is high, and the histogram is calculated directly. The number of elements contained in the histogram is too large.
    - Solution: bag of words model
  Bag of words model process:如图5-6
  - 1. Create a dictionary/codebook
    - The dictionary contains N words (vectors).
    - The way to build a dictionary: the clustering method.
    Commonly used clustering methods: K-means/K-means method .
    
    It is a partition-based clustering algorithm that uses the distance between samples as the standard for measuring the similarity between samples. (That is, the smaller the distance between samples, the higher the similarity, the more likely they are in the same class) (There are many ways to calculate the distance between samples, and K-means uses Euclidean distance to calculate the distance between samples )
    
    K: the number of categories
    
    Means: The mean value of the data objects in the class (can be regarded as a description of the class center). Therefore, it is also called K-means algorithm
    
    K-means algorithm process:
    
    (1) Enter the K value ( the number of categories obtained by clustering )
    
    (2) Randomly select K samples from the data set as the initial cluster center
    
    (3) For each sample in the set, calculate its distance from each initial cluster center and divide it into the nearest category
    
    (4) Calculate the mean value of each category as the new cluster center
    
    (5) If the distance change between the new cluster center and the old cluster center is greater than the given threshold, repeat steps (3)~(5)
  - 2. Use the histogram of the word in the dictionary to represent the texture. (Number of elements in the histogram = number of words in the dictionary N)
    - For each pixel in the texture image, compute its summary vector.
    - To judge which word (vector) in the dictionary is similar ( closer ), use the word (vector) to represent the summary vector.
    - Use the number of word occurrences in the dictionary corresponding to the summary vector for each pixel to build a histogram representation of the texture (a histogram of the frequency of occurrence of each word (vector) in the dictionary )

insert image description here
Figure 5-6

5.3 Texture synthesis

Texture synthesis refers to the process of generating a large piece of the texture image through an algorithm given a small piece of texture image.

Texture synthesis has important applications in graphics and image hole filling

Texture synthesis method :

Assume that given a texture image, one of the pixel values is missing.

Select a window around the pixel, match it in other areas of the image , find the window that best matches the window , use the value of the center pixel of the matching window to fill this pixel, and calculate the sum of squared differences between the two windows (Sum of Squared Differences ,SSD) to match.

Encounter problems:

Problem 1: Can't find a window that satisfies the threshold
- Cause: When matching between windows, simply set a threshold, and then select those windows whose distance is smaller than the threshold for texture completion.
- Solution: A better strategy is to select all windows whose distance is less than (1+α)smin.
  - smin is the distance between the window closest to the window to be matched and the window to be matched.
  - α is a parameter to ensure that a matching window can be found for texture synthesis every time.
Question 2: The size of the selected window during texture synthesis has a great influence on the texture synthesis effect.
- The selection window is too small to capture features on the larger scale of the texture.
- For example: 图5-8the dotted texture of the first row as an example
  - The window is too small to capture the shape characteristics of the point, and the generated texture is some stripes
  - After the window becomes larger, the generated texture has the characteristics of a ring, but it does not capture the uniform distribution between the rings
  - When the window becomes larger, evenly distributed textures can be generated
  Figure 5-8
Problem 3: Texture compositing in units of pixels is slow.
- Solution: When compositing textures, the compositing is performed in units of image blocks , and the compositing method is similar to the pixel-based method.
- Texture synthesis method based on image stitching:如图5-9
  - First, a small random image is taken from the texture image of the original image and placed on a blank target image
    Find the image block most similar to the area shown in the box in the figure from the original texture image. When searching, the part of the grid in the box is not calculated, and only the part outside the grid in the box is used to calculate the similarity between image blocks. .
    
    The similarity between images is calculated using SSD, and the windows whose distance is less than the threshold are selected for completion
  - Secondly, texture generation is performed in the order from left to right and top to bottom
    
    Figure 5-9

Texture synthesis purpose : filling the holes in the image

If the image area where the hole is located is a relatively irregular texture area, making it impossible to find similar image blocks for filling, the hole can be filled by synthesizing textures.

Problem: When filling holes with texture compositing, the order of filled pixels can have a big impact on the result.

Solution: Generally, the texture is synthesized from the boundary of the hole.

This is because the pixels in these places have the most known pixels around them and are the easiest to match, but doing so may make the long borders in the graph disappear.

The order of texture synthesis affects the result of texture synthesis. example:如图5-12

If composited from the edge of the hole, the light pole in the image will disappear;

And compositing from the edge of the image, you can preserve these long edges.

Figure 5-12