[Python] How to understand vector norm, distance between vectors, and how to use numpy to implement vector norm calculation and distance calculation between vectors?

What is a norm?

Norm is a concept in mathematics that is used to measure the size or distance of vectors, matrices, etc.

Norm can be simply understood as "distance". Because vectors have both magnitude and direction, vectors cannot be compared directly. Norms provide a way to convert all vectors into a real number to compare vector sizes.

The norm can be equal to the distance from the point to the zero point of the coordinate.

In layman's terms, norm is a measurement method artificially defined for the convenience of measurement. It mainly selects a unified quantification standard to facilitate measurement and comparison when facing complex spaces and multi-dimensional arrays.

vector norm

Vector norm represents the size of vectors in vector space.

L-1 norm of vector

The L1 norm of a vector is the sum of the absolute values of each element in the vector. The calculation formula is as follows:

L-2 norm of vector

The L2 norm (second-order norm) of a vector is the sum of the squares of each element in the vector and then the square root. The calculation formula is as follows:

LP norm of vector

The LP norm of a vector refers to summing each element in the vector to the power of P, and then taking the root of the sum to the power of P.

Calculated as follows:

When P takes 2, it is the L-2 norm; when P takes 1, it is the L-1 norm.

L∞ norm of vector

Positive infinite norm: The largest absolute value among all elements.

Negative infinity norm: The smallest absolute value among all elements.

What is the distance between vectors?

We often need to calculate the difference between vectors to evaluate the similarity and category of vectors. The distance between two vectors in space is a reflection of the similarity of the two vectors. Depending on the characteristics of the data, different measurement methods can be used. Generally speaking, to define a function d(x, y) , if it is a "distance measure", it needs to satisfy the following basic properties:

The LP norm calculation formula for the distance between vectors is as follows:

Suppose there are two points A(2,3) and B(3,1) on the two-dimensional plane, as shown in the figure:

A and B are also two vectors at the same time. Applying the L-2 norm calculation formula, the L-2 norm of vector A(2,3) can be calculated to be approximately: 3.6. This value is also the modulus of vector A, that is The distance from point A to the origin (0,0).

In the same way, the L-2 norm of vector B(3,1) is approximately: 3.16, that is, the distance from vector B(3,1) to the 0 origin (0,0) is 3.16.

How to calculate the distance from vector A to vector B? Applying the above formula, the result is approximately 2.236. The calculation process is as follows:

Common distances between vectors:

Assume that X and Y are both n-dimensional vectors, as shown in the figure below:

Manhattan distance and Euclidean distance (Euclidean distance)

Manhattan Distance, also known as city block distance, refers to the number of grids passed along a right-angled path from one point to another in a grid-shaped city block.

The calculation formula of Manhattan distance in n-dimensional space is as follows:

The L-1 norm of a vector can be seen as a special case of Manhattan distance, with the y vector being 0.

The calculation formula of Euclidean distance in n-dimensional space is as follows:

The L-2 norm of a vector can be seen as a special case of Euclidean distance, with the y vector being 0.

The red line in the above figure represents the Manhattan distance, while the blue and yellow represent the equivalent Manhattan distance. Green represents the Euclidean distance, which is the straight-line distance.

Chebyshev distance

Chebyshev distance defines the distance between two points as the maximum value of the numerical difference between their coordinates. The calculation formula is as follows:

The L infinite norm of a vector can be seen as a special case of Chebyshev distance, with the y vector being 0.

Minkowski distance

The LP norm of a vector can be seen as a special case of Minkowski distance, with the y vector being 0.

How to calculate vector or matrix norm in Numpy

numpy.linalg.normfunction is a function in the numpy library that is used to calculate the norm of a vector or matrix . Its function prototype is as follows:

numpy.linalg.norm(x, ord=None, axis=None, keepdims=False)

Among them, the parameter description is as follows:

x:Input array or vector.

ord:The type of norm, which can take one of the following values:

None: The default is 2 norm, which is the Euclidean norm.

1: L1 norm, that is, the sum of the absolute values of each element in the vector is used as the norm.

np.inf: Infinite norm, that is, the maximum value of the absolute value of each element in the vector is used as the norm.

-np.inf: Negative infinity norm, that is, the minimum absolute value of each element in the vector is used as the norm.

axis:Specify along which axis the norm is calculated. If not specified, the calculation will be performed on the entire array by default.

keepdims: Whether to keep the dimensions of the output result consistent with the input result, the default is False.

Compute vector and matrix L1 norms

import numpy as np

# 创建一个3x3的矩阵
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# 计算L1范数
l1_norm = np.linalg.norm(a, ord=1)
print("矩阵的L1范数为：", l1_norm)
l1_norm_c = np.linalg.norm(a, ord=1, axis=0)
print("列向量L1范数为：", l1_norm_c)
l1_norm_r = np.linalg.norm(a, ord=1, axis=1)
print("行向量L1范数为：", l1_norm_r)

Compute vector and matrix L2 norm

import numpy as np

# 创建一个3x3的矩阵
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# 计算L2范数
l2_norm = np.linalg.norm(a, ord=2)
print("矩阵L2范数为：", l1_norm)
l2_norm_c = np.linalg.norm(a, ord=2, axis=0)
print("列向量L2范数为：", l1_norm_c)
l2_norm_r = np.linalg.norm(a, ord=2, axis=1)
print("行向量L2范数为：", l1_norm_r)

Compute the infinite norm of a vector

import numpy as np

# 创建一个一维的矩阵
a = np.array([-11, 2, 3, 4, 5, 6, 7, 8, 9])
# 计算负无穷范数
n_inf_norm = np.linalg.norm(a, ord=-np.inf)
print("矩阵负无穷范数为：", n_inf_norm)

# 计算正无穷范数
p_inf_norm = np.linalg.norm(a, ord=np.inf)
print("矩阵正无穷范数为：", p_inf_norm)

Calculate the Manhattan distance between vectors

import numpy as np

def manhattan_distance(x, y):
    import numpy as np
    x = np.array(x)
    y = np.array(y)
    return np.sum(np.abs(x-y))


x = [1, 3, 5]
y = [2, 4, 6]
print('x-y:',manhattan_distance(x, y))
print('y-x:',manhattan_distance(y, x))

Calculate the Euclidean (Euclidean distance) between vectors

import numpy as np
def euclidean_distance(x, y):
    import numpy as np
    x = np.array(x)
    y = np.array(y)
    return np.sqrt(np.sum(np.square(x-y)))

x = [1, 3, 5]
y = [2, 4, 6]
print('x-y:',euclidean_distance(x, y))
print('y-x:',euclidean_distance(y, x))

Compute Minkowski distance between vectors

import numpy as np
def minkowski_distance(x, y, p):
    import math
    import numpy as np
    xy= zip(x, y)
    return math.pow(np.sum([math.pow(np.abs(i[0]-i[1]), p) for i in xy]), 1/p)

x = [1, 3, 5]
y = [2, 4, 6]
print('x-y:',minkowski_distance(x, y, 3))
print('y-x:',minkowski_distance(y, x, 3))

Calculate Chebyshev distance between vectors

import numpy as np
# 切比雪夫距离
def chebyshev_distance(x, y):
    import numpy as np
    x = np.array(x)
    y = np.array(y)
    return np.max(np.abs(x-y))


x = [10, 3, 5]
y = [2, 4, 16]
print('x-y:',chebyshev_distance(x, y))
print('y-x:',chebyshev_distance(y, x))