1. K-nearest neighbor algorithm (KNN)
1. Define
KNN
K: It is a natural number
N: nearest, nearest
N: neighbor, neighbor.
If a sample has the k most similar (that is, the nearest neighbor in the feature space) samples in the feature space, most of them belong to a certain category. , then the sample also belongs to this category
k = 1 and is easily affected by outliers.
2. Suppose there is a map of Beijing. I don’t know where I am. The purpose is to know which district of Beijing I am in.
This is a classification problem
. I don’t know where I am, but I know the distance between me and these people. And know which district these 5 people are in
KNN core idea: your "neighbors" to infer your category
3. Calculate distance formula.
The distance between two samples can be calculated by the following formula, also called Euclidean distance.
Note: The two-dimensional space is d = sqrt((x2 - x1)^2 + (y2 - y1)^2)
Other distance formulas:
Manhattan distance—absolute value distance
Minkowski distance
4. Example
movie type analysis, assuming we now have a training set
Now there is an unknown movie, which genre it is based on its characteristics
We can use the idea of K nearest neighbor algorithm
When k = 1, match (romance movie)
When k = 2, match (romance film)
When k = 6, it cannot be determined. . .
5. Analysis
(1) When the k value is too small, it is easily affected by outliers
(2) When the k value is too large, it is easy to make misclassifications and is easily affected by sample imbalance
(3) When Use the k-nearest neighbor algorithm to perform dimensionless processing (standardization)
6. K-nearest neighbor algorithm API
sklearn.neighbors.KNeighborsClassifier(n_neighbors=5, algorithm='auto')
n_neighbors: int, optional (default = 5), which is the k value. Query the default number of neighbors
algorithm: {'auto ','ball_tree','kd_tree','brute'}, optional algorithm for calculating nearest neighbors: 'ball_tree' will use BallTree, 'kd_tree' will use KDTree, 'auto' will try to match the value passed to the fit method value to determine the most appropriate algorithm. (Different implementation methods affect efficiency)
7. Call the method
(1) Instantiate the KNeighborsClassifier classifier
(2) Call the fit method and pass in the feature values and target values of the training set, which is equivalent to getting the model
(3) Call the predict method to get the prediction value, compare the predicted value with the real value
(4) You can also call the score method to directly calculate the accuracy
2. Prediction of iris flower species
1. Introduction to the data set
The Iris data set is a commonly used classification experimental data set, collected and organized by Fisher, 1936. Iris, also known as the iris flower data set, is a type of multivariate analysis data set.
Iris contains 150 samples, corresponding to each row of data in the data set. Each row of data contains four features of each sample and the category information of the sample, so the iris data set is a two-dimensional table with 150 rows and 5 columns.
2. Attribute
sepal length: sepal length (cm)
sepal width: sepal width (cm)
petal length: petal length (cm)
petal width: petal width (cm)
class: Setosa mountain iris, Versicolour color-changing iris, Virginica Virginia iris
3. Steps
(1) Obtaining data
(2) Data set partitioning
(3) Feature engineering
standardization
(4) Machine learning training: KNN predictor process
(5) Model evaluation
4、day02_machine_learning.py
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
def KNN_iris():
"""
用KNN算法对鸢尾花进行分类
"""
# 1、获取数据
iris = load_iris()
# 2、划分数据集
x_train, x_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=6)
# 3、特征工程：标准化
transfer = StandardScaler()
x_train = transfer.fit_transform(x_train)
# 用训练集的平均值和标准差对测试集的数据来标准化
# 这里测试集和训练集要有一样的平均值和标准差，而fit的工作就是计算平均值和标准差，所以train的那一步用fit计算过了，到了test这就不需要再算一遍自己的了，直接用train的就可以
x_test = transfer.transform(x_test)
# 4、KNN算法预估器
estimator = KNeighborsClassifier(n_neighbors=3)
estimator.fit(x_train, y_train)
# 5、模型评估
# 方法1：直接比对真实值和预测值
y_predict = estimator.predict(x_test)
print("y_predict：\n", y_predict)
print("直接比对真实值和预测值：\n", y_test == y_predict)
# 方法2：计算准确率
score = estimator.score(x_test, y_test)
print("准确率为：\n", score)
return None
if __name__ == "__main__":
# 代码1：用KNN算法对鸢尾花进行分类
KNN_iris()
operation result:
y_predict：
[0 2 0 0 2 1 1 0 2 1 2 1 2 2 1 1 2 1 1 0 0 2 0 0 1 1 1 2 0 1 0 1 0 0 1 2 1
2]
直接比对真实值和预测值：
[ True True True True True True False True True True True True
True True True False True True True True True True True True
True True True True True True True True True True False True
True True]
准确率为：
0.92105263157894733. Code description
1. The iris data set we use is tabular data, for example
2. The iris = load_iris() data set obtained in the code contains the feature value iris.data and the target value iris.target
iris.data：
[[5.1 3.5 1.4 0.2]
[4.9 3. 1.4 0.2]
[4.7 3.2 1.3 0.2]
[4.6 3.1 1.5 0.2]
[5. 3.6 1.4 0.2]
[5.4 3.9 1.7 0.4]
[4.6 3.4 1.4 0.3]
[5. 3.4 1.5 0.2]
[4.4 2.9 1.4 0.2]
[4.9 3.1 1.5 0.1]
[5.4 3.7 1.5 0.2]
[4.8 3.4 1.6 0.2]
[4.8 3. 1.4 0.1]
[4.3 3. 1.1 0.1]
[5.8 4. 1.2 0.2]
[5.7 4.4 1.5 0.4]
[5.4 3.9 1.3 0.4]
[5.1 3.5 1.4 0.3]
[5.7 3.8 1.7 0.3]
[5.1 3.8 1.5 0.3]
[5.4 3.4 1.7 0.2]
[5.1 3.7 1.5 0.4]
[4.6 3.6 1. 0.2]
[5.1 3.3 1.7 0.5]
[4.8 3.4 1.9 0.2]
[5. 3. 1.6 0.2]
[5. 3.4 1.6 0.4]
[5.2 3.5 1.5 0.2]
[5.2 3.4 1.4 0.2]
[4.7 3.2 1.6 0.2]
[4.8 3.1 1.6 0.2]
[5.4 3.4 1.5 0.4]
[5.2 4.1 1.5 0.1]
[5.5 4.2 1.4 0.2]
[4.9 3.1 1.5 0.2]
[5. 3.2 1.2 0.2]
[5.5 3.5 1.3 0.2]
[4.9 3.6 1.4 0.1]
[4.4 3. 1.3 0.2]
[5.1 3.4 1.5 0.2]
[5. 3.5 1.3 0.3]
[4.5 2.3 1.3 0.3]
[4.4 3.2 1.3 0.2]
[5. 3.5 1.6 0.6]
[5.1 3.8 1.9 0.4]
[4.8 3. 1.4 0.3]
[5.1 3.8 1.6 0.2]
[4.6 3.2 1.4 0.2]
[5.3 3.7 1.5 0.2]
[5. 3.3 1.4 0.2]
[7. 3.2 4.7 1.4]
[6.4 3.2 4.5 1.5]
[6.9 3.1 4.9 1.5]
[5.5 2.3 4. 1.3]
[6.5 2.8 4.6 1.5]
[5.7 2.8 4.5 1.3]
[6.3 3.3 4.7 1.6]
[4.9 2.4 3.3 1. ]
[6.6 2.9 4.6 1.3]
[5.2 2.7 3.9 1.4]
[5. 2. 3.5 1. ]
[5.9 3. 4.2 1.5]
[6. 2.2 4. 1. ]
[6.1 2.9 4.7 1.4]
[5.6 2.9 3.6 1.3]
[6.7 3.1 4.4 1.4]
[5.6 3. 4.5 1.5]
[5.8 2.7 4.1 1. ]
[6.2 2.2 4.5 1.5]
[5.6 2.5 3.9 1.1]
[5.9 3.2 4.8 1.8]
[6.1 2.8 4. 1.3]
[6.3 2.5 4.9 1.5]
[6.1 2.8 4.7 1.2]
[6.4 2.9 4.3 1.3]
[6.6 3. 4.4 1.4]
[6.8 2.8 4.8 1.4]
[6.7 3. 5. 1.7]
[6. 2.9 4.5 1.5]
[5.7 2.6 3.5 1. ]
[5.5 2.4 3.8 1.1]
[5.5 2.4 3.7 1. ]
[5.8 2.7 3.9 1.2]
[6. 2.7 5.1 1.6]
[5.4 3. 4.5 1.5]
[6. 3.4 4.5 1.6]
[6.7 3.1 4.7 1.5]
[6.3 2.3 4.4 1.3]
[5.6 3. 4.1 1.3]
[5.5 2.5 4. 1.3]
[5.5 2.6 4.4 1.2]
[6.1 3. 4.6 1.4]
[5.8 2.6 4. 1.2]
[5. 2.3 3.3 1. ]
[5.6 2.7 4.2 1.3]
[5.7 3. 4.2 1.2]
[5.7 2.9 4.2 1.3]
[6.2 2.9 4.3 1.3]
[5.1 2.5 3. 1.1]
[5.7 2.8 4.1 1.3]
[6.3 3.3 6. 2.5]
[5.8 2.7 5.1 1.9]
[7.1 3. 5.9 2.1]
[6.3 2.9 5.6 1.8]
[6.5 3. 5.8 2.2]
[7.6 3. 6.6 2.1]
[4.9 2.5 4.5 1.7]
[7.3 2.9 6.3 1.8]
[6.7 2.5 5.8 1.8]
[7.2 3.6 6.1 2.5]
[6.5 3.2 5.1 2. ]
[6.4 2.7 5.3 1.9]
[6.8 3. 5.5 2.1]
[5.7 2.5 5. 2. ]
[5.8 2.8 5.1 2.4]
[6.4 3.2 5.3 2.3]
[6.5 3. 5.5 1.8]
[7.7 3.8 6.7 2.2]
[7.7 2.6 6.9 2.3]
[6. 2.2 5. 1.5]
[6.9 3.2 5.7 2.3]
[5.6 2.8 4.9 2. ]
[7.7 2.8 6.7 2. ]
[6.3 2.7 4.9 1.8]
[6.7 3.3 5.7 2.1]
[7.2 3.2 6. 1.8]
[6.2 2.8 4.8 1.8]
[6.1 3. 4.9 1.8]
[6.4 2.8 5.6 2.1]
[7.2 3. 5.8 1.6]
[7.4 2.8 6.1 1.9]
[7.9 3.8 6.4 2. ]
[6.4 2.8 5.6 2.2]
[6.3 2.8 5.1 1.5]
[6.1 2.6 5.6 1.4]
[7.7 3. 6.1 2.3]
[6.3 3.4 5.6 2.4]
[6.4 3.1 5.5 1.8]
[6. 3. 4.8 1.8]
[6.9 3.1 5.4 2.1]
[6.7 3.1 5.6 2.4]
[6.9 3.1 5.1 2.3]
[5.8 2.7 5.1 1.9]
[6.8 3.2 5.9 2.3]
[6.7 3.3 5.7 2.5]
[6.7 3. 5.2 2.3]
[6.3 2.5 5. 1.9]
[6.5 3. 5.2 2. ]
[6.2 3.4 5.4 2.3]
[5.9 3. 5.1 1.8]]
iris.target：
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
2 2]You can see that iris.data is the four attributes of the data set, and iris.target is the species, represented by 0, 1, and 2.
3. train_test_split divides the data set
x_train: the characteristic value of the training set
x_test: the characteristic value of the test set
y_train: the target value of the training set
y_test: the target value of the test set
As you can see from the following code, the test set is 38, then The training set is 150-38=112
4. Then do feature engineering standardization.
Standardize the feature values of the training set and standardize the feature values of the test set.
5. Instantiate the KNN classifier.
Call the fit method and pass in the standardized feature values of the training set and the target value of the training set to obtain the model.
6. Call the predict method, pass in the characteristic values of the standardized test set, and obtain the prediction of the genus and species of this row of data.
y_predict：
[0 2 0 0 2 1 1 0 2 1 2 1 2 2 1 1 2 1 1 0 0 2 0 0 1 1 1 2 0 1 0 1 0 0 1 2 1
2]7. Then directly compare the predicted value with the target value of the test set, and the calculation accuracy is 92%. If there are several wrong predictions, the data
set is divided differently, and the prediction results will be different. You can try random_state=22 or twenty three
8. Question: Does the larger the data set, the more accurate the prediction of similar records? (Big Data)
4. Summary of K-nearest neighbor algorithm
1. Advantages
: Simple, easy to understand, easy to implement, no training required
2. Disadvantages
: Lazy algorithm requires a large amount of calculation and memory overhead when classifying test samples. The
K value must be specified. If the K value is improperly selected, the classification accuracy cannot be guaranteed.
3. Usage scenarios:
small data scenarios, thousands to tens of thousands of samples, tested in specific scenarios and specific businesses.