Statistical learning Notes (a) -k nearest neighbor principle and achieve python

Input: training set
$$ T = \ {(x_1, y_1), (x_2, y_2), ..., (x_N, y_N) \} $$
wherein, $ x_i \ in \ mathcal { X} \ subseteq \ mathcal {R ^ n} $ is an example of a feature vector, $ y_i \ in \ mathcal { Y} = \ {c_1, c_2, ..., c_K \} $ is an example of class, $ i = 1,2, .. ., N $; $ X $ example feature vector;

Output: Examples of X $ $ $ Y belongs to the class $
(. 1) according to a given distance metric, identify the nearest $ X $ $ k in the training set $ T $ $ points, which covers $ k $ points $ X $ neighborhood base $ n_k (x) $;
(2) $ (such as a majority vote) determined in $ X $ $ n_k (x) according to the classification category decision rule $ Y $:
$$ y = \ arg \ max_ {cj } \ sum_ {x_i \ in n_k (x)} I (y_i = c_i), i = 1,2, ..., N; j = 1,2, ..., K $$

In the above formula, $ I $ is an exponential function, i.e., when the $ $ y_i = c_i $ I $ 1, $ I $ is 0 otherwise.

Note: When $ k = 1 $, $ k $ nearest neighbor method, also known as nearest neighbor algorithm, which is a special case $ k $ nearest neighbor algorithm.

 

import math
from itertools import combinations

#定义距离
def Distance(x,y,p=2):
    if len(x)==len(y) and len(x)>1:
        sum=0
        for i in range(len(x)):
            sum+=math.pow(abs(x[i]-y[i]),p)
        return math.pow(sum,1/p)
    else:
        return 0