KNN算法实现

1.《机器学习实战》代码

计算原理：
1）计算已知类别数据集中的点与当前点的距离；
2）按照距离递增次序排序；
3）选取与当前点距离最小的k个点；
4）确定前k个点所在类别出现的频率；
5）返回前k个点出现频率最高的类别作为当前点的预测分类。

预测函数代码如下：

def classify0(inX,dataSet,labels,k):
    dataSetSize = dataSet.shape[0]#数据行数
    #tile函数的对数据进行重复，构造一个n行1列的数据。求出数据inX和数据dataset的差值
    diffMat = tile(inX,(dataSetSize,1)) - dataSet
    #和的二次方
    sqDiffMat = diffMat ** 2
    #对数据进行求和
    sqDistances = sqDiffMat.sum(axis = 1)
    #距离是对和进行开方                                                                                                                                                                                                                                                                                                                                                
    distances = sqDistances ** 0.5
    #对数据进行排序，返回数据从小到大排列的索引值
    sortedDistIndiciecs = distances.argsort()
    #构建空字典，字典存储类别及其出现频率
    classCount = {}
    for i in range(k):## 根据索引获取其对应的类别标签
        voteIlabel = labels[sortedDistIndiciecs[i]]
        #记录每个类别出现的频率
        classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1
    #对每个类别出现的频率进行由大到小排序
    sortedClassCount = sorted(classCount.iteritems(),key = operator.itemgetter(1),reverse = True)
    #选择最大的类别
    return sortedClassCount[0][0]  

2.李航《统计学习》代码

class KNN:
    def __init__(self, X_train, y_train, n_neighbors=3, p=2):
        """
        parameter: n_neighbors 临近点个数
        parameter: p 距离度量
        """
        self.n = n_neighbors
        self.p = p
        self.X_train = X_train
        self.y_train = y_train
    
    def predict(self, X):
        # 取出n个点
        knn_list = []
        for i in range(self.n):#i为从0到n的整数，得到X和每一个临近点的范数
            #linalg.norm求范数
            dist = np.linalg.norm(X - self.X_train[i], ord=self.p)
            knn_list.append((dist, self.y_train[i]))
            
        for i in range(self.n, len(self.X_train)):#
            max_index = knn_list.index(max(knn_list, key=lambda x: x[0]))
            dist = np.linalg.norm(X - self.X_train[i], ord=self.p)
            if knn_list[max_index][0] > dist:
                knn_list[max_index] = (dist, self.y_train[i])
                
        # 统计
        knn = [k[-1] for k in knn_list]
        count_pairs = Counter(knn)
        max_count = sorted(count_pairs, key=lambda x:x)[-1]
        return max_count
    
    def score(self, X_test, y_test):
        right_count = 0
        n = 10
        for X, y in zip(X_test, y_test):
            label = self.predict(X)
            if label == y:
                right_count += 1
        return right_count / len(X_test)

实用学习链接：

1. https://leileiluoluo.com/posts/kdtree-algorithm-and-implementation.html
2. https://blog.csdn.net/gamer_gyt/article/details/51232210
3. https://github.com/scikit-learn/scikit-learn/blob/7813f7efb/sklearn/neighbors/classification.py#L133