FCM算法理论及其Python实现

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/a19990412/article/details/88167365

FCM算法

全名为Fuzzy C-Means，是一种聚类算法。

Fuzzy c-means (FCM) is a method of clustering which allows one piece of data to belong to two or more clusters. This method (developed by Dunn in 1973 and improved by Bezdek in 1981) is frequently used in pattern recognition. It is based on minimization of the following objective function:
FCM是一种聚类的方法，可以允许一个数据属于两个或以上的类。这种方法（由Dunn在1973年提出，由Bezdek在1981年改进。）被频繁地用于模式识别。

• J. C. Dunn (1973): “A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters”, Journal of Cybernetics 3: 32-57
• J. C. Bezdek (1981): “Pattern Recognition with Fuzzy Objective Function Algoritms”, Plenum Press, New York

基于最小化下面的目标函数:

$J_m = \sum^{N}_{i=1}{\sum^{C}_{j=1}{u_{ij}^m||x_i-c_j||^2}}$

• m是任何大于1的实数
• $u_ij$是 $x_i$在类j上隶属度
• 其中 $x_i$是第i个d维测量向量
• $c_j$是一个d维的类中心

FCM主要通过迭代更新u和c来得到最终的解。

迭代终点一般选取两个：

• 当u变化的无穷范数小于某个值时
• 当迭代次数达到某个极限时候

这个过程可能会收敛到局部最小值或者是鞍点上

算法流程

1. 初始化矩阵U （描述每个点在不同的类中的概率）
2. 通过U算类中心
   $C_j=\frac{\sum^{N}_{i=1}{u_{ij}^m * x_i}}{\sum^{N}_{i=1}{u_{ij}^m}}$
3. 更新U，通过之前算出来的类中心
   $u_{ij} = \frac{1}{\sum^{C}_{k=1}{(\frac{||x_i-c_j||}{||x_i-ck||})^{\frac{2}{m-1}}}}$
4. 如果u变化的无穷范小于某个值的时候，就停下来，否者就转到第2步

其实看到这里，一般来说直接公式来做，就可以把算法实现了。

但我偷了个懒，找了一份代码，原始的代码有些问题，我做了修改后，然后写上了备注（部分备注我写的是英文的~）

Python代码实现

• accuracy函数是用来验证的
• 其他函数都是实现fcm的部分
• 数据文件我放到后面，大家复制之后另存为“SPECTF_New.csv”即可

import pandas as pd
import numpy as np
import random
import operator
import math

df_full = pd.read_csv("SPECTF_New.csv")
columns = list(df_full.columns)
features = columns[:len(columns) - 1]
class_labels = list(df_full[columns[-1]])
df = df_full[features]

# Number of Attributes
num_attr = len(df.columns) - 1

# Number of Clusters
k = 2

# Maximum number of iterations
MAX_ITER = 100

# Number of data points
n = len(df)

# Fuzzy parameter
m = 2.00


def accuracy(cluster_labels, class_labels):
    county = [0, 0]
    countn = [0, 0]
    tp = [0, 0]
    tn = [0, 0]
    fp = [0, 0]
    fn = [0, 0]

    for i in range(len(df)):
        # Yes = 1, No = 0
        if cluster_labels[i] == 1 and class_labels[i] == 'Yes':
            tp[0] = tp[0] + 1
        if cluster_labels[i] == 0 and class_labels[i] == 'No':
            tn[0] = tn[0] + 1
        if cluster_labels[i] == 1 and class_labels[i] == 'No':
            fp[0] = fp[0] + 1
        if cluster_labels[i] == 0 and class_labels[i] == 'Yes':
            fn[0] = fn[0] + 1

    for i in range(len(df)):
        # Yes = 0, No = 1
        if cluster_labels[i] == 0 and class_labels[i] == 'Yes':
            tp[1] = tp[1] + 1
        if cluster_labels[i] == 1 and class_labels[i] == 'No':
            tn[1] = tn[1] + 1
        if cluster_labels[i] == 0 and class_labels[i] == 'No':
            fp[1] = fp[1] + 1
        if cluster_labels[i] == 1 and class_labels[i] == 'Yes':
            fn[1] = fn[1] + 1

    a0 = float((tp[0] + tn[0])) / (tp[0] + tn[0] + fn[0] + fp[0])
    a1 = float((tp[1] + tn[1])) / (tp[1] + tn[1] + fn[1] + fp[1])
    p0 = float(tp[0]) / (tp[0] + fp[0])
    p1 = float(tp[1]) / (tp[1] + fp[1])
    r0 = float(tp[0]) / (tp[0] + fn[0])
    r1 = float(tp[1]) / (tp[1] + fn[1])

    accuracy = [a0 * 100, a1 * 100]
    precision = [p0 * 100, p1 * 100]
    recall = [r0 * 100, r1 * 100]

    return accuracy, precision, recall


# 返回的矩阵是每个点属于不同的集群的概率（隶属度）
# 随机生成
def initializeMembershipMatrix():
    membership_mat = list()
    for i in range(n):
        random_num_list = [random.random() for i in range(k)]
        summation = sum(random_num_list)
        temp_list = [x / summation for x in random_num_list]
        membership_mat.append(temp_list)
    return membership_mat


def calculateClusterCenter(membership_mat):
    cluster_mem_val = list(zip(*membership_mat))
    # cluster_mem_val 's shape is (k, n)
    cluster_centers = list()
    for j in range(k):  # 选定一个类别
        x = cluster_mem_val[j]
        xraised = [e ** m for e in x]  # m is a Fuzzy parameter
        denominator = sum(xraised)  # 求和

        # 将对应点向量乘上它在该类别上的概率 得到概率向量
        temp_num = list()
        for i in range(n):
            data_point = list(df.iloc[i])
            prod = [xraised[i] * val for val in data_point]
            temp_num.append(prod)

        # 将所有点在对应特征下的概率累积求和
        numerator = map(sum, zip(*temp_num))
        center = [z / denominator for z in numerator]
        # the shape of center is (num_attr,) , which is the number of the features.
        # 本质上做的是加权平均，关于不同的特征，在不同的点上做加权。权重即为初始概率
        cluster_centers.append(center)
    return cluster_centers


def updateMembershipValue(membership_mat, cluster_centers):
    # m is the Fuzzy parameter.
    p = float(2 / (m - 1))
    # cluster_centers : (k, num_attr)
    for i in range(n):
        x = list(df.iloc[i])
        # 算和两个中心的距离，这里采用的是二范数
        distances = [np.linalg.norm(list(map(operator.sub, x, cluster_centers[j]))) for j in range(k)]
        # 更新概率矩阵
        for j in range(k):
            den = sum([math.pow(float(distances[j] / distances[c]), p) for c in range(k)])
            membership_mat[i][j] = float(1 / den)
    return membership_mat


def getClusters(membership_mat):
    cluster_labels = list()
    for i in range(n):
        max_val, idx = max((val, idx) for (idx, val) in enumerate(membership_mat[i]))
        cluster_labels.append(idx)
    return cluster_labels


def fuzzyCMeansClustering():
    # Membership Matrix
    membership_mat = initializeMembershipMatrix()
    curr = 0
    while curr <= MAX_ITER:
        cluster_centers = calculateClusterCenter(membership_mat)
        # cluster_centers: (k, num_attr). k means k clusters. And num_attr means num_attr features.
        # 更新 membership_mat 矩阵
        membership_mat = updateMembershipValue(membership_mat, cluster_centers)
        cluster_labels = getClusters(membership_mat)
        curr += 1
    # print(membership_mat)
    return cluster_labels, cluster_centers


labels, centers = fuzzyCMeansClustering()
print(labels)
print(len(labels))
# a, p, r = accuracy(labels, class_labels)
#
# print("Accuracy = " + str(a))
# print("Precision = " + str(p))
# print("Recall = " + str(r))

• 数据

Attr_1,Attr_2,Attr_3,Attr_4,Attr_5,Attr_6,Attr_7,Attr_8,Attr_9,Attr_10,Attr_11,Attr_12,Attr_13,Attr_14,Attr_15,Attr_16,Attr_17,Attr_18,Attr_19,Attr_20,Attr_21,Attr_22,Attr_23,Attr_24,Attr_25,Attr_26,Attr_27,Attr_28,Attr_29,Attr_30,Attr_31,Attr_32,Attr_33,Attr_34,Attr_35,Attr_36,Attr_37,Attr_38,Attr_39,Attr_40,Attr_41,Attr_42,Attr_43,Attr_44,Class
57,69,68,75,69,74,73,71,57,61,72,74,73,69,61,58,60,55,71,62,79,70,77,71,65,63,69,55,61,68,75,74,63,64,63,58,69,67,79,77,72,70,61,65,Yes
76,59,82,76,80,56,74,67,67,58,77,71,56,68,60,64,66,66,66,66,82,79,40,45,49,48,75,70,64,35,71,60,39,28,71,66,73,61,71,49,53,45,29,15,Yes
65,62,67,68,65,67,71,71,64,56,73,72,68,69,56,57,67,62,74,66,80,76,80,78,53,47,48,36,68,65,74,73,60,60,67,63,74,63,77,79,68,70,59,56,Yes
70,54,66,66,76,46,74,58,68,52,81,58,67,58,68,32,73,59,76,51,82,57,76,54,58,30,69,41,59,59,67,73,62,55,60,55,65,56,65,44,73,36,51,28,Yes
64,64,70,75,70,71,74,71,59,60,62,68,70,66,69,72,69,69,61,63,56,60,62,66,69,71,62,63,67,65,62,58,52,51,67,66,61,56,64,65,71,73,57,63,Yes
65,67,69,76,62,68,65,66,65,64,74,73,60,75,66,63,64,62,73,65,77,74,69,69,66,59,68,59,69,69,76,79,65,63,60,60,69,64,69,74,69,70,62,57,No
68,70,69,74,65,70,72,73,71,68,73,70,68,74,63,65,65,63,70,67,74,70,69,72,62,63,63,61,66,66,74,80,70,72,69,67,60,57,69,62,66,75,50,50,No
64,66,68,71,62,64,74,73,63,67,66,74,70,74,59,64,75,73,70,66,79,81,79,78,61,62,76,72,67,67,71,75,65,62,70,69,68,65,75,72,62,64,57,54,No
58,63,80,71,76,70,70,71,64,63,74,78,77,75,62,61,62,56,71,52,82,71,84,85,71,71,57,47,42,39,70,70,50,70,50,46,58,60,76,73,82,77,65,66,No
77,79,79,77,74,76,76,81,65,68,66,66,74,73,72,68,67,73,63,62,72,67,76,69,68,64,64,61,69,68,73,75,70,66,64,70,70,70,73,76,79,73,65,63,No
54,53,73,68,77,65,65,72,60,50,69,64,73,75,74,76,69,72,66,60,74,67,68,69,69,78,56,67,69,61,76,71,67,64,73,70,55,49,70,61,73,70,61,61,Yes
59,57,67,71,66,68,68,70,56,62,77,61,67,71,75,71,67,64,62,54,64,75,71,72,76,79,75,70,71,77,71,69,56,54,62,64,56,53,71,68,64,63,56,56,No
68,64,74,80,76,72,78,75,67,64,75,80,78,77,66,64,67,67,70,60,78,82,70,68,63,60,64,60,54,56,70,73,59,65,55,58,50,51,73,70,69,65,42,41,No
78,73,68,74,68,69,63,74,68,67,73,73,66,71,64,67,66,68,61,66,75,71,60,62,64,66,65,67,66,62,74,75,61,61,63,66,68,65,71,62,69,67,61,59,Yes
64,72,70,74,63,70,73,74,61,69,65,75,61,71,65,67,73,69,72,71,74,81,69,69,65,65,78,75,67,71,66,74,54,60,64,68,64,68,69,71,65,65,56,58,Yes
70,64,68,67,76,68,76,69,67,64,69,65,62,65,70,67,74,68,65,65,74,75,64,69,63,63,64,64,56,61,62,68,66,66,62,58,57,48,75,64,79,74,59,58,No
71,75,78,78,68,67,75,72,67,68,72,75,74,74,67,66,66,67,66,66,78,80,73,75,67,72,67,67,67,65,77,78,61,64,63,66,51,57,77,67,78,76,60,59,No
56,56,63,66,76,76,68,73,62,54,65,62,70,65,74,72,65,64,64,53,64,56,63,61,73,64,66,60,66,74,76,75,65,61,71,73,60,53,61,73,67,68,59,56,No
72,66,75,67,61,59,64,63,61,67,75,76,66,48,61,56,69,68,68,68,68,75,69,67,68,71,70,68,48,47,74,79,63,75,62,62,64,67,56,52,69,83,59,73,Yes
71,76,74,79,71,69,77,75,64,64,75,78,69,70,59,52,71,62,47,50,68,65,67,62,56,48,58,50,65,63,75,75,64,62,60,56,64,63,61,65,64,58,51,38,Yes
68,70,66,72,63,71,77,82,61,63,61,62,61,65,65,62,72,77,69,73,72,78,74,77,69,69,77,74,64,63,66,70,58,60,65,69,75,77,77,77,69,77,65,64,No
64,53,74,70,65,63,70,70,57,57,64,64,73,74,65,59,65,63,63,62,72,73,79,76,75,68,60,58,69,66,72,76,64,65,63,65,63,65,75,80,74,67,71,67,No
74,73,72,79,66,61,76,66,65,64,78,74,62,57,48,36,62,50,67,63,79,70,61,57,52,36,69,49,55,65,74,73,58,60,64,62,73,69,62,67,60,56,53,46,Yes
69,64,73,72,49,70,66,71,57,56,64,62,76,74,65,62,63,58,63,63,75,76,78,80,75,77,51,62,74,68,77,77,70,68,68,64,59,58,69,66,74,75,62,59,No
70,75,72,72,67,71,71,78,63,67,73,76,71,74,59,61,67,64,74,71,77,77,70,72,61,61,62,58,63,69,76,75,64,65,66,67,68,70,70,71,64,67,56,54,No
63,63,69,72,67,62,65,57,68,53,68,71,73,78,64,58,61,54,70,61,72,67,72,68,57,55,61,53,66,60,76,77,73,66,66,58,75,70,77,67,78,68,64,58,Yes
63,58,66,55,56,58,69,74,44,48,63,60,76,67,73,73,58,66,68,63,72,74,70,72,77,75,70,71,71,67,75,73,60,59,71,70,65,62,70,69,71,70,58,61,Yes
71,61,74,74,76,74,69,56,68,78,71,78,58,64,70,72,71,68,72,71,79,78,67,68,63,60,67,67,76,74,67,79,67,71,71,64,70,74,83,76,74,73,54,54,No
67,57,73,78,63,68,72,73,61,59,59,76,71,72,69,66,70,68,65,77,79,68,71,75,62,66,70,78,68,69,70,76,65,65,66,67,62,72,69,72,70,68,60,59,No
62,67,68,70,65,70,73,77,69,70,69,73,71,74,71,71,76,75,66,67,73,73,70,74,63,67,58,68,66,69,78,79,69,70,71,73,72,71,73,77,72,76,64,66,No
70,72,65,64,71,69,75,76,65,68,69,79,65,74,62,67,73,73,53,51,72,69,74,68,49,44,52,48,65,68,64,72,57,58,66,69,64,69,76,73,66,69,50,53,Yes
70,64,52,58,75,89,70,72,26,30,46,55,54,59,40,40,39,37,35,17,59,52,66,72,23,46,8,31,17,20,49,72,61,70,31,13,40,23,31,30,57,67,41,57,Yes
67,65,77,74,67,66,67,70,65,64,75,78,66,74,62,60,65,65,73,72,75,76,74,81,66,65,65,63,63,67,76,80,63,64,63,64,73,72,76,75,72,74,65,64,No
76,72,73,69,67,73,74,72,60,65,73,66,66,73,68,67,69,70,60,58,66,76,69,75,65,64,63,60,74,71,77,79,61,68,71,70,62,63,73,76,62,69,52,59,No
61,63,58,62,56,60,67,75,61,57,64,71,56,59,66,62,73,76,71,75,83,84,69,71,69,69,69,71,49,43,59,64,56,61,64,72,70,73,69,70,65,68,65,62,Yes
74,81,80,78,70,69,74,77,69,71,73,76,68,68,62,61,68,68,67,70,74,80,68,74,57,62,57,65,61,65,71,76,63,65,67,67,70,74,63,72,68,70,61,64,No
75,71,54,51,53,50,68,69,46,55,11,12,43,48,61,60,73,77,46,45,68,59,65,73,54,60,55,66,54,41,53,52,58,63,72,65,33,23,64,54,36,46,45,52,Yes
68,64,65,68,63,64,77,73,75,72,80,77,70,71,61,61,73,68,63,62,76,73,69,69,48,59,62,44,66,59,75,74,64,64,63,61,70,69,74,67,51,48,45,45,Yes
70,72,70,70,65,69,70,65,69,72,77,78,68,71,63,70,69,71,68,74,75,74,69,65,71,68,63,64,66,68,73,75,67,67,69,65,66,71,69,65,78,75,66,63,No
76,75,68,78,71,72,72,75,61,65,67,70,67,75,60,58,63,67,59,63,67,72,74,73,56,56,52,52,67,68,73,78,65,68,61,67,69,74,77,75,74,70,63,61,No
69,66,62,75,67,71,72,76,69,70,66,69,71,80,66,64,71,77,65,61,72,67,71,69,65,57,69,65,68,65,76,73,63,64,69,70,72,72,69,68,70,73,63,59,Yes
80,76,75,75,69,68,74,75,77,77,76,78,74,70,66,65,67,75,74,73,74,77,68,67,61,58,60,67,61,63,75,75,66,62,59,61,77,74,69,67,65,66,61,58,No
72,62,69,67,78,82,74,65,69,63,70,70,72,74,70,71,72,75,66,65,73,78,74,79,74,69,69,70,71,69,72,70,62,65,65,71,63,60,69,73,67,71,56,58,Yes
71,75,76,74,71,68,67,68,69,75,75,74,59,58,71,69,70,74,74,72,71,78,69,72,69,72,63,61,59,70,72,75,61,66,70,71,59,64,64,60,72,61,55,63,Yes
60,51,75,60,65,45,64,55,55,61,66,74,61,50,62,41,70,63,60,62,76,69,70,54,51,47,77,80,69,48,74,59,72,57,76,68,69,63,62,53,57,31,46,30,Yes
74,73,72,75,63,62,67,67,73,74,75,79,70,71,64,67,65,69,79,78,81,80,71,73,60,62,69,67,69,69,75,75,66,67,67,66,71,73,66,69,62,65,55,56,No
75,75,70,77,67,75,75,75,67,66,74,73,68,72,64,70,76,70,67,63,74,75,72,68,69,68,75,69,71,74,75,76,63,70,71,69,66,63,70,73,66,68,58,59,No
62,67,64,70,59,58,67,74,60,66,68,68,73,71,60,63,64,74,64,65,74,77,69,73,59,58,58,67,65,69,78,76,61,62,64,67,72,74,71,71,71,69,66,61,No
61,60,60,62,64,72,68,67,74,68,76,70,74,71,76,74,74,70,75,66,69,62,65,60,66,65,68,59,64,59,72,65,55,56,66,66,66,60,60,58,60,67,49,52,Yes
78,76,71,72,65,71,75,74,70,64,65,76,65,73,59,57,65,65,73,73,81,80,68,66,59,44,62,63,62,59,71,74,59,60,64,61,77,76,62,67,44,42,44,30,Yes
70,66,61,66,61,58,69,69,72,68,62,71,71,71,63,59,74,75,70,69,83,77,73,70,41,37,39,40,58,46,75,73,65,66,67,69,70,66,70,64,60,55,49,41,Yes
73,74,65,66,69,69,67,81,65,62,69,65,67,68,70,67,72,68,66,66,78,61,67,70,67,63,68,66,70,70,69,70,66,66,66,69,72,70,74,76,75,72,67,63,No
79,78,66,63,69,62,78,70,72,71,73,78,75,65,68,62,76,71,68,68,72,71,52,48,23,26,66,59,66,66,72,74,56,58,67,63,66,69,70,74,34,33,11,12,Yes
66,81,75,72,69,67,74,81,67,73,76,75,69,64,58,57,74,74,56,62,74,78,55,50,37,38,65,73,61,61,73,73,61,63,67,66,60,71,46,58,35,37,24,20,Yes
76,77,69,70,64,69,76,80,50,56,70,74,61,55,54,59,65,65,53,72,76,73,51,47,43,46,66,80,73,73,71,65,60,53,69,70,54,62,64,66,51,42,28,22,Yes
66,64,64,66,67,72,72,77,65,66,64,64,57,61,70,66,74,78,73,72,77,75,60,58,58,61,63,60,59,60,65,65,58,58,69,73,76,75,73,75,72,72,62,62,No
72,63,68,62,72,63,79,61,57,49,76,75,55,57,43,37,54,52,57,56,78,78,57,55,35,37,57,57,41,35,67,70,75,72,53,46,63,62,53,43,38,35,32,26,Yes
32,41,76,34,65,53,30,54,16,51,61,43,74,70,62,21,34,42,61,37,58,66,54,49,17,19,11,42,53,30,71,9,61,16,31,43,67,61,29,31,17,8,18,11,Yes
69,78,74,76,70,67,69,73,68,75,75,76,71,77,58,61,66,70,67,72,76,72,56,62,56,61,57,62,67,73,76,74,58,63,64,68,66,76,70,72,64,68,60,56,No
40,73,61,74,61,67,56,75,64,71,75,76,63,61,56,67,70,73,69,60,74,63,65,73,57,63,66,69,60,66,72,71,57,64,64,69,67,68,59,66,55,62,51,60,Yes
65,66,71,72,67,75,76,83,70,74,70,76,70,68,63,71,69,76,72,73,76,80,69,68,57,59,59,56,62,68,75,73,65,61,69,73,66,70,63,65,65,67,53,42,Yes
80,74,82,77,74,74,73,77,64,61,73,73,73,72,62,66,66,68,63,61,69,75,70,75,59,64,63,69,66,66,70,77,62,62,60,65,67,66,74,71,66,71,59,62,Yes
67,64,73,75,77,77,74,70,65,62,74,75,65,67,68,70,66,69,67,60,74,75,62,64,66,71,62,61,64,69,73,76,64,66,61,64,65,60,68,75,74,80,67,68,No
71,71,69,71,65,65,76,73,67,66,69,79,76,76,63,62,74,71,58,66,76,78,78,74,68,66,69,68,68,68,71,74,59,57,65,66,73,71,78,74,68,70,57,55,No
63,61,75,72,68,71,69,70,64,56,70,75,76,73,70,72,76,73,68,68,79,76,68,76,66,73,58,68,72,75,72,75,65,61,66,67,58,57,72,72,68,71,57,58,No
70,69,67,66,68,60,76,77,70,67,71,71,79,79,70,64,77,76,63,54,68,65,72,67,59,52,56,50,67,61,74,72,67,59,68,66,73,68,74,68,77,69,65,62,Yes
59,75,70,76,62,70,65,74,65,67,75,76,70,73,63,61,74,67,78,69,75,73,70,68,67,64,79,68,70,75,76,77,59,63,72,69,64,64,65,72,61,61,51,55,No
64,58,70,78,66,66,74,72,66,63,72,74,72,65,59,58,64,67,63,61,75,74,67,58,60,53,61,58,62,61,76,72,64,65,62,64,77,71,74,72,74,73,70,59,No
59,58,69,74,71,73,70,68,57,55,64,68,76,75,67,66,66,63,72,67,77,75,80,75,73,69,65,61,71,75,71,77,65,66,66,66,61,56,74,71,72,69,62,60,No
72,70,75,80,73,70,76,73,66,56,72,70,73,75,67,65,69,69,73,72,81,77,80,79,67,64,64,66,69,68,70,75,63,59,66,63,74,77,81,78,79,75,65,66,No
68,59,77,69,77,64,75,71,64,53,67,63,66,74,74,62,67,65,70,61,73,66,78,75,72,67,67,53,72,64,71,77,66,60,73,73,70,58,70,59,77,71,70,63,Yes
61,68,62,70,76,79,71,71,73,77,75,77,68,73,72,72,71,68,72,75,81,80,72,73,61,67,62,61,66,68,72,71,67,67,69,69,66,71,67,66,74,71,58,58,No
75,73,72,77,68,67,76,73,67,65,76,78,66,74,67,62,70,69,66,64,77,74,75,73,67,69,73,69,68,68,72,74,61,61,70,67,72,71,79,75,77,75,67,71,No
70,66,66,68,71,69,64,61,68,67,50,53,73,71,73,63,71,73,80,81,82,82,67,71,52,47,67,64,66,67,66,75,58,62,65,65,71,67,70,71,67,64,52,53,Yes
69,68,75,74,78,72,75,72,61,57,72,71,75,72,70,68,68,62,62,66,67,67,74,78,64,68,62,62,64,63,75,77,66,67,69,65,62,63,64,59,74,75,63,67,Yes
73,80,78,78,75,73,78,75,70,66,77,77,67,71,60,63,71,74,77,75,74,76,61,67,60,60,64,63,62,58,72,77,59,58,65,70,68,69,64,64,63,63,56,51,No
55,66,58,75,71,77,68,73,63,69,77,78,60,69,59,60,69,69,67,70,72,80,67,72,53,60,70,58,46,67,68,84,69,79,70,74,62,71,67,66,63,61,53,60,Yes
66,67,63,70,69,70,73,72,61,62,68,68,70,71,71,67,71,69,65,65,76,78,71,70,65,64,68,71,70,71,73,74,58,63,68,71,70,72,77,79,79,79,66,66,No
70,69,60,62,58,60,71,77,69,69,73,68,68,70,69,65,76,75,63,64,67,74,56,60,54,44,68,69,68,68,74,73,61,59,68,67,64,68,64,76,64,61,54,49,Yes
75,72,75,79,72,68,79,77,69,66,73,77,67,73,57,58,69,69,67,65,77,68,69,65,58,54,68,60,67,66,75,78,63,66,68,64,72,69,73,61,52,44,34,37,Yes
61,76,71,68,77,69,77,69,64,75,71,81,75,72,71,69,70,73,61,71,69,79,64,65,62,66,61,65,71,68,67,71,59,64,66,65,60,68,74,71,69,68,63,59,No
76,73,74,76,60,69,76,76,68,69,78,79,57,62,69,69,67,66,73,69,80,81,58,68,75,69,73,70,58,65,79,76,74,71,66,64,65,62,78,68,75,68,62,60,No
59,62,72,74,66,66,74,76,63,67,72,76,71,74,66,64,70,69,63,66,70,72,70,76,65,69,61,66,64,65,67,72,57,63,65,71,67,69,77,78,77,76,70,70,No
69,71,70,78,61,63,67,65,59,59,66,69,71,75,65,58,60,55,62,59,67,66,74,74,64,60,57,54,70,73,69,76,62,64,61,61,66,65,72,73,68,68,59,63,Yes
62,66,66,68,73,76,68,71,62,62,63,68,74,75,63,68,71,68,58,58,65,66,65,76,64,70,63,60,70,72,71,77,67,69,72,71,63,63,70,72,75,79,62,59,No
59,68,69,67,69,59,78,73,66,65,77,73,74,66,66,55,71,66,69,68,75,73,80,79,69,65,69,66,68,65,75,71,59,61,65,64,73,71,81,75,74,65,69,66,No
70,65,65,62,68,67,77,74,62,61,66,61,69,74,64,62,71,71,74,70,77,78,69,70,67,65,67,70,49,48,73,71,65,73,73,73,75,71,73,72,73,70,65,64,Yes
59,52,70,67,73,66,72,61,58,52,72,71,70,77,66,65,67,55,61,57,68,66,72,74,63,64,56,54,67,54,76,74,65,67,66,56,62,56,72,62,74,74,64,67,Yes
62,56,66,57,74,75,68,59,65,59,74,66,71,65,67,69,66,66,71,72,80,74,71,63,62,72,62,66,66,61,73,68,59,59,63,62,73,73,76,67,77,71,62,58,Yes
74,69,75,70,70,74,77,77,65,67,73,72,69,73,68,66,69,68,67,63,78,74,71,63,66,61,68,63,74,69,74,75,65,61,66,67,63,61,71,68,66,65,54,57,No
67,66,65,77,66,70,72,72,72,67,76,72,73,76,74,71,74,73,69,61,78,70,76,73,70,70,62,51,70,68,79,77,75,68,72,71,69,63,65,61,73,73,64,67,Yes
71,62,70,64,67,64,79,65,70,69,72,71,68,65,61,61,73,71,75,74,80,74,54,47,53,37,77,68,72,59,72,68,60,60,73,70,66,65,64,55,61,41,51,46,Yes
64,60,71,69,71,65,66,64,68,59,63,67,68,64,73,59,73,60,72,56,77,68,69,69,66,59,57,40,55,53,69,63,71,63,66,55,66,58,65,65,75,64,61,56,Yes
64,70,71,69,72,70,75,78,61,66,69,68,68,70,71,70,75,76,73,72,80,78,79,81,74,70,72,79,73,75,77,73,65,64,72,72,59,62,71,74,68,67,58,57,No
72,74,67,69,69,70,74,81,66,70,73,78,69,80,65,65,70,77,69,70,73,79,64,68,56,58,67,64,68,68,69,74,62,67,66,70,73,77,74,77,71,72,63,65,No
62,71,78,84,64,68,72,74,53,57,71,70,71,72,54,53,63,67,54,57,71,71,70,74,54,57,54,57,62,64,75,72,62,65,60,64,64,67,72,77,62,62,61,65,No
77,61,62,68,62,58,72,68,77,71,76,77,72,75,62,57,77,74,61,58,72,76,69,68,56,53,57,54,69,70,73,79,65,70,66,68,67,66,71,67,60,60,53,57,Yes
65,68,70,78,65,72,72,74,64,69,71,73,72,68,62,62,65,62,72,75,79,78,72,76,66,67,62,61,68,76,72,78,65,64,67,63,64,67,67,77,66,66,59,57,No
76,69,78,73,68,67,75,70,77,70,79,73,79,75,74,71,76,68,81,79,77,78,75,76,66,76,65,67,67,57,68,75,50,62,62,59,47,49,75,65,74,70,51,48,Yes
68,65,72,72,47,74,76,74,67,66,71,69,69,67,63,64,68,68,70,74,77,77,73,60,49,48,42,69,70,69,76,79,63,66,64,69,71,73,73,75,68,56,58,44,No
72,61,64,66,64,59,68,66,76,66,77,75,71,72,72,62,72,68,77,78,77,81,67,67,69,68,65,68,69,64,72,73,56,56,69,64,67,71,69,68,65,73,56,52,No
65,69,66,76,58,67,65,72,66,64,77,75,64,62,63,59,71,61,67,45,74,45,66,46,64,48,62,27,65,67,72,72,62,66,71,64,71,70,72,69,70,65,63,61,Yes
67,51,73,65,69,56,72,63,64,56,68,67,70,62,70,58,68,62,65,59,77,69,68,59,64,55,65,60,70,60,72,65,58,51,66,59,71,62,74,60,76,65,62,56,Yes
73,76,68,74,56,59,73,76,54,48,75,78,47,53,25,19,60,56,56,54,80,79,47,53,19,14,58,50,67,71,63,54,49,48,66,65,62,58,57,72,31,30,15,11,Yes
59,65,53,60,72,74,67,69,64,67,64,65,71,70,68,71,72,70,70,70,75,78,72,71,71,71,67,64,67,71,71,74,62,66,72,73,57,57,64,71,70,69,53,49,No
76,65,60,40,32,34,65,50,53,37,66,53,33,31,30,30,69,75,64,45,68,65,57,43,23,23,53,77,48,32,55,32,48,23,62,51,65,59,43,39,35,30,24,21,Yes
70,71,71,74,68,66,72,70,66,69,74,72,70,69,64,66,73,72,75,73,81,81,76,75,71,70,74,82,69,72,76,74,60,56,66,66,64,64,73,72,66,63,54,58,No
68,76,79,78,63,73,68,78,64,71,73,77,67,71,58,57,61,63,52,64,64,74,53,72,36,44,52,54,49,56,73,81,65,80,53,60,63,70,58,64,52,57,49,50,Yes
64,63,62,59,60,61,70,74,68,63,66,68,64,66,67,66,72,68,63,59,75,72,67,62,63,55,66,56,65,64,70,71,61,63,67,63,69,69,77,65,74,62,60,54,Yes
66,54,69,66,69,69,75,72,63,62,68,66,68,70,71,68,70,69,66,68,73,72,65,73,67,63,60,57,70,68,75,75,65,67,69,65,65,64,67,69,71,68,59,59,No