FCM algorithm
I. Introduction principle
Fuzzy C-means (Fuzzy C-means) algorithm referred to as FCM algorithm is a clustering algorithm based on fuzzy objective function is mainly used for cluster analysis of the data. Theory mature, widely used, is an excellent clustering algorithm.
First, some fuzzy concept is the so-called fuzzy uncertainty, uncertainty of what something is what it is, not what to say things like certainty. For example, the young and the young age of 20 as a standard, then a 21-year-old man is divided in accordance with certainty belong not young, but our impression is that the concept of 21-year-old and very young, this time you can blur a bit, 21-year-old believes there 0.9 points like the young, not young like 0.1 points, 0.9 and 0.1 is not here probability, but a similar degree, to which a sample belongs to a similar extent such membership is called the sample results, usually with u He said to represent a different result of a similar degree index sample.
Based on this assumed that the data set is X, if these data into the class c, then there is a class corresponding to the center c is C, each sample belonging to a group membership j i is uijuij, then define a FCM The objective function (1) and the constraints (2) as follows:
J = Σi = 1cΣj 1numij = xj || it || 2 (1)
(1) J = Σi = 1cΣj 1nuijm = xj || it || 2
Σi = 1cuij = 1, j = 1,2 ..., n (2)
(2) = Σi 1cuij = 1, j = 1,2 ..., n
Look at the objective function (Formula 1) with knowledge, by respective samples of the membership of each class from the center of the sample multiplied composition, m is a membership factor, the degree of personal understanding of the samples belonging to gentle, it like x2x2 and x3x3 such as. Formula (2) as the constraint condition, which is a sample belonging to all classes of membership To sum 1. Observation formula (1) can be found, where the variables have uij, ciuij, ci, and there are constraints, how this objective function extremum it?
Here the first Lagrange multiplier method constraints to the objective function to get preceded by coefficients, and equation (2) to expand all j, then becomes formula (1) shown below:
J = [Sigma i = 1cΣj = 1numij || xj- ci || 2 + λ1 (Σi = 1cui1-1) + ... + λj (Σi = 1cuij-1) + ... + λn (Σi = ncuin -1)) (. 3)
(. 3) J = [sigma] l = 1cΣj XJ-CI = 1nuijm || + || 2 lambda ([sigma] l = 1cui1-1) + ... + [lambda] j ([sigma] l = 1cuij- 1) + ... + λn (Σi = ncuin-1))
Now it requires the objective function extreme formula, wherein the variables are then uij, ciuij, ci the derivative, firstly uijuij derivative.
Analysis of formula (3), the first derivative of the two summing uijuij first portion, if the form of the summation of the direct derivation unfamiliar to be expanded as the sum:
⎡⎣⎢⎢⎢⎢⎢⎢⎢⎢⎢ ⎢um11 || x1-c1 || 2 ⋮ umi1 || x1-ci || 2 ⋮ umc1 || x1-cc || 2 ⋯ ⋮ ⋯ ⋮ ⋯ um1j || xj-c1 || 2 ⋮ umij || xj- ci || 2 ⋮ umcj || xj- cc || 2 ⋯ ⋮ ⋯ ⋮ ⋯ um1n || xn-c1 || 2 ⋮ umin || xn-ci || 2 ⋮ umcn || xn-cc || 2⎤⎦ ⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥
[u11m C1-X1 || || || 2 ⋯ u1jm XJ-C1 || || 2 ⋯ u1nm Xn-C1 || || 2 ⋮⋮⋮⋮⋮ ui1m X1- ci || 2 ⋯ uijm || xj- ci || 2 ⋯ uinm || xn-ci || 2 ⋮⋮⋮⋮⋮ uc1m || x1-cc || 2 ⋯ ucjm || xj-cc || 2 ⋯ ucnm || xn-cc || 2]
This matrix requires uijuij turned on, see umij || xj-ci || 2uijm || xj-ci || 2 uijuij corresponding reserved only, all other entries because they do not contain uijuij, so the derivative is 0 . Then umij || xj-ci || 2uijm || xj-ci || 2 is a pair of uijuij derivative after m || xj-ci || 2um- 1ijm || xj-ci || 2uijm-1.
Seeking to look behind that uijuij guide, the same sum expand, and then remove irrelevant uijuij (derivation is 0), then only this one: λj (uij-1) λj (uij-1), it uijuij the derivation is λjλj.
Then the final result of the derivative J and allowed uijuij is equal to 0:
∂J∂uij m || = || 2um XJ-CI-1ij + [lambda] j = 0
∂J∂uij XJ = m-CI || || 2uijm -1 + λj = 0
Simplifying this expression, it is the solution uijuij:
UM-1ij = XJ-CI -λjm || || 2
uijm. 1-XJ = -λjm || || 2-CI
Further:
uij = (- XJ λjm || || 2-CI). 1-1M = (- λjm). 1-1M (CI-XJ. 1 || || (. 1-2M)) (. 4) (. 4) = uij (-λjm || xj-ci || 2 ) 1m-1 = (- λjm) 1m-1 (1 || xj-ci || (2m-1))
to a solution should λjλj uijuij need to remove the job. Here again using Equation (2) constraints, and to count out uijuij substituted into the formula (2) are:
1 = Σi = 1cuij = Σi = 1c (-λjm) 1m-1 (1 || xj-ci || (2m-1)) = (- λjm) 1m-1Σi = 1c (1 || xj -ci || (2m-1)) 1 = Σi = 1cuij = Σi = 1c (-λjm) 1m-1 (1 || xj-ci || (2m-1)) = (- λjm) 1m- 1Σi = 1c (1 || xj- ci || (2m-1))
so there is (wherein the symbol i into K):
(-λjm). 1-1M = (1C 1Σi = (. 1 | | xj-ci || (2m- 1))) = (1Σk = 1c (1 || xj-ck || (2m-1))) (- λjm) 1m-1 = (1Σi = 1c (1 || xj-ci || ( 2m-1))) = (1Σk = 1c (1 || xj-ck || (2m-1)))
this back into Equation (4) has :
uij = (1C 1Σk = (XJ-CK. 1 || || (2M-. 1))) (CI-XJ. 1 || || (2M-. 1)) = (1C 1Σk = (|| xj-ci || (2m-1 )) || xj-ck || (2m-1))) = 1Σk = 1c (|| xj-ci |||| xj-ck ||) (2m- 1) (5) (5) uij = (1Σk = 1c (1 || xj-ck || (2m-1))) (1 || xj-ci || (2m-1)) = (1 Σk = 1c (|| xj-ci || (2m-1)) || xj-ck || (2m-1))) = 1Σk = 1c (|| xj-ci |||| xj- ck ||) (2m-1)
well, equation (5) is the final uijuij iterative formula.
The following is the derivative of cici to seek J. By equation (2) can be seen only Σi = 1cΣj = 1numij || xj- ci || 2Σi = 1cΣj = 1nuijm || xj-ci || 2 which contains this part cici, its two as previously shown expand summing stage, then its derivative is cici:
∂J∂ci = Σj = 1N (-umij * 2 * (XJ-CI)) = = Σj = 1N 0∂J∂ci (-uijm * 2 * (xj- ci)) = 0
, ie:
Σj = 1N (umijci) = Σj = 1N (xjumij) Σj = 1N (uijmci) = Σj = 1N (xjuijm) CI = Σj = 1n (xjumij) Σj = 1numij (6) (6) ci = Σj = 1n (xjuijm) Σj = 1nuijm
well, equation (6) is iterative formula class center.
We found uijuij with cici are interrelated, included at each other, there is a problem is when the fcm algorithm is neither the beginning nor uijuij cici, it is up to how to solve it? Very simple, when the program started we just assigned to one of uijuij or cici, as long as the value of the condition can be. Then began iteration, such as general are assigned to uijuij, it can be calculated with uijuij cici, then they can calculate with cici uijuij, repeatedly, in this process there is an objective function J has been changing gradually tend to be stable value. So when J is not considered changing the algorithm converges to a good knot. I can see uijuij and cici in the objective function J seems to constitute a positive feedback, like, much like the EM algorithm, the first E in M, with M in E, M until the optimum.
Equation (5), (6) is the key to the algorithm. Now again from a macro perspective to the overall look of these two formulas, look at (5), written again in
uij = 1Σk = 1c (|| xj -ci |||| xj-ck ||) (2m -1)
uij = 1Σk = 1C (CI |||| || XJ-XJ-CK ||) (. 1-2M)
Hypothesis sample 1 to the sample set membership of each cluster center, then the time j = 1, i from 1 to class c, the denominator of the above formula which case the summation, this molecule is the point with respect to a certain class of class center distance and the denominator is the point relative to the center from all classes of the class sum, they represent what the division two, is not that the point to a class at the center of the distance between the point and the center of all classes proportion. When the sum is smaller molecule which is not to say that the closer the class, then the overall score the greater, the greater is the corresponding uijuij, the more specific class, the image is as follows:
again see Macro See equation (6), i consider when determining the type, the denominator sum of formula (6) is actually a constant, then the formula (6) can be written as:
CI = Σj = 1N (xjumij) = Σj = Σj 1numij 1numijΣj = 1numijxj =
CI = Σj = 1N (xjuijm) = Σj = Σj = 1nuijm 1nuijmΣj = 1nuijmxj
This is an updated law class center. Before saying this, let us think about how kmeans class center is updated, usually the simplest is to find all sample points belonging to a class, then this type of class center is the average of these sample points. So how the FCM class center? See equation is a weighted average can be found, the class i is determined, all points to the first class membership summed u, then for each point, and this is divided by the membership proportion, multiplied xjxj this is the point for this kind of contribution to the value of i.
(B) to achieve a simple procedure
we achieve the above-described process FCM formula (5) and (6) with at most basic cycle matlab below. First, we need to generate the data can be used for FCM, in order to facilitate visualization, we generated two-dimensional data to facilitate a display on the coordinate axes, each sample is characterized by two (or x-coordinate and y-coordinate configuration), generating 100 this point, of course, we look at the man-made changes, so that these points belong to at least look different classes. Generated stippling out as follows:
we say that the general steps FCM algorithm is as follows:
(1) determine the number of categories, the value of the index m, determine the number of iterations (this is the end of the condition, of course, the end can have a variety of conditions).
(2) initializing a membership U (note that conditions - and 1);
(3) is calculated according to the cluster center C U;
(. 4) may be calculated this time the objective function J
(5) returns to C is calculated according to U, back to step 3, has until the end of the cycle.
Also it needs to be said that it is important, when the end of the program, how to determine which category it belongs to what point? At the end, there must be the last calculation of U of it, for every point, it belongs to each class, there will be a u, u then find the maximum of which it considers the points fall into this category. Based on a foundation of this program is as follows:
CLC
Clear
Close All
%% Create the Samples:
for I =. 1: 100
X1 (I) = RAND () *. 5;% guarantee human difference
Y1 (I) = RAND () *. 5;
X2 (I) = RAND () * 5 + 3;% artificial ensure difference
Y2 (I) = RAND () * 5 + 3;
End
X = [X1, X2];
Y = [Y1, Y2];
Data = [X; Y];
Data = data ';% Usually each row represents a data sample
% plot (data (:, 1 ), data (:, 2),' * ');% drawn
%% ---
cluster_n = 2; the number of classes%
iter = 50;% iterations
m = 2;% index
num_data = size (data, 1) ;% The number of samples
num_d = size (data, 2) ;% Sample Dimension
% - initialized membership u, and with the proviso that each column is. 1
the U-= RAND (cluster_n, num_data);
col_sum SUM = (the U-);
the U-= U./col_sum(ones(cluster_n,1), :);
%% cycle - a predetermined number of iterations as the end condition
for I =. 1: ITER
% update C
for J =. 1: cluster_n
u_ij_m the U-= (J,:). ^ m;
sum_u_ij = SUM (u_ij_m);
sum_1d = u_ij_m./sum_u_ij;
. C (J, :) = u_ij_m * Data / sum_u_ij;
End
% - calculated objective function J
temp1 = zeros ( cluster_n, num_data);
for = J. 1: cluster_n
for K =. 1: num_data
temp1 (J, K) = the U-(J, K) ^ m * (NORM (Data (K,:) - C (J, :)) ) ^ 2;
End
End
J (I) = SUM (SUM (temp1));
% to update the U-
for J =. 1: cluster_n
for K =. 1: num_data
SUM1 = 0;
for J1 =. 1: cluster_n
TEMP = (NORM (Data (K,:) - . C (J,:)) / NORM (Data (K,:) - C (J1,:))) ^ (2 / (m-. 1));
SUM1 = SUM1 + TEMP;
End
the U-(J, K) 1./sum1 =;
End
End
End
Figure;
the subplot (1,2,1), Plot (J);
[~, label] = max (the U-);% found class belongs
subplot (1,2,2);
gscatter (data (:, 1) , data (:, 2), label)
to give the following results:
Based on this, the result is fairly correct. But I have to say a problem is the efficiency of the algorithm. For calculation formulas and consistent, easy to understand, this program there are a lot of cycling, when a large number of categories that sample a little more time, so writing is very slow, matlab known as matrix laboratory, so use as little as possible cycle, direct matrix operation, the above operation can be cyclic many places into the matrix calculation, here to introduce the matlab fcm own function, is used for the matrix operation.
Matlab下help fcm既可以查阅相关用法们这里只是简单介绍，fcm函数输入需要2个或者3个参数，返回3个参数，如下：
[center, U, obj_fcn] = fcm(data, cluster_n, options)
对于输入：data数据集（注意每一行代表一个样本，列是样本个数）
cluster_n为聚类数。
options是可选参数，完整的options包括：
OPTIONS(1): U的指数 (default: 2.0)
OPTIONS(2): 最大迭代次数 (default: 100)
OPTIONS(3): 目标函数的最小误差 (default: 1e-5)
OPTIONS(4): 是否显示结果 (default: 1，显示)
options都有默认值，自带的fcm结束的条件是OPTIONS(2)与OPTIONS(3)有一个满足就结束。
对于输出：center聚类中心，U隶属度，obj_fcn目标函数值，这个迭代过程的每一代的J都在这里面存着。
为了验证我们写的算法是否正确，用它的fcm去试试我们的数据（前提是数据一样），分成3类，画出它们的obj_fcn看看如下：
可以看到，虽然迭代的中间过程不一样，但是结果却是一样的。
K-means算法
一.原理介绍
K-means算法是最为经典的基于划分的聚类方法，是十大经典数据挖掘算法之一。K-means算法的基本思想是：以空间中k个点为中心进行聚类，对最靠近他们的对象归类。通过迭代的方法，逐次更新各聚类中心的值，直至得到最好的聚类结果。
k-means 算法接受参数 k ，然后将事先输入的n个数据对象划分为 k个聚类以便使得所获得的聚类满足，同一聚类中的对象相似度较高，而不同聚类中的对象相似度较小。聚类相似度是利用各聚类中对象的均值所获得一个“中心对象”（引力中心）来进行计算的。
实现原理
KMeans算法的基本思想是初始随机给定K个簇中心，按照最邻近原则把待分类样本点分到各个簇。然后按平均法重新计算各个簇的质心，从而确定新的簇心。一直迭代，直到簇心的移动距离小于某个给定的值。
K-Means聚类算法主要分为三个步骤：
(1)第一步是为待聚类的点寻找聚类中心；
(2)第二步是计算每个点到聚类中心的距离，将每个点聚类到离该点最近的聚类中去；
(3)第三步是计算每个聚类中所有点的坐标平均值，并将这个平均值作为新的聚类中心；
反复执行(2)、(3)，直到聚类中心不再进行大范围移动或者聚类次数达到要求为止。
下图展示了对n个样本点进行K-means聚类的效果，这里k取2：
(a)未聚类的初始点集；
(b)随机选取两个点作为聚类中心；
(c)计算每个点到聚类中心的距离，并聚类到离该点最近的聚类中去；
(d)计算每个聚类中所有点的坐标平均值，并将这个平均值作为新的聚类中心；
(e)重复(c),计算每个点到聚类中心的距离，并聚类到离该点最近的聚类中去；
(f)重复(d),计算每个聚类中所有点的坐标平均值，并将这个平均值作为新的聚类中心。
该算法的最大优势在于简洁和快速。算法的关键在于初始中心的选择和距离公式。
数据描述：Iris数据集包含150个牡丹花模式样本，其中 每个模式样本采用5维的特征描述
X = (x1,x2,x3,x4,w);
x1: 萼片长度(厘米）
x2: 萼片宽度(厘米）
x3: 花瓣长度(厘米）
x4：花瓣宽度(厘米）
w(类别属性 ): 红牡丹(Iris setosa)，白牡丹(Iris versicolor)和黄牡丹(Iris virginica)
% 函数功能 ： 实现对iris.data数据集的分类，并根据分类结果进行精度评价
clear;clc;close all;
%手动选取文件，选用iris.data
[filename,fpath] = uigetfile(...
{...
'*.m;*.txt;*.data',...
'Data(*.m;*.txt;*.data)';...
'*.*','All Files(*.*)'...
},'Select data');
if ~filename
return
end
% filename = 'iris.data';
[X1,X2,X3,X4,X5] = textread(filename,'%f%f%f%f%s','delimiter',','); %Get data
clear filename fpath
X = [X1 X2 X3 X4];
[m,~] = size(X);
%分配索引
DataLabel = zeros(m,1);
DataLabel(strcmp(X5,'Iris-setosa')) = 1;
DataLabel(strcmp(X5,'Iris-versicolor')) = 2;
DataLabel(strcmp(X5,'Iris-virginica')) = 3;
clear m X5
%二维结果
[MyCenter1,ClusterLabel12] = FindCluster([X1 X2],3,DataLabel);
[MyCenter2,ClusterLabel13] = FindCluster([X1 X3],3,DataLabel);
[MyCenter3,ClusterLabel14] = FindCluster([X1 X4],3,DataLabel);
[MyCenter4,ClusterLabel23] = FindCluster([X2 X3],3,DataLabel);
[MyCenter5,ClusterLabel24] = FindCluster([X2 X4],3,DataLabel);
[MyCenter6,ClusterLabel34] = FindCluster([X3 X4],3,DataLabel);
hold on;
subplot(231)
MyPlot2(X1,X2,DataLabel,MyCenter1),xlabel('X1'),ylabel('X2')
subplot(232)
MyPlot2(X1,X3,DataLabel,MyCenter2),xlabel('X1'),ylabel('X3')
subplot(233)
MyPlot2(X1,X4,DataLabel,MyCenter3),xlabel('X1'),ylabel('X4')
subplot(234)
MyPlot2(X2,X3,DataLabel,MyCenter4),xlabel('X2'),ylabel('X3')
subplot(235)
MyPlot2(X2,X4,DataLabel,MyCenter5),xlabel('X2'),ylabel('X4')
subplot(236)
MyPlot2(X3,X4,DataLabel,MyCenter6),xlabel('X3'),ylabel('X4')
clear MyCenter1 MyCenter2 MyCenter3 MyCenter4 MyCenter5 MyCenter6
%三维结果
[MyCenter7,ClusterLabel123] = FindCluster([X1,X2,X3],3,DataLabel);
[MyCenter8,ClusterLabel124] = FindCluster([X1,X2,X4],3,DataLabel);
[MyCenter9,ClusterLabel134] = FindCluster([X1,X3,X4],3,DataLabel);
[MyCenter10,ClusterLabel234] = FindCluster([X2,X3,X4],3,DataLabel);
hold on;
figure,title('3D');
subplot(221)
MyPlot3(X1,X2,X3,DataLabel,MyCenter7)
xlabel('X1'),ylabel('X2'),zlabel('X3');
subplot(222)
MyPlot3(X1,X2,X4,DataLabel,MyCenter8)
xlabel('X1'),ylabel('X2'),zlabel('X4');
subplot(223)
MyPlot3(X1,X3,X4,DataLabel,MyCenter9)
xlabel('X1'),ylabel('X3'),zlabel('X4');
subplot(224)
MyPlot3(X2,X3,X4,DataLabel,MyCenter10)
xlabel('X2'),ylabel('X3'),zlabel('X4');
clear MyCenter7 MyCenter8 MyCenter9 MyCenter10
%聚类精度评价
%二维结果
ClusterLabel_2D = [ClusterLabel12,ClusterLabel13,ClusterLabel14...
ClusterLabel23,ClusterLabel24,ClusterLabel34];
ClusterAccuracy_2D = Accuracy(DataLabel,ClusterLabel_2D);
clear ClusterLabel12 ClusterLabel13 ClusterLabel14
clear ClusterLabel23 ClusterLabel24 ClusterLabel34 ClusterLabel_2D
%三维结果
ClusterLabel_3D = [ClusterLabel123,ClusterLabel124,ClusterLabel134,ClusterLabel234];
ClusterAccuracy_3D = Accuracy(DataLabel,ClusterLabel_3D);
clear ClusterLabel123 ClusterLabel124 ClusterLabel134 ClusterLabel234
clear ClusterLabel_3D
FindCluster.m
%函数功能 ： 输入数据集、聚类中心个数与样本标签
% 得到聚类中心与聚类样本标签
function [ClusterCenter,ClusterLabel] = FindCluster(MyData,ClusterCounts,DataLabel)
[m,n] = size(MyData);
ClusterLabel = zeros(m,1); %用于存储聚类标签
% MyLabel = unique(DataLabel,'rows');
% for i = 1:size(MyLabel,2);
% LabelIndex(1,i) = i; %为数据标签创建索引
% end
%已知数据集的每个样本的中心
OriginCenter = zeros(ClusterCounts,n);
for q = 1:ClusterCounts
DataCounts = 0;
for p = 1:m
%按照数据标签，计算样本中心
if DataLabel(p) == q
OriginCenter(q,:) = OriginCenter(q,:) + MyData(p,:);
DataCounts = DataCounts + 1;
end
end
OriginCenter(q,:) = OriginCenter(q,:) ./ DataCounts;
end
%按照第一列对样本中心排序
%排序是为了解决新聚类中心因随机分配而与样本最初的聚类中心不匹配的问题
SortCenter1 = sortrows(OriginCenter,1);
FalseTimes = 0;
CalcuateTimes = 0;
%此循环用于纠正分类错误的情况
while (CalcuateTimes < 15)
ClusterCenter = zeros(ClusterCounts,n); %初始化聚类中心
for p = 1:ClusterCounts
ClusterCenter(p,:) = MyData( randi(m,1),:); %随机选取一个点作为中心
end
%此循环用于寻找聚类中心
%目前还未解决该循环陷入死循环的问题，所以设置一个参数来终止循环
kk = 0;
while (kk < 15)
Distance = zeros(1,ClusterCounts); %存储单个样本到每个聚类中心的距离
DataCounts = zeros(1,ClusterCounts); %记录每个聚类的样本数目
NewCenter = zeros(ClusterCounts,n);
for p = 1:m
for q = 1:ClusterCounts
Distance(q) = norm(MyData(p,:) - ClusterCenter(q,:));
end
%index返回最小距离的索引，亦即聚类中心的标号
[~,index] = min(Distance);
ClusterLabel(p) = index;
end
k = 0;
for q = 1:ClusterCounts
for p = 1:m
%按照标记，对样本进行分类,并计算聚类中心
if ClusterLabel(p) == q
NewCenter(q,:) = NewCenter(q,:) + MyData(p,:);
DataCounts(q) = DataCounts(q) + 1;
end
end
NewCenter(q,:) = NewCenter(q,:) ./ DataCounts(q);
%若新的聚类中心与上一个聚类中心相同，则认为算法收敛
if norm(NewCenter(q,:) - ClusterCenter(q,:)) < 0.1
k = k + 1;
end
end
ClusterCenter = NewCenter;
%判断是否全部收敛
if k == ClusterCounts
break;
end
kk = kk + 1 ;
end
%再次判断每个聚类是否分类正确，若分类错误则进行惩罚
trueCounts = ClusterCounts;
SortCenter2 = sortrows(ClusterCenter,1);
for p = 1:ClusterCounts
if norm(SortCenter1(p,:) - SortCenter2(p,:)) > 0.5
trueCounts = trueCounts - 1;
FalseTimes = FalseTimes + 1;
break;
end
end
if trueCounts == ClusterCounts
break;
end
CalcuateTimes = CalcuateTimes + 1;
end
% FalseTimes
% CalcuateTimes
% trueCounts
% kk
% DataCounts
% OriginCenter
% NewCenter
%理论上每个聚类的标签应是123排列的，但实际上，由于每个聚类中心都是随机选取的，
%实际分类的顺序可能是213，132等，所以需要对分类标签进行纠正，这对之后的精度评
%价带来了方便。
%对分类标签进行纠正：
%算法原理：从第一个已知的样本中心开始，寻找离其最近的聚类中心，然后将归类于该
% 聚类中心的样本的聚类标签更换为i
for i = 1:ClusterCounts %遍历原始样本中心
for j = 1:ClusterCounts %遍历聚类中心，与原样本中心比较
if norm(OriginCenter(i,:) - ClusterCenter(j,:)) < 0.6
% for p = 1:m
% if ClusterLabel(p) == j
% ClusterLabel(p) = 2 * ClusterCounts + i;
% end
% end
ClusterLabel(ClusterLabel == j) = 2 * ClusterCounts + i;
end
end
end
ClusterLabel = ClusterLabel - 2 * ClusterCounts;
%Temp = [MyData(:,:),ClusterLabel]
%函数功能 : 输入样本，样本标签及求出的聚类中心，显示二维图像，实现数据可视化
function MyPlot2(X1,X2,DataLabel,ClusterCenter)
[m,~] = size(X1);
hold on;
% for p = 1:m
% if(DataLabel(p) == 1)
% plot(X1(p),X2(p),'*r')
% elseif(DataLabel(p) == 2)
% plot(X1(p),X2(p),'*g')
% else
% plot(X1(p),X2(p),'*b')
% end
% end
p = find(DataLabel == 1);
plot(X1(p),X2(p),'*r')
p = find(DataLabel == 2);
plot(X1(p),X2(p),'*g')
p = find(DataLabel == 3);
plot(X1(p),X2(p),'*b')
% xlabel(who('X1'));
% ylabel(who('X2'));
%PS ： 我想在坐标轴中根据输入的形参的矩阵的名字，转换成字符串，来动态输出
% 坐标轴名称，不知道该怎么做？用上面注释的语句不行。。
[n,~] = size(ClusterCenter);
plot(ClusterCenter(1:1:n,1),ClusterCenter(1:1:n,2),'ok')
grid on;
MyPlot3.m
function MyPlot3(X1,X2,X3,DataLabel,ClusterCenter)
[m,~] = size(X1);
hold on;
% for i = 1:m
% if(DataLabel(i) == 1)
% plot3(X1(i),X2(i),X3(i),'.r')
% elseif(DataLabel(i) == 2)
% plot3(X1(i),X2(i),X3(i),'.g')
% else
% plot3(X1(i),X2(i),X3(i),'.b')
% end
% end
p = find(DataLabel == 1);
plot3(X1(p),X2(p),X3(p),'.r')
p = find(DataLabel == 2);
plot3(X1(p),X2(p),X3(p),'.g')
p = find(DataLabel == 3);
plot3(X1(p),X2(p),X3(p),'.b')
% xlabel('X1');
% ylabel('X2');
% zlabel('X3');
%PS ： 我想在坐标轴中根据输入的形参的矩阵的名字，转换成字符串，来动态输出
% 坐标轴名称，不知道该怎么做？用上面注释的语句不行。。
[n,~] = size(ClusterCenter);
% for i = 1:n
% plot3(ClusterCenter(i,1),ClusterCenter(i,2),ClusterCenter(i,3),'ok')
% end
plot3(ClusterCenter(1:1:n,1),ClusterCenter(1:1:n,2),ClusterCenter(1:1:n,3),'ok')
view([1 1 1]);
grid on;
运行结果：
数据集：
5.1,3.5,1.4,0.2,Iris-setosa
4.9,3.0,1.4,0.2,Iris-setosa
4.7,3.2,1.3,0.2,Iris-setosa
4.6,3.1,1.5,0.2,Iris-setosa
5.0,3.6,1.4,0.2,Iris-setosa
5.4,3.9,1.7,0.4,Iris-setosa
4.6,3.4,1.4,0.3,Iris-setosa
5.0,3.4,1.5,0.2,Iris-setosa
4.4,2.9,1.4,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
5.4,3.7,1.5,0.2,Iris-setosa
4.8,3.4,1.6,0.2,Iris-setosa
4.8,3.0,1.4,0.1,Iris-setosa
4.3,3.0,1.1,0.1,Iris-setosa
5.8,4.0,1.2,0.2,Iris-setosa
5.7,4.4,1.5,0.4,Iris-setosa
5.4,3.9,1.3,0.4,Iris-setosa
5.1,3.5,1.4,0.3,Iris-setosa
5.7,3.8,1.7,0.3,Iris-setosa
5.1,3.8,1.5,0.3,Iris-setosa
5.4,3.4,1.7,0.2,Iris-setosa
5.1,3.7,1.5,0.4,Iris-setosa
4.6,3.6,1.0,0.2,Iris-setosa
5.1,3.3,1.7,0.5,Iris-setosa
4.8,3.4,1.9,0.2,Iris-setosa
5.0,3.0,1.6,0.2,Iris-setosa
5.0,3.4,1.6,0.4,Iris-setosa
5.2,3.5,1.5,0.2,Iris-setosa
5.2,3.4,1.4,0.2,Iris-setosa
4.7,3.2,1.6,0.2,Iris-setosa
4.8,3.1,1.6,0.2,Iris-setosa
5.4,3.4,1.5,0.4,Iris-setosa
5.2,4.1,1.5,0.1,Iris-setosa
5.5,4.2,1.4,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
5.0,3.2,1.2,0.2,Iris-setosa
5.5,3.5,1.3,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
4.4,3.0,1.3,0.2,Iris-setosa
5.1,3.4,1.5,0.2,Iris-setosa
5.0,3.5,1.3,0.3,Iris-setosa
4.5,2.3,1.3,0.3,Iris-setosa
4.4,3.2,1.3,0.2,Iris-setosa
5.0,3.5,1.6,0.6,Iris-setosa
5.1,3.8,1.9,0.4,Iris-setosa
4.8,3.0,1.4,0.3,Iris-setosa
5.1,3.8,1.6,0.2,Iris-setosa
4.6,3.2,1.4,0.2,Iris-setosa
5.3,3.7,1.5,0.2,Iris-setosa
5.0,3.3,1.4,0.2,Iris-setosa
7.0,3.2,4.7,1.4,Iris-versicolor
6.4,3.2,4.5,1.5,Iris-versicolor
6.9,3.1,4.9,1.5,Iris-versicolor
5.5,2.3,4.0,1.3,Iris-versicolor
6.5,2.8,4.6,1.5,Iris-versicolor
5.7,2.8,4.5,1.3,Iris-versicolor
6.3,3.3,4.7,1.6,Iris-versicolor
4.9,2.4,3.3,1.0,Iris-versicolor
6.6,2.9,4.6,1.3,Iris-versicolor
5.2,2.7,3.9,1.4,Iris-versicolor
5.0,2.0,3.5,1.0,Iris-versicolor
5.9,3.0,4.2,1.5,Iris-versicolor
6.0,2.2,4.0,1.0,Iris-versicolor
6.1,2.9,4.7,1.4,Iris-versicolor
5.6,2.9,3.6,1.3,Iris-versicolor
6.7,3.1,4.4,1.4,Iris-versicolor
5.6,3.0,4.5,1.5,Iris-versicolor
5.8,2.7,4.1,1.0,Iris-versicolor
6.2,2.2,4.5,1.5,Iris-versicolor
5.6,2.5,3.9,1.1,Iris-versicolor
5.9,3.2,4.8,1.8,Iris-versicolor
6.1,2.8,4.0,1.3,Iris-versicolor
6.3,2.5,4.9,1.5,Iris-versicolor
6.1,2.8,4.7,1.2,Iris-versicolor
6.4,2.9,4.3,1.3,Iris-versicolor
6.6,3.0,4.4,1.4,Iris-versicolor
6.8,2.8,4.8,1.4,Iris-versicolor
6.7,3.0,5.0,1.7,Iris-versicolor
6.0,2.9,4.5,1.5,Iris-versicolor
5.7,2.6,3.5,1.0,Iris-versicolor
5.5,2.4,3.8,1.1,Iris-versicolor
5.5,2.4,3.7,1.0,Iris-versicolor
5.8,2.7,3.9,1.2,Iris-versicolor
6.0,2.7,5.1,1.6,Iris-versicolor
5.4,3.0,4.5,1.5,Iris-versicolor
6.0,3.4,4.5,1.6,Iris-versicolor
6.7,3.1,4.7,1.5,Iris-versicolor
6.3,2.3,4.4,1.3,Iris-versicolor
5.6,3.0,4.1,1.3,Iris-versicolor
5.5,2.5,4.0,1.3,Iris-versicolor
5.5,2.6,4.4,1.2,Iris-versicolor
6.1,3.0,4.6,1.4,Iris-versicolor
5.8,2.6,4.0,1.2,Iris-versicolor
5.0,2.3,3.3,1.0,Iris-versicolor
5.6,2.7,4.2,1.3,Iris-versicolor
5.7,3.0,4.2,1.2,Iris-versicolor
5.7,2.9,4.2,1.3,Iris-versicolor
6.2,2.9,4.3,1.3,Iris-versicolor
5.1,2.5,3.0,1.1,Iris-versicolor
5.7,2.8,4.1,1.3,Iris-versicolor
6.3,3.3,6.0,2.5,Iris-virginica
5.8,2.7,5.1,1.9,Iris-virginica
7.1,3.0,5.9,2.1,Iris-virginica
6.3,2.9,5.6,1.8,Iris-virginica
6.5,3.0,5.8,2.2,Iris-virginica
7.6,3.0,6.6,2.1,Iris-virginica
4.9,2.5,4.5,1.7,Iris-virginica
7.3,2.9,6.3,1.8,Iris-virginica
6.7,2.5,5.8,1.8,Iris-virginica
7.2,3.6,6.1,2.5,Iris-virginica
6.5,3.2,5.1,2.0,Iris-virginica
6.4,2.7,5.3,1.9,Iris-virginica
6.8,3.0,5.5,2.1,Iris-virginica
5.7,2.5,5.0,2.0,Iris-virginica
5.8,2.8,5.1,2.4,Iris-virginica
6.4,3.2,5.3,2.3,Iris-virginica
6.5,3.0,5.5,1.8,Iris-virginica
7.7,3.8,6.7,2.2,Iris-virginica
7.7,2.6,6.9,2.3,Iris-virginica
6.0,2.2,5.0,1.5,Iris-virginica
6.9,3.2,5.7,2.3,Iris-virginica
5.6,2.8,4.9,2.0,Iris-virginica
7.7,2.8,6.7,2.0,Iris-virginica
6.3,2.7,4.9,1.8,Iris-virginica
6.7,3.3,5.7,2.1,Iris-virginica
7.2,3.2,6.0,1.8,Iris-virginica
6.2,2.8,4.8,1.8,Iris-virginica
6.1,3.0,4.9,1.8,Iris-virginica
6.4,2.8,5.6,2.1,Iris-virginica
7.2,3.0,5.8,1.6,Iris-virginica
7.4,2.8,6.1,1.9,Iris-virginica
7.9,3.8,6.4,2.0,Iris-virginica
6.4,2.8,5.6,2.2,Iris-virginica
6.3,2.8,5.1,1.5,Iris-virginica
6.1,2.6,5.6,1.4,Iris-virginica
7.7,3.0,6.1,2.3,Iris-virginica
6.3,3.4,5.6,2.4,Iris-virginica
6.4,3.1,5.5,1.8,Iris-virginica
6.0,3.0,4.8,1.8,Iris-virginica
6.9,3.1,5.4,2.1,Iris-virginica
6.7,3.1,5.6,2.4,Iris-virginica
6.9,3.1,5.1,2.3,Iris-virginica
5.8,2.7,5.1,1.9,Iris-virginica
6.8,3.2,5.9,2.3,Iris-virginica
6.7,3.3,5.7,2.5,Iris-virginica
6.7,3.0,5.2,2.3,Iris-virginica
6.3,2.5,5.0,1.9,Iris-virginica
6.5,3.0,5.2,2.0,Iris-virginica
6.2,3.4,5.4,2.3,Iris-virginica
5.9,3.0,5.1,1.8,Iris-virginica三.优缺点
优点：原理简单，实现容易
缺点：
1.收敛太慢
2.算法复杂度高O(nkt)
3.不能发现非凸形状的簇，或大小差别很大的簇
4.需样本存在均值（限定数据种类）
6.需先确定k的个数
7.对噪声和离群点敏感
8.最重要是结果不一定是全局最优，只能保证局部最优。