[Artificial Intelligence] - Unsupervised learning, K-means clustering (K-means clustering), K-means loss function, objective function

unsupervised learning

Unsupervised learning refers to learning on unlabeled data, that is, model training without the guidance of supervisory signals. In unsupervised learning, we mainly focus on learning the low-dimensional structure and hidden patterns of data from unlabeled data.

With unlabeled data, we can predict the following:

1. Low-dimensional structure: Through unsupervised learning algorithms such as principal component analysis (PCA), we can discover the main direction of change and low-dimensional representation in the data, helping us understand the internal structure of the data and perform data dimensionality reduction.
2. Cluster structure: Through clustering algorithms such as K-means clustering or hierarchical clustering, we can divide unlabeled data into different groups or categories, thereby discovering the cluster structure in the data.
3. Association rules: Through association rule mining algorithms such as Apriori algorithm or FP-growth algorithm, we can discover frequent itemsets and association rules in the data, and reveal the association relationship between different features.

Clustering

• Group data objects into subsets or "clusters":

• High similarity within the cluster
• low similarity between clusters

• Clustering is a common and important task with widespread applications in science, engineering, information science, and other fields:

• Group genes with the same function
• Group individuals with similar political views
• Categorize documents with similar topics
• Identify similar objects from pictures

Input: Training set of input points
Output: Assign each point to a cluster
where $Dtrain=\{x1,...,xn\}$ is the training set of input points
$(C(1),...,C(n))$ is the cluster to which each point is assigned, where$C\left(i\right)$ belongs to{ 1 , . . . , k } \{1, ..., k\}{ 1,...,k } indicates the cluster to which the i-th point belongs.

K-means clustering (K-means clustering)

The goal of K-means is to minimize the sum of squares of the Euclidean distances between all data points and the center points of the clusters they belong to.

The algorithm process of K-means is as follows:

1. Randomly select K initial cluster center points (centroids).
2. Assigns each data point to the cluster corresponding to its nearest cluster center point.
3. Update the center point of each cluster to be the mean of all data points in that cluster.
4. Repeat step 2 and step 3 until the center point of the cluster no longer changes significantly or reaches the predetermined number of iterations.

K-means loss function

$\text{x}$ is the sample,$m_{C(j}$Represents a certain cluster $The\; center\; of\; C\left(j\right)$ ,
the following formula expresses that$x_j$assigned to $When\; C\left(j\right)$ is on this cluster, to the cluster center$m_{C(j}$The sum of the Euclidean distances of

objective function

Find out how to divide the cluster $C.$ How to choose the cluster center$\mu$ , so that the sum of the Euclidean distances from each cluster sample to the cluster center is the smallest

Strategy: Alternate Minimization

• Step 1: If the cluster centers are known, the best $C$

  • Fixed $\mu$ , optimize$C$
    
    assigns each point to the nearest cluster center

• Step 2: If you know the cluster allocation $C$ , can find the best cluster center$m$

  • Fixed $C$ , optimize$\mu$
    
    Solutions: Part$Select\; the\; average\; value\; of\; all\; points\; of\; the\; cluster\; in\; the\; i$ cluster as the cluster center again, which is exactly step 2 (reselect the cluster center)