机器学习笔记之聚类 (Machine Learning -Clustering)

Machine Learning - Clustering 聚类

Concept

1. Clustering is a process to find similarity groups in data, called clusters
   • Group data instances that are similar or near to each other in one cluster
   • Data instances that are (very) different or far away from each other should be in differenct clusters
   • Clusters are unlabelled（未标记） and no a priori grouping of the data instances are given
   • Thus, ofter knowns as unsupervised（非监督性） learning

Approaches

• K-means
  $$labe{l}_i=\arg \min \sqrt{(\sum _{i=1}^n(x_i-a_j{)}^2)}$$
  $$a_j=\frac{1}{N(C_j)}\sum _{i\in C_j}{x}_i$$
  K-means algorithm assume sample set is: $$T=X_1,X_2,X_3,...,X_m$$
  according to Euclid distance formula [1], the algorithmic steps following：
• Determine(决定) the value for K (number of clusters)
• Randomly choose initial K centroids
• Repeat:
  • Assign each data point to the nearest centroid(中心点)
  • Update the centroids based on data partitioning
• Until the stopping criterion is met(直到达到终止条件)
  
  However, how to determine it is a good clustering(accoriding to K-means)?
• Minimise the Sum of Squared Error (SSE) from data points to their corresponding centroids
  $$SSE=\sum _{j=1}^k\sum {}_{x\in C_j}dist(x,m_j{)}^2$$
• $C_j$ denotes the $j^{th}$ cluster, $m_j$ is the centroid of cluster $C_j$, and $dist(x,m_j)$ denotes the distance between data point x and its centroid.
• Hence, the stopping criteria for the iterative estimation of the centroids is often based on the change in SSE
  • Very small changes in SSE indicates convergence.
  • Sometimes, fixed number of iterations is used.

Example:

• step 1: random isitialisation of centroids
  
• step 2: assign each data to nearest centroid
  
  -Step 3: recalculate centroids
  
• Repeat steps 2 and 3:
  
• Until converges
  

	import numpy as np
	import matplotlib.pyplot as plt
	from sklearn.datasets.samples_generator import make_blobs
	from sklearn.cluster import KMeans
	x, y = make_blobs(n_samples=2000, n_features=2, centers=[[-1,-1], [0,0], [1,1], [2,2]],cluster_std = [0.4, 0.2, 0.2, 0.2], random_state=9)
	plt.scatter(X[:, 0], X[:, 1], marker='+')
	plt.show()
	y_pred = KMeans(n_clusters=4, random_state=9).fit_predict(X)
	plt.scatter(X[:, 0], X[:, 1], c=y_pred)
	plt.show()

Summary

• Generally fast (although an iterative process)
• Still one of the most popular clustering algorithms
  • Fuzzified version often is more robust
• Have to know the number of clusters to start with, different K values obtain the different results
  • For some case, its not easy
• Provides a local solution
  • Results depends on initialisation
• Sensitive to outliers

[1] Euclide distance formula
欧几里得距离公式
Usually the Euclide distance in 2D dimension is the distance between two points.
2D dimension formula: $$distance=\sqrt{((x_1-x_2{)}^2+(y_1-y_2{)}^2)}$$
3D dimension formula: $$distance=\sqrt{((x_1-x_2{)}^2+(y_1-y_2{)}^2)+(z_1-z_2{)}^2}$$
So that if we follow this pattern:
ND dimension formula gonna be:
$$distance=\sqrt{\sum _i(x{i}_1-x{i}_2{)}^2}(i=1,2,...,n)$$

[2] chaowu1993.(2018).KMeans原理与源码实现.Retrieved from https://blog.csdn.net/weixin_40479663/article/details/82974625