Clustering

Clustering is an unsupervised learning technique that groups similar data points together based on their features. Unlike classification, clustering does not require labeled data — it discovers natural structures and patterns within the dataset.

Centroid-Based Clustering

K-Means

K-Means partitions data into k clusters by iteratively assigning points to the nearest centroid and updating centroids to the mean of their assigned points. It is fast and works well on spherical clusters but requires specifying k beforehand.

from sklearn.cluster import KMeans
import numpy as np

X = np.array([[1, 2], [1, 4], [1, 0],
              [10, 2], [10, 4], [10, 0]])

kmeans = KMeans(n_clusters=2, random_state=42, n_init='auto')
kmeans.fit(X)

print(f"Centroids: {kmeans.cluster_centers_}")
print(f"Labels: {kmeans.labels_}")

K-Medoids

K-Medoids is similar to K-Means but uses actual data points (medoids) as cluster centers instead of means. This makes it more robust to outliers and noise, as medians are less affected by extreme values than averages.

Mean-Shift Clustering

Mean-Shift is a non-parametric algorithm that does not require specifying the number of clusters. It works by shifting a sliding window toward regions of higher density until convergence, making it suitable for finding arbitrarily shaped clusters.

Hierarchical Clustering

Agglomerative Clustering

Agglomerative clustering is a bottom-up approach where each data point starts as its own cluster, and pairs of clusters are merged as they move up the hierarchy. The merging continues until a single cluster or a predefined number of clusters remains.

from sklearn.cluster import AgglomerativeClustering

hierarchical = AgglomerativeClustering(n_clusters=3, linkage='ward')
labels = hierarchical.fit_predict(X)
print(f"Hierarchical labels: {labels}")

Dendrograms

A dendrogram is a tree-like diagram that visualizes the hierarchical clustering process. The vertical lines represent merges, and the height indicates the distance at which clusters were merged. Cutting the dendrogram at a certain level gives a desired number of clusters.

Density-Based Clustering

DBSCAN

DBSCAN groups points that are closely packed together, marking points in low-density regions as outliers. It does not require specifying the number of clusters and can find arbitrarily shaped clusters. The two key parameters are eps (neighborhood radius) and min_samples.

from sklearn.cluster import DBSCAN

dbscan = DBSCAN(eps=3, min_samples=2)
labels = dbscan.fit_predict(X)
print(f"DBSCAN labels: {labels}")

# Note: -1 labels indicate noise/outliers

OPTICS

OPTICS (Ordering Points To Identify the Clustering Structure) addresses DBSCAN's difficulty with varying density clusters. It orders points so that clusters of different densities can be identified, making it more flexible than DBSCAN for real-world data.

HDBSCAN

HDBSCAN extends DBSCAN to find clusters of varying densities. It builds a hierarchy of clusters and extracts the most stable ones, automatically determining the optimal number of clusters without requiring the eps parameter.

BIRCH

BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) is designed for large datasets. It builds a Clustering Feature Tree (CF-tree) summarizing the data and then applies a global clustering algorithm on these summaries. It is memory-efficient and fast.

from sklearn.cluster import Birch

birch = Birch(n_clusters=2, threshold=0.5)
birch.fit(X)
print(f"BIRCH labels: {birch.labels_}")

Affinity Propagation

Affinity Propagation finds clusters by sending messages between data points about their relative suitability as cluster exemplars. It automatically determines the number of clusters based on the data, though it can be computationally expensive for large datasets.

Distribution-Based Clustering

Gaussian Mixture Models (GMM)

GMM assumes data is generated from a mixture of several Gaussian distributions. It uses the Expectation-Maximization (EM) algorithm to estimate the parameters (mean, covariance, weight) of each mixture component, assigning soft probabilities to cluster membership.

from sklearn.mixture import GaussianMixture

gmm = GaussianMixture(n_components=2, random_state=42)
gmm.fit(X)

print(f"Means: {gmm.means_}")
print(f"Covariances: {gmm.covariances_}")
print(f"Predictions: {gmm.predict(X)}")