K-Means is one of the best-known unsupervised learning algorithms. Unlike classification models, K-Means does not require class labels. Its goal is to discover natural groups, called clusters, inside the data.
For example, a company may have thousands of customers without predefined segments. K-Means can group them according to purchase value, frequency, engagement, or other numeric features.
The Core Idea
Suppose we want K = 3 clusters.
The algorithm starts with three cluster centers. Every data point is assigned to its nearest center. The center of each cluster is then recalculated using the mean of the points assigned to it.
This assignment-and-update cycle repeats until the cluster centers or assignments stop changing significantly.
Standard Steps
- Choose K.
- Initialize K centroids.
- Calculate the distance from each point to every centroid.
- Assign each point to the nearest centroid.
- Recompute each centroid as the mean of its assigned points.
- Repeat until convergence.
What Is a Centroid?
A centroid is the mean position of the points in a cluster.
For the points:
(2, 4)
(4, 6)
(6, 8)
the centroid is:
((2+4+6)/3, (4+6+8)/3) = (4, 6)
A centroid does not need to be an actual sample from the dataset.
Euclidean Distance
K-Means commonly uses Euclidean distance:
d = sqrt((x2 - x1)^2 + (y2 - y1)^2)
Each sample is assigned to the centroid with the smallest distance.
Simple Example
Consider:
(1, 1)(1, 2)(2, 1)(8, 8)(8, 9)(9, 8)
With K = 2, two natural groups exist: one around the lower coordinates and another around the higher coordinates.
As the algorithm iterates, the centroids move toward the centers of those two groups.
How Do We Choose K?
The number of clusters is one of the most important design decisions.
Elbow Method
Run K-Means using several K values and measure within-cluster error, often with a metric such as Within-Cluster Sum of Squares.
The error decreases as K increases, but at some point the improvement begins to slow. This bend in the graph resembles an elbow and may indicate a useful K.
Silhouette Score
The Silhouette Score measures how well a sample fits its own cluster compared with neighboring clusters.
It can also help compare different K values.
Feature Scaling
K-Means depends on distance, so feature scale matters.
If one variable ranges from 0 to 10 while another ranges from 0 to 1,000,000, the second variable may dominate the clustering.
Standardization or Min-Max Scaling is therefore often an important preprocessing step.
Centroid Initialization
K-Means can be sensitive to its initial centroids.
Poor initialization may lead to a weak local solution. K-Means++ is a popular initialization strategy designed to choose more useful starting centers.
The JavaScript implementation in the supplied project initializes cluster centers using the first K data points. This is simple for educational purposes, but production implementations commonly use stronger initialization or multiple runs.
What Is Convergence?
After every iteration, points may switch clusters and centroids may move.
When assignments stop changing, or centroid movement becomes sufficiently small, the algorithm is considered to have converged.
Practical Applications
Customer Segmentation
Customers can be grouped according to purchase frequency, monetary value, activity, or engagement.
Product User Segmentation
Application users can be clustered according to usage behavior.
Marketing Analysis
Clusters can reveal groups with different campaign responses or behavioral characteristics.
Image Compression
A classic example uses K-Means to group similar colors and reduce the number of colors in an image.
Exploratory Data Analysis
K-Means can provide a first look at possible structure in an unlabeled dataset.
Advantages
- Simple and intuitive
- Relatively efficient for many datasets
- Useful for discovering basic structure
- Suitable for segmentation tasks
- Scales reasonably well compared with many more complex clustering methods
Limitations
K Must Be Chosen
The algorithm does not automatically know the correct number of clusters.
Outlier Sensitivity
Extreme points can pull centroids away from representative locations.
Cluster Shape Assumptions
K-Means works best when clusters are relatively compact and separable with distance-based boundaries.
Initialization Sensitivity
Different initial centroids can produce different results.
Numeric Features
The standard algorithm is fundamentally based on means and distances, so purely categorical data requires different techniques.
Implementation Logic
The algorithm can be summarized as:
choose K centroids
repeat:
assign every point to nearest centroid
recompute each centroid as cluster mean
until assignments stop changing
The supplied JavaScript implementation builds a distance matrix, assigns every data point to its nearest cluster, recomputes centroid coordinates from the mean of assigned samples, and repeats until the assignments stabilize.
K-Means vs KNN
The similar names can be misleading.
KNN is generally supervised and uses labeled neighbors to predict a new sample.
K-Means is unsupervised and discovers groups without class labels.
Conclusion
K-Means is a foundational clustering algorithm. By repeatedly assigning points to the nearest centroid and updating those centroids, it gradually discovers useful structure in unlabeled data.
Successful use depends on choosing K carefully, scaling features, handling outliers, selecting sensible initialization, and checking whether the cluster shapes fit the assumptions of the algorithm.