Clustering is a fascinating aspect of machine learning, particularly in the realm of unsupervised learning. Unlike supervised learning which depends on labeled data, clustering groups data points based on their inherent similarities. This method uncovers hidden patterns that might not be immediately visible.
In this blog post, we will explore two key applications of clustering: user segmentation and recommender systems. Additionally, we will illustrate their effectiveness with a real-world example.
What is Clustering?
Clustering organizes data into groups called clusters in unsupervised machine learning. Each cluster consists of data points that are more alike than those in other clusters. This technique is often used with unlabeled data, making it useful for discovering hidden patterns or structures in the information. A simple example of how clusters look is given in the image below.
Key Features of Clustering
No Supervision Needed: Clustering is a type of learning that doesnโt need any set labels or results. It analyzes raw data to find natural groups on its own.
Grouping by Similarity: In clustering, data points are put together based on how similar they are. This similarity can be measured using different methods like Euclidean distance, Manhattan distance, or other special measures.
Use Cases Of Clustering
Segmenting Customers into Categories
For instance, an online retail site categorizes its customers into segments such as value shoppers, loyal patrons, and high-end consumers by analyzing their buying habits, willingness to pay and expenditure trends. This information is invaluable for targeting focused marketing strategies for different users.
Classifying Articles
For example, a news aggregation service sorts articles into sections like politics, sports, technology, and entertainment by examining keywords and similarities in content, which simplifies the process for users seeking specific news topics.
Compressing Image Sizes
In the realm of image optimization, clustering techniques (such as K-Means) group similar pixel shades to decrease the variety of colours, effectively lowering the file size while maintaining visual integrity. Don’t miss out on our latest blogs on computer vision- do check!
Identifying Anomalous Trends
For instance, financial institutions employ clustering methods to spot irregular transaction patterns, such as an unusual number of small withdrawals or international purchases, which could signal potential fraud.
Recommending Products or Services
Streaming services like Netflix categorize users based on their viewing habits and suggest films or series that are favoured by others within the same group, enhancing the personalization of the viewing experience.
Algorithms for Clustering
K-Means
K-Means is a widely used clustering algorithm that focuses on centroids to partition a dataset into a predetermined number of clusters (k). The main objective of this is to reduce the distance between data points and their corresponding cluster centroids, effectively grouping similar data points together. For each data point X and each cluster centroid C, the distance is calculated using the Euclidean distance formula:
How It Operates
1. Select the Number of Clusters (k): Determine how many clusters you wish to create.
2. Randomly Set Centroids: Choose k initial centroids at random from the dataset.
3. Allocate Data Points: For each data point, assign it to the closest centroid based on distance (typically using Euclidean distance).
4. Adjust Centroids: Update the centroids by calculating the average of all data points assigned to each cluster.
5. Iterate: Repeat steps 3 and 4 until the centroids stabilize, indicating that convergence has been reached.
When to Implement K-Means
Cluster Shape: Most effective for spherical or distinctly separated clusters.
Data Type: Best for large datasets featuring numerical attributes.
Scalability: Works efficiently with large datasets due to its linear complexity with the number of data points.
Example Use Case
A real-world application of K-Means clustering is in categorizing customers based on their spending habits in a retail environment. By examining purchasing behaviours and willingness to pay, retailers can pinpoint unique customer segments, allowing for targeted marketing efforts and customized promotions.
K-Means uses the mean (centroid) as the cluster center, which can be sensitive to outliers. On the other hand a very similar algorithm, K-Medoids uses actual data points (medoids) as cluster centers, making it more robust to outliers but computationally more expensive.
DBSCAN – Density-Based Spatial Clustering of Applications with Noise
DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a clustering algorithm that groups data points based on their density. It effectively identifies clusters of varying shapes and can distinguish noise and outliers from these clusters.
Understanding the Process
Set Parameters –
1. Epsilon (ฮต): This parameter determines the size of the neighborhood surrounding a data point.
2. MinPts: This is the minimum number of points we need to establish a dense area.
Procedure –
- Letโs Begin with a data point that hasn’t been visited yet.
- If this point has a minimum of MinPts neighbours within the ฮต radius, then we identify it as a core point, leading to the creation of a new cluster.
- If we falls short of this requirement, we categorise it as either noise or a border point.
- We can expand the cluster by including all points that are reachable within ฮต from the core points.
When to Use DBSCAN
- Suitable for datasets with arbitrary-shaped clusters.
- Effective when the data contains noise or outliers.
Example Application
A practical example of DBSCAN is detecting fraudulent transactions in financial data, such as identifying unusual spending patterns.
Hierarchical Clustering
Hierarchical clustering creates a structured arrangement of clusters by either combining smaller clusters (agglomerative) or dividing larger ones (divisive). This method results in a dendrogram, which visually illustrates the hierarchy of clusters.
How it Works:
- Start with each individual data point as a separate cluster
- Identify and merge the two closest clusters using a distance measure (such as Euclidean or Manhattan distance).
- Continue this process until all data points are unified into a single cluster.
- Utilize the dendrogram to determine the ideal number of clusters by “cutting” it at the preferred level.
When to Use: Ideal for smaller datasets or when a visual depiction of the cluster hierarchy is required.
Example: Classifying genes in bioinformatics according to their expression levels.
Comparison
| Algorithm | Strengths | Limitations |
| K-Means | Fast and efficient for large datasets. | Sensitive to initial centroids, the number of clusters (k) must be predefined, and it struggles with noise. |
| DBSCAN | Handles noise and arbitrary-shaped clusters well. | Struggles with varying densities and high-dimensional data. |
| Hierarchical | Creates a visual hierarchy of clusters, no need to predefine k. | Computationally expensive for large datasets. |
Applications of Clustering
1. User Grouping
Customer segmentation plays a vital role in the retail sector, enabling businesses to customize their marketing strategies and enhance customer experiences. A prime illustration of successful customer segmentation is found in the practices of H&M, a renowned global fashion retailer.
H&M’s Customer Segmentation Strategy
H&M utilizes demographic segmentation by leveraging customers’ birth dates to craft personalized marketing initiatives. This strategy allows the brand to deliver targeted promotions, such as birthday discounts, which not only boost customer engagement but also stimulate sales. For example, H&M provides a 25% discount on purchases made during a customer’s birthday month, encouraging shopping while making customers feel special.
Highlights of H&M’s Strategy:
Personalization: By sending tailored emails featuring birthday offers, H&M creates a sense of connection with its clientele.
Enhanced Engagement: The birthday promotion acts as a nudge for customers to either visit the store or shop online, leading to increased foot traffic and online transactions.
Customer Loyalty: These personalized interactions can strengthen customer loyalty, as shoppers feel valued and acknowledged.
2. Recommender System
Recommender systems are essential for improving user experiences on different platforms by offering personalized suggestions for products, movies, or services that align with individual tastes. Clustering is vital in these systems, as it organizes users with shared interests, allowing for tailored recommendations that reflect the collective actions of users within each group.
Streaming Services – Netflix
Streaming services like Netflix employ clustering methods to examine user viewing habits and preferences. By pinpointing groups of users with comparable interests, these platforms can suggest content that resonates with the collective tastes of each group. Here’s how it’s implemented –
- Data Gathering: The service gathers information on user activities, such as films viewed, ratings provided, and duration of viewing.
- User Clustering: Users are grouped according to their viewing behaviours. For instance, one group might include users who mainly enjoy action-thrillers.
- Content Recommendation: When a new action-thriller is launched, the system can propose it to all users in that group, even if they haven’t watched similar movies before. This strategy enhances the chances of user engagement, as the recommendations are tailored to the shared interests of the group.
Matrix Factorization Technique – Explained!
Matrix factorization is a method in machine learning commonly used in recommendation systems to estimate missing information, like how a user would rate an item (such as a movie, product, or song). The main concept is to divide a large and complex dataset (like a user-item interaction matrix) into smaller, more manageable pieces to reveal hidden patterns.
Now for all the new bees, what is matrix exactly here? Here the rows represent users. Columns represent items (e.g., movies, products, or books). Each cell in the matrix contains a number, such as the rating a user has given to an item. If the user hasnโt rated the item, the cell is empty (unknown). Our goal is to make sure if User A loves action movies and rates them highly, and a new action movie appears, the system might recommend it to User A.
How Does Matrix Factorization Work?
User Matrix (U): Shows what each user likes, such as their preference for action movies or comedies.
Item Matrix (V): Describes the features of each item, like if a movie is action, romantic, or a combination.
Multiplying these two matrices helps recreate the original matrix and can provide missing information.
Letโs say we have 3 users and 3 movies. Hereโs the matrix:
| Movie 1 | Movie 2 | Movie 3 | |
| User 1 | 5 | ? | 3 |
| User 2 | ? | 4 | ? |
| User 3 | 2 | ? | ? |
By using the formula –
R: User-Item matrix (known and unknown ratings), U: User matrix (users ร latent features) and V^T: Transposed Item matrix (latent features ร items). Note that , latent features are hidden patterns or characteristics in data that are not directly visible
STEP1 – Factorize R into U and V^T, such that U*V^T is closer to R, for known rating. So that we can estimate unknown ratings
V^T: Captures movie attributes (e.g., whether Movie 1 is action or romantic).
U: Captures user preferences (e.g., how much User 1 likes action movies, etc.).
STEP2 – Multiply U and V^T to approximate R, predicting missing values.
STEP3 – The final step is nothing but filling up in the blanks:
For an example – If User 1 and User 2 likes action movies, then their values in U matrix will be similar, such that when multiplied with V^T, the corresponding ratings to actions movies are high.
Matrix factorization is highly efficient, as it works well with sparse matrices containing many empty cells. It provides excellent personalization by uncovering hidden user preferences and item characteristics, making recommendations more accurate. Additionally, it is highly scalable, enabling its use in large systems like Netflix, Spotify, and Amazon to handle massive datasets effectively.
Conclusion
In summary, clustering is a powerful technique that significantly enhances the effectiveness of recommender systems. By grouping users with similar preferences, platforms like Netflix can deliver personalized content recommendations that resonate with individual tastes.
This not only improves user satisfaction but also fosters engagement and loyalty, ultimately driving success in competitive markets. As data continues to grow, the importance of clustering in tailoring user experiences will only increase, making it an essential tool for businesses aiming to connect with their audiences more effectively.