Achieving Natural Clustering by Validating Results of Iterative Evolutionary Clustering Approach

Abstract

Clustering is an essential process that leads to the classification of a given set of instances based on user-specified criteria; and different factors may lead to different clustering results. Thus, a large number of clustering algorithms exist to satisfy different purposes. However, scalability and the fact that algorithms in general need the number of clusters be specified apriori, which is mostly hard to estimate even for domain experts, are two challenges that motivate the development of new algorithms. This paper presents a novel approach to handle these two issues. We mainly developed a clustering method that works as an iterative approach to handle the scalability problem; and we utilize multi-objective genetic algorithm combined with validity indexes to decide on the number of clusters. The basic idea is to partition the dataset first; then cluster each partition separately. Finally, each obtained cluster is treated as a single instance (represented by its centroid) and a conquer process is performed to get the final clustering of the complete dataset. Test results on one large real dataset demonstrate the applicability and effectiveness of the proposed approach. © 2006 IEEE.