Most traditional data mining algorithms struggle to cope with the sheer scale of data efficiently. In this paper, we propose a general framework to accelerate existing clustering algorithms to cluster large-scale datasets which contain large numbers of attributes, items, and clusters. Our framework makes use of locality sensitive hashing (LSH) to significantly reduce the cluster search space. We also theoretically prove that our framework has a guaranteed error bound in terms of the clustering quality. This framework can be applied to a set of centroid-based clustering algorithms that assign an object to the most similar cluster, and we adopt the popular K-Modes categorical clustering algorithm to present how the framework can be applied. We validated our framework with five synthetic datasets and a real world Yahoo! Answers dataset. The experimental results demonstrate that our framework is able to speed up the existing clustering algorithm between factors of 2 and 6, while maintaining comparable cluster purity.
|Title of host publication||Proceedings of the 2016 IEEE 32nd International Conference on Data Engineering (ICDE)|
|Publisher||Institute of Electrical and Electronics Engineers (IEEE)|
|Publication status||Published - 01 May 2016|
|Event||32nd IEEE International Conference on Data Engineering - Finland, Helsinki, Finland|
Duration: 16 May 2016 → 20 May 2016
|Conference||32nd IEEE International Conference on Data Engineering|
|Period||16/05/2016 → 20/05/2016|