Clustering-Based Ensemble Method as an Alternative to Stacking

Anna Jurek, Yaxin Bi, Shengli Wu, Chris D. Nugent

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

One of the most popular techniques of generating classifier ensembles is known as stacking which is based on a meta-learning approach. In this paper, we introduce an alternative method to stacking which is based on cluster analysis. Similar to stacking, instances from a validation set are initially classified by all base classifiers. The output of each classifier is subsequently considered as a new attribute of the instance. Following this, a validation set is divided into clusters according to the new attributes and a small subset of the original attributes of the instances. For each cluster, we find its centroid and calculate its class label. The collection of centroids is considered as a meta-classifier. Experimental results show that the new method outperformed all benchmark methods, namely Majority Voting, Stacking J48, Stacking LR, AdaBoost J48, and Random Forest, in 12 out of 22 data sets. The proposed method has two advantageous properties: it is very robust to relatively small training sets and it can be applied in semi-supervised learning problems. We provide a theoretical investigation regarding the proposed method. This demonstrates that for the method to be successful, the base classifiers applied in the ensemble should have greater than 50% accuracy levels.
Original languageEnglish
Pages (from-to)2120-2137
Number of pages18
JournalIEEE Transactions on Knowledge and Data Engineering
Volume26
Issue number9
Early online date20 Mar 2013
DOIs
Publication statusPublished - Sep 2014

Fingerprint Dive into the research topics of 'Clustering-Based Ensemble Method as an Alternative to Stacking'. Together they form a unique fingerprint.

Cite this