Concordance and consensus

Cees Elzinga*, Hui Wang, Zhiwei Lin, Yash Kumar

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

30 Citations (Scopus)


This paper deals with the measurement of concordance and the construction of consensus in preference data, either in the form of preference rankings or in the form of response distributions with Likert-items. We propose a set of axioms of concordance in preference orderings and a new class of concordance measures. The measures outperform classic measures like Kendall's τ and W and Spearman's ρ in sensitivity and apply to large sets of orderings instead of just to pairs of orderings. For sets of N orderings of n items, we present very efficient and flexible algorithms that have a time complexity of only O(Nn2). Remarkably, the algorithms also allow for fast calculation of all longest common subsequences of the full set of orderings. We experimentally demonstrate the performance of the algorithms. A new and simple measure for assessing concordance on Likert-items is proposed.

Original languageEnglish
Pages (from-to)2529-2549
Number of pages21
JournalInformation Sciences
Issue number12
Publication statusPublished - 15 Jun 2011

Bibliographical note

Funding Information:
We express our gratitude to the anonymous reviewers for their suggestions that helped to substantially improve our paper. A major part of this research was conducted during the first authors’ visit to the Computer Science Research Institute of the University of Ulster, Northern Ireland, in June 2010, funded by the University of Ulster.

Copyright 2011 Elsevier B.V., All rights reserved.


  • All common subsequences
  • Concordance
  • Consensus
  • Kendall τ
  • Kendall W
  • Likert item
  • Preference ranking
  • Rank correlation
  • String kernels

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence


Dive into the research topics of 'Concordance and consensus'. Together they form a unique fingerprint.

Cite this