Concordance and consensus

Cees Elzinga; Hui Wang; Zhiwei Lin; Yash Kumar

doi:10.1016/j.ins.2011.02.001

Concordance and consensus

Cees Elzinga^*, Hui Wang, Zhiwei Lin, Yash Kumar

^*Corresponding author for this work

School of Electronics, Electrical Engineering and Computer Science

Research output: Contribution to journal › Article › peer-review

29 Citations (Scopus)

Abstract

This paper deals with the measurement of concordance and the construction of consensus in preference data, either in the form of preference rankings or in the form of response distributions with Likert-items. We propose a set of axioms of concordance in preference orderings and a new class of concordance measures. The measures outperform classic measures like Kendall's τ and W and Spearman's ρ in sensitivity and apply to large sets of orderings instead of just to pairs of orderings. For sets of N orderings of n items, we present very efficient and flexible algorithms that have a time complexity of only O(Nn²). Remarkably, the algorithms also allow for fast calculation of all longest common subsequences of the full set of orderings. We experimentally demonstrate the performance of the algorithms. A new and simple measure for assessing concordance on Likert-items is proposed.

Original language	English
Pages (from-to)	2529-2549
Number of pages	21
Journal	Information Sciences
Volume	181
Issue number	12
DOIs	https://doi.org/10.1016/j.ins.2011.02.001
Publication status	Published - 15 Jun 2011

Bibliographical note

Funding Information:
We express our gratitude to the anonymous reviewers for their suggestions that helped to substantially improve our paper. A major part of this research was conducted during the first authors’ visit to the Computer Science Research Institute of the University of Ulster, Northern Ireland, in June 2010, funded by the University of Ulster.

Copyright:
Copyright 2011 Elsevier B.V., All rights reserved.

Keywords

All common subsequences
Concordance
Consensus
Kendall τ
Kendall W
Likert item
Preference ranking
Rank correlation
String kernels

ASJC Scopus subject areas

Software
Control and Systems Engineering
Theoretical Computer Science
Computer Science Applications
Information Systems and Management
Artificial Intelligence

Access to Document

10.1016/j.ins.2011.02.001

Cite this

@article{543f4e0a9e14453398aa25d2aed7c8cb,

title = "Concordance and consensus",

abstract = "This paper deals with the measurement of concordance and the construction of consensus in preference data, either in the form of preference rankings or in the form of response distributions with Likert-items. We propose a set of axioms of concordance in preference orderings and a new class of concordance measures. The measures outperform classic measures like Kendall's τ and W and Spearman's ρ in sensitivity and apply to large sets of orderings instead of just to pairs of orderings. For sets of N orderings of n items, we present very efficient and flexible algorithms that have a time complexity of only O(Nn2). Remarkably, the algorithms also allow for fast calculation of all longest common subsequences of the full set of orderings. We experimentally demonstrate the performance of the algorithms. A new and simple measure for assessing concordance on Likert-items is proposed.",

keywords = "All common subsequences, Concordance, Consensus, Kendall τ, Kendall W, Likert item, Preference ranking, Rank correlation, String kernels",

author = "Cees Elzinga and Hui Wang and Zhiwei Lin and Yash Kumar",

note = "Funding Information: We express our gratitude to the anonymous reviewers for their suggestions that helped to substantially improve our paper. A major part of this research was conducted during the first authors{\textquoteright} visit to the Computer Science Research Institute of the University of Ulster, Northern Ireland, in June 2010, funded by the University of Ulster. Copyright: Copyright 2011 Elsevier B.V., All rights reserved.",

year = "2011",

month = jun,

day = "15",

doi = "10.1016/j.ins.2011.02.001",

language = "English",

volume = "181",

pages = "2529--2549",

journal = "Information Sciences",

issn = "0020-0255",

publisher = "Elsevier Inc.",

number = "12",

}

TY - JOUR

T1 - Concordance and consensus

AU - Elzinga, Cees

AU - Wang, Hui

AU - Lin, Zhiwei

AU - Kumar, Yash

N1 - Funding Information: We express our gratitude to the anonymous reviewers for their suggestions that helped to substantially improve our paper. A major part of this research was conducted during the first authors’ visit to the Computer Science Research Institute of the University of Ulster, Northern Ireland, in June 2010, funded by the University of Ulster. Copyright: Copyright 2011 Elsevier B.V., All rights reserved.

PY - 2011/6/15

Y1 - 2011/6/15

N2 - This paper deals with the measurement of concordance and the construction of consensus in preference data, either in the form of preference rankings or in the form of response distributions with Likert-items. We propose a set of axioms of concordance in preference orderings and a new class of concordance measures. The measures outperform classic measures like Kendall's τ and W and Spearman's ρ in sensitivity and apply to large sets of orderings instead of just to pairs of orderings. For sets of N orderings of n items, we present very efficient and flexible algorithms that have a time complexity of only O(Nn2). Remarkably, the algorithms also allow for fast calculation of all longest common subsequences of the full set of orderings. We experimentally demonstrate the performance of the algorithms. A new and simple measure for assessing concordance on Likert-items is proposed.

AB - This paper deals with the measurement of concordance and the construction of consensus in preference data, either in the form of preference rankings or in the form of response distributions with Likert-items. We propose a set of axioms of concordance in preference orderings and a new class of concordance measures. The measures outperform classic measures like Kendall's τ and W and Spearman's ρ in sensitivity and apply to large sets of orderings instead of just to pairs of orderings. For sets of N orderings of n items, we present very efficient and flexible algorithms that have a time complexity of only O(Nn2). Remarkably, the algorithms also allow for fast calculation of all longest common subsequences of the full set of orderings. We experimentally demonstrate the performance of the algorithms. A new and simple measure for assessing concordance on Likert-items is proposed.

KW - All common subsequences

KW - Concordance

KW - Consensus

KW - Kendall τ

KW - Kendall W

KW - Likert item

KW - Preference ranking

KW - Rank correlation

KW - String kernels

UR - http://www.scopus.com/inward/record.url?scp=79953030264&partnerID=8YFLogxK

U2 - 10.1016/j.ins.2011.02.001

DO - 10.1016/j.ins.2011.02.001

M3 - Article

AN - SCOPUS:79953030264

VL - 181

SP - 2529

EP - 2549

JO - Information Sciences

JF - Information Sciences

SN - 0020-0255

IS - 12

ER -

Concordance and consensus

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this