Let’s Measure Concentration and Dispersion in Ordinal Data

Abstract

In the social sciences, we often use ordinal data, like ‘agree’ to ‘disagree’. Frequently, we also want to know whether answers are clustered, whether they are polarized, etc. While many measures have been designed to capture what is variably referred to as agreement, consensus, concentration, or dispersion and polarization in ordinal data, little is known how they differ in practice—including how they react to changes in the distribution of responses. To better understand how different measures work in practice, we compare consensus scores across specific situations: constructed cases, simulated data where we know the underlying distribution, and empirical data. We highlight the similarities as well as the distinctive features of different approaches that have been developed across a disparate literature. In most cases, the different measures are highly correlated, but there are cases where the choice of method leads to substantially different conclusions. These situations are difficult for non-mathematical users to predict. We recommend a combination of measures for robustness and graphs to examine the distribution qualitatively.