Skip to contents

rankingQ: Design-Based Methods for Improving Ranking Questions

DOI

Overview

Ranking data offer valuable insights into social science by allowing researchers to study how people make comparative judgments about multiple social and political options. However, a common practical concern is that data collected from ranking survey questions are often prone to measurement error due to insensible, random responses.

rankingQ implements design-based methods to estimate various ranking-based quantities while correcting for such measurement error. With an additional ranking question for detecting random responses, it provides direct bias correction, inverse-probability weighting (IPW), visualization helpers, and diagnostics for assessing anchor-ranking questions.

For the underlying methodology, see Atsusaka and Kim (2025), “Addressing Measurement Errors in Ranking Questions for the Social Sciences,” Political Analysis, 33(4), 339-360. Visit the package site for vignettes and references.

Correction Inputs

rankingQ supports three ways to handle random or inattentive responding in its correction functions.

  • Use anc_correct when you have an anchor-ranking question.
  • Use p_random when you want to externally supply a plausible proportion of random or inattentive respondents.
  • Supply neither if you do not want to apply a correction; this is the default behavior, and the functions will let you know that no correction was applied. Both imprr_direct and imprr_weights will return the uncorrected estimates in this case, but still print useful outputs such as average rankings, top-k rankings, and so on.

Installation

Currently, you can install the development version from GitHub:

remotes::install_github("sysilviakim/rankingQ", dependencies = TRUE)

For a full walkthrough, see the Getting Started vignette.

Key Features

  • Bias correction via plug-in estimator (imprr_direct): estimates average ranks, marginal rank probabilities, pairwise preferences, and top-k rankings with confidence intervals
  • Bias correction via IPW (imprr_weights): reweights observed ranking distributions to correct for random responses
  • Visualization (plot_avg_ranking): plots corrected average rankings with uncertainty bounds
  • Diagnostics: tools for detecting bias and assessing anchor-ranking questions

Citation

If you use rankingQ, please cite:

Atsusaka, Yuki, and Seo-young Silvia Kim. 2025. “Addressing Measurement Errors in Ranking Questions for the Social Sciences.” Political Analysis 33(4): 339-360. https://doi.org/10.1017/pan.2024.33

@article{atsusaka_addressing_2025,
  author  = {Atsusaka, Yuki and Kim, Seo-young Silvia},
  title   = {Addressing Measurement Errors in Ranking Questions for the Social Sciences},
  journal = {Political Analysis},
  volume  = {33},
  number  = {4},
  pages   = {339--360},
  year    = {2025},
  doi     = {10.1017/pan.2024.33}
}