Question Order Bias Revisited: A Split-Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users

Publikation: Bidrag til tidsskriftTidsskriftartikelfagfællebedømt

Standard

Question Order Bias Revisited : A Split-Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users. / Thau, Mads; Mikkelsen, Maria Falk; Pedersen, Mogens Jin; Larsen, Morten Hjortskov.

I: Public Administration, Bind 99, Nr. 1, 2020, s. 189-204.

Publikation: Bidrag til tidsskriftTidsskriftartikelfagfællebedømt

Harvard

Thau, M, Mikkelsen, MF, Pedersen, MJ & Larsen, MH 2020, 'Question Order Bias Revisited: A Split-Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users', Public Administration, bind 99, nr. 1, s. 189-204. https://doi.org/10.1111/padm.12688

APA

Thau, M., Mikkelsen, M. F., Pedersen, M. J., & Larsen, M. H. (2020). Question Order Bias Revisited: A Split-Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users. Public Administration, 99(1), 189-204. https://doi.org/10.1111/padm.12688

Vancouver

Thau M, Mikkelsen MF, Pedersen MJ, Larsen MH. Question Order Bias Revisited: A Split-Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users. Public Administration. 2020;99(1):189-204. https://doi.org/10.1111/padm.12688

Author

Thau, Mads ; Mikkelsen, Maria Falk ; Pedersen, Mogens Jin ; Larsen, Morten Hjortskov. / Question Order Bias Revisited : A Split-Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users. I: Public Administration. 2020 ; Bind 99, Nr. 1. s. 189-204.

Bibtex

@article{6b3e95c47adb44fe9a893879ececefc0,
title = "Question Order Bias Revisited: A Split-Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users",
abstract = "Public decision‐makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split‐ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large‐scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias.",
author = "Mads Thau and Mikkelsen, {Maria Falk} and Pedersen, {Mogens Jin} and Larsen, {Morten Hjortskov}",
year = "2020",
doi = "10.1111/padm.12688",
language = "English",
volume = "99",
pages = "189--204",
journal = "Public Administration",
issn = "0033-3298",
publisher = "Wiley-Blackwell",
number = "1",

}

RIS

TY - JOUR

T1 - Question Order Bias Revisited

T2 - A Split-Ballot Experiment on Satisfaction with Public Services among Experienced and Professional Users

AU - Thau, Mads

AU - Mikkelsen, Maria Falk

AU - Pedersen, Mogens Jin

AU - Larsen, Morten Hjortskov

PY - 2020

Y1 - 2020

N2 - Public decision‐makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split‐ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large‐scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias.

AB - Public decision‐makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split‐ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large‐scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias.

U2 - 10.1111/padm.12688

DO - 10.1111/padm.12688

M3 - Journal article

VL - 99

SP - 189

EP - 204

JO - Public Administration

JF - Public Administration

SN - 0033-3298

IS - 1

ER -

ID: 245416137