Understanding uncertainty with a new take on questionnaires

Researchers from the University of Nottingham have been working to address uncertainty in questionnaires across a variety of problems with support from the UK EPSRC and the National Cyber Security Centre (NCSC).  They have developed an interval-valued response method as an alternative to the commonly used questionnaire approaches. Their research has been published in Behavior Research Methods.

Filling out surveys, questionnaires and reviews is a part of everyday life. These are used as a way of collecting feedback and data for a range of applications, but current models are inflexible in the responses they can collect, often using scales of 1-5, likely to unlikely, or agree to disagree, where the respondent can only choose one score. 

Christian Wagner, Professor of Computer Science, has been leading the research. He explains that “current methods for collecting quantitative or qualitative data each have their own limitations and we are exploring the viability of alternatives designed to maximize data capture for effort invested. Specifically, we have focused on an issue we believe has received relatively little attention given its potential significance—the efficient capture of uncertainty as perceived by an individual respondent. We have all been faced with a conventional scale and been unsure which option to choose, whether the question is badly worded or ambiguous, or whether we don’t know the exact answer or indeed feel that multiple options apply. Our research offers an alternative response system that takes this uncertainty into account, providing better quality data for little to no added effort. This has a lot of benefits, and it could even be used to improve data privacy in the future.” 

The new method uses ellipses to allow respondents to choose a range when answering a question, in an intuitive and accessible way. So instead of choosing one discrete point along the scale, the respondent can draw an interval around all options that apply. The team have tested these interval-valued questionnaires in a variety of areas; from vulnerability assessments in cyber security, to consumer perceptions of products, patient understanding in healthcare, and environmental management. 

Zack Ellerby, lead author of the paper elaborates that “data is used to inform decision making across a wide range of areas, and our research shows that there is an efficient way to capture better quality data without requiring much more time or money – and that people can effectively use this approach to communicate their uncertainty.”  

Source: Read Full Article