c © European Survey Research Association Estimation of the effects of measurement characteristics on the quality of survey questions (original) (raw)
Related papers
Estimation of the Effects of Measurement Characteristics on the Quality of Survey Questions
Survey Research Methods, 2007
When designing questionnaires, many choices have to be made. Because the consequences of these choices for the quality of the questions are largely unknown, it has often been said that designing a questionnaire is an art. To make it a more scientific activity we need to know more about the consequences of these choices. In order to further such an approach we have:
International Statistical Review, 2008
In this chapter, we will first discuss the difference between concepts-by-intuition and the concepts-by-postulation. After that we will illustrate the different ways in which concepts-by-postulation can be defined by concepts-by-intuition. In doing so, we will make a distinction between concepts-by-postulation, namely between concepts with reflective and formative indicators. These illustrations make it clear that there are many different ways to define concepts-by-postulation. The effects that the wording of survey questions can have on their responses have been studied in depth by
Just Why you Need a Well Designed and Properly Constructed Questionnaire
Questionnaire construction has overtime evolved with consistency and rarely, it has been skipped in the world’s researches. Questionnaires form the basis for which most pieces of information can be obtained. In the very light, response rates to questions and accuracy of data findings are possible through the use of questionnaire usage. Where a questionnaire is poorly constructed, one faces the risk of missing out vital information which could be forming the basis for research. This paper discusses the relevance and importance of questionnaire construction in data collection and research. An attempt is made to show questionnaire usage in social research and other research processes. Ultimately, questionnaire construction is considered just as important as any other research process used while collecting data. Some key recommendations that could make questionnaire usage in research better are also briefly considered.
Effects of questionnaire design on the quality of survey data
Public Opinion Quarterly, 1992
The importance of format, graphic layout, and question routing instructions on the quality of survey data has been acknowledged for self-administered survey instruments, but the significance of these factors for questionnaires administered by interviewers has not been recognized equally. This paper examines two studies in which interviewers used different questionnaire designs to administer the same set of survey questions to randomly selected adult respondents in housing units that made up area probability samples of the Detroit metropolitan area. The paper presents empirical evidence of specific effects (questions skipped in error, unprobed answers) directly attributable to the choice of questionnaire design. The analysis shows that questionnaire design choices can either help or hurt the quality of data collected by interviewers. Furthermore, the behaviors of experienced and inexperienced interviewers are affected in similar ways. In other words, interviewing experience does not compensate for format deficits in the design of survey instruments.
Design, Evaluation, and Analysis of Questionnaires for Survey Research
John Wiley & Sons, Inc. eBooks, 2007
In this chapter, we will first discuss the difference between concepts-by-intuition and the concepts-by-postulation. After that we will illustrate the different ways in which concepts-by-postulation can be defined by concepts-by-intuition. In doing so, we will make a distinction between concepts-by-postulation, namely between concepts with reflective and formative indicators. These illustrations make it clear that there are many different ways to define concepts-by-postulation. The effects that the wording of survey questions can have on their responses have been studied in depth by
Evaluating Surveys and Questionnaires
Critical Thinking in Psychology, 2006
Much of what we know about human behavior is based on self-reports. When we want to learn about individuals' health behaviors, consumer habits, family problems, media consumption, values or political beliefs, we ask appropriate questions. The answers provided to these questions serve as input into scientific analyses and provide the basis of statistical indicators used to describe the state of a society. Obviously, these data are only as meaningful as the questions we ask and the answers we receive. Moreover, whom we ask is of crucial importance to our ability to draw conclusions that extend beyond the particular people who answered our questions. Accordingly, the processes underlying question answering and the appropriate selection of respondents are of great importance to many areas of social research.
Evaluating survey questions: a comparison of methods
2010
This study compares five techniques to evaluate survey questions --expert reviews, cognitive interviews, quantitative measures of reliability and validity, and error rates from latent class models. It is the first such comparison that includes both quantitative and qualitative methods. We examined several sets of items, each consisting of three questions intended to measure the same underlying construct. We found low consistency across the methods in how they rank ordered the items within each set. Still, there was considerable agreement between the expert ratings and the latent class method and between the cognitive interviews and the validity estimates. Overall, the methods yield different and sometimes contradictory conclusions with regard to the 15 items pretested. The findings raise the issue of whether results from different testing methods should agree.
Using Response Reliability to Guide Questionnaire Design
1999
Background Often in recent years, Census Bureau staff working on demographic surveys have wanted to expand upon laboratory research by conducting split-sample field experiments to compare different questionnaire design strategies, wording, sequencing, etc. In most cases, the only available option has been to piggyback onto one of the large demographic surveys in production mode, which typically presents many constraints and
4 the Kiss Principle in Survey Design : Question Length and Data Quality
2016
Writings on the optimal length for survey questions are characterized by a variety of perspectives and very little empirical evidence. Where evidence exists, support seems to favor lengthy questions in some cases and shorter ones in others. However, on the basis of theories of the survey response process, the use of an excessive number of words may get in the way of the respondent’s comprehension of the information requested, and because of the cognitive burden of longer questions, there may be increased measurement errors. Results are reported from a study of reliability estimates for 426 (exactly replicated) survey questions in face-to-face interviews in six large-scale panel surveys conducted by the University of Michigan’s Survey Research Center. The findings suggest that, at least with respect to some types of survey questions, there are declining levels of reliability for questions with greater numbers of words and provide further support for the advice given to survey researc...