Using screen video capture software to aide and inform cognitive interviewing (original) (raw)
Related papers
RESEARCH SYNTHESIS: THE PRACTICE OF COGNITIVE INTERVIEWING PAULC.BEATTY GORDONB.WILLIS
Cognitive interviewing has emerged as one of the more prominent methods for identifying and correcting problems with survey questions. We define cognitive interviewing as the administration of draft survey questions while collecting additional verbal information about the survey responses, which is used to evaluate the quality of the response or to help determine whether the question is generating the information that its author intends. But beyond this general categorization, cognitive interviewing potentially includes a variety of activities that may be based on different assumptions about the type of data that are being collected and the role of the interviewer in that process. This synthesis reviews the range of current cognitive interviewing practices, focusing on three considerations: what are the dominant paradigms of cognitive interviewing-what is produced under each, and what are their apparent advantages; ( what key decisions about cognitive interview study design need to be made once the general approach is selected (e.g., who should be interviewed, how many interviews should be conducted, and how should probes be selected), and what bases exist for making these decisions; and (3) how cognitive interviewing data should be evaluated, and what standards of evidence exist for making questionnaire design decisions based on study findings. In considering these issues, we highlight where standards
Degree of cognitive interviewer involvement in questionnaire pretesting on trending survey modes
Modern technology allows for surveying through different media (e.g. Internet, mobile phones, tablets), which may influence the quality of collected data with additional effects due to the survey mode and should be pretested with the purpose of avoiding effects that would deteriorate the quality of collected data. In the present study, we analysed the technological development of surveying tools by testing the applicability of cognitive interviews on several different survey modes. We focused on the importance of the cognitive interviewer and the effect of reducing their degree of involvement on the quality of the interview results. We carried out personal interviews, interviews using voice-over-Internet protocol, interviews using programs for instant messaging, and web-based interviews; these enabled us to analyse the quality of each survey mode and recognize their advantages and deficiencies. Through the comparison of these modes and their assigned technique, we showed that the role of a cognitive interviewer is important for the quality of interviews regardless of the degree of their involvement in the survey process. However, the requirement for the pretesting situation to resemble the actual final process leads to a necessity to develop new, enhanced approaches to cognitive interviewing on trending survey modes.
Journal of Patient-Reported Outcomes
Background Cognitive interviewing is a well-established qualitative method used to develop and refine PRO measures. A range of digital technologies including phone, web conferencing, and electronic survey platforms can be leveraged to support the conduct of cognitive interviewing in both children and adults. These technologies offer a potential solution to enrolling underrepresented populations, including those with rare conditions, functional limitations and geographic or socioeconomic barriers. In the aftermath of the COVID-19 pandemic, the use of digital technologies for qualitative interviewing will remain essential. However, there is limited guidance about adapting cognitive interviewing procedures to allow for remote data capture, especially with children. Methods Synthesizing the literature and our research experiences during the COVID-19 pandemic, we examine considerations for implementing digitally supported cognitive interviews with children, adolescents, and adults. We of...
Research synthesis: The practice of cognitive interviewing
Public Opinion Quarterly, 2007
Cognitive interviewing has emerged as one of the more prominent methods for identifying and correcting problems with survey questions. We define cognitive interviewing as the administration of draft survey questions while collecting additional verbal information about the survey ...
The Mouse or the Pencil? A Psychometric Comparison of Web-Based and Traditional Survey Methodologies
This paper discusses the results of a study comparing the psychometric qualities of two forms of a survey, one administered in paperand-pencil format and the other administered in Web format. The survey addressed the topic of college course anxiety and was used to survey a sample of undergraduate students (n=36). The psychometric qualities investigated include the overall reliability (internal consistency) and item analyses of the 56 individual survey items for both the paper-and-pencil and Web surveys. With respect to reliability, results reveal a good deal of similarity between the two modes of survey delivery. Analysis of the paper-and-pencil survey data generated a Cronbach's alpha of 0.91; analysis of the Web survey data generated value of 0.88. Further internal comparisons of the item analyses revealed very similar patterns in the responses to individual items, indicating that the two modes of survey delivery were quite comparable. The Web version of the survey is attached. (Contains 1 table, 2 figures, and 17 references.) (Author/SLD) Reproductions supplied by EDRS are the best that can be made from the original document.
Cognitive Interviews to Test and Refine Questionnaires
Public Health Nursing, 2011
Survey data are compromised when respondents do not interpret questions in the way researchers expect. Cognitive interviews are used to detect problems respondents have in understanding survey instructions and items, and in formulating answers. This paper describes methods for conducting cognitive interviews and describes the processes and lessons learned with an illustrative case study. The case study used cognitive interviews to elicit respondents' understanding and perceptions of the format, instructions, items, and responses that make up the Diabetes Symptom Self-Care Inventory (DSSCI), a questionnaire designed to measure Mexican Americans' symptoms of type 2 diabetes and their symptom management strategies. Responses to cognitive interviews formed the basis for revisions in the format, instructions, items, and translation of the DSSCI. All those who develop and revise surveys are urged to incorporate cognitive interviews into their instrumentation methods so that they may produce more reliable and valid measurements.
Design, Evaluation, and Analysis of Questionnaires for Survey Research
John Wiley & Sons, Inc. eBooks, 2007
In this chapter, we will first discuss the difference between concepts-by-intuition and the concepts-by-postulation. After that we will illustrate the different ways in which concepts-by-postulation can be defined by concepts-by-intuition. In doing so, we will make a distinction between concepts-by-postulation, namely between concepts with reflective and formative indicators. These illustrations make it clear that there are many different ways to define concepts-by-postulation. The effects that the wording of survey questions can have on their responses have been studied in depth by
Journal of official statistics, 1994
Results are reported from a preliminary study testing a new technology for survey data collection: audio computer-assisted self interviewing. This technology has the theoretical potential of providing privacy (or anonymity) of response equivalent to that of paper self-administered questionnaires (SAQs). In addition, it could offer the advantages common to all computer-assisted methods such as the ability to implement complex questionnaire logic, consistency checking, etc.. In contrast to Video-CASI, Audio-CASI proffers these potential advantages without limiting data collection to the literate segment of the population. In this preliminary study, results obtained using RTI's Audio-CASI system were compared to those for paper SAQs and for Video-CASI. Survey questionnaires asking about drug use, sexual behavior, income, and demographic characteristics were administered to a small sample (N = 40) of subjects of average and below-average reading abilities using each method of data c...
Computer Audio-Recorded Interviewing as a Tool for Survey Research
Social Science Computer Review, 2013
""Computer Audio-Recorded Interviewing as a Tool for Survey Research" M. Rita Thissen; rthissen@rti.org; RTI International, PO Box 12194, Research Triangle Park, NC 27709, USA Best practices in software development call for developers to understand requirements from and potential benefits to the user population, especially for applications employed outside the computing profession. Computer audio-recorded interviewing (CARI) is coming into widespread use in survey research, because it allows a data collection manager to listen to the exact circumstances of how questions were asked and answered during the interview, a more powerful approach than any prior indirect quality control method. Yet design can be complex when planning an integrated system for capturing and reviewing audio recordings from interviews. This paper looks explores a part of the operational world of survey research from the eye of the system developer, offering context for those unfamiliar with survey research. Discussion focuses on benefits, requirements, user needs, system design challenges and interface options. "