Coverage Error in Data Collection Combining Mobile Surveys With Passive Measurement Using Apps: Data From a German National Survey (original) (raw)
Related papers
RePEc: Research Papers in Economics, 2017
This paper examines non-response in a mobile app study designed to collect expenditure data. We invited 2,383 members of the nationally representative Understanding Society Innovation Panel in Great Britain to download an app to record their spending on goods and services: participants were asked to scan receipts or report spending directly in the app every day for a month. We examine coverage of mobile devices and participation in the app study at different stages of the process. We use data from the prior wave of the panel to examine the prevalence of potential barriers to participation, including access, ability and willingness to use different mobile technologies. We also examine bias in who has devices and in who participates, considering socio-demographic characteristics, financial position and financial behaviours. While the participation rate was low, drop out was also low: over 80% of participants remained in the study for the full month. The main barriers to participation were access to, and frequency of use of mobile devices, willingness to download an app for a survey, and general cooperativeness with the survey. We found extensive coverage bias in who has and does not have mobile devices, and some bias in who participates conditional on having a device. In the full sample, biases remain in who participates in terms of socio-demographic characteristics and financial behaviours. Crucially, however, we observe no biases for several key correlates of spending.
Survey research methods, 2019
This paper examines non-response in a mobile app study designed to collect expenditure data. We invited 2,383 members of the nationally representative Understanding Society Innovation Panel in Great Britain to download an app to record their spending on goods and services: participants were asked to scan receipts or report spending directly in the app every day for a month. We examine participation at different stages of the process. We further use data from the prior wave of the panel to examine the prevalence of potential barriers to participation, including access, ability and willingness to use different mobile technologies, and biases in the types of people who participate, considering socio-demographic characteristics, financial position and financial behaviours. While the participation rate was low, drop out was also low: over 80% of participants remained in the study for the full month. The main barriers to participation were access to, and frequency of use of mobile devices...
Survey Methods: Insights from the Field, 2013
With the decrease of landline phones in the last decade, telephone survey methodologists face a new challenge to overcome coverage bias. In this study we investigate coverage error for telephone surveys in Europe over time and compare two situations: classical surveys that rely on landline only with surveys that also include mobile phones. We analyzed Eurobarometer data, which are collected by means of face-to-face interviews and contain information on ownership of landline and mobile phones. We show that for the period 2000-2009, time has a significant effect on both mobile phone penetration and coverage bias. In addition, the countries’ development significantly affects the pace of these changes.
1 Mobile Phone Surveys : The Slovenian Case Study
2005
In 2004, the number of mobile phone subscriptions in Slovenia reached the total number of inhabitants. Consequently, the fixed telephone coverage has started to decline; almost 10% of households are now available only over the mobile phone. With this, Slovenia positions itself as a typical EU country and can serve as a case study for issues related to mobile phone interview surveys. The paper addresses the general context of mobile phone usage and the calculations of mobile phone coverage rates. It also discusses the non-coverage problems related to mobile and mobile-only households. It is shown, that even with a relatively small non-coverage the corresponding estimates can be considerably biased, as in the case of the unemployment rate in the Slovenian Labour Force Survey. There are severe methodological problems with mobile phone interview surveys. In particular, a pilot mobile phone survey confirmed the disadvantages of costs, frames and response rates, at least when compared to ...
Mobile Phone Surveys: The Slovenian Case Study
In 2004, the number of mobile phone subscriptions in Slovenia reached the total number of inhabitants. Consequently, the fixed telephone coverage has started to decline; almost 10% of households are now available only over the mobile phone. With this, Slovenia positions itself as a typical EU country and can serve as a case study for issues related to mobile phone interview surveys.
Quality & Quantity, 2012
The continuously growing mobile-only population raises concerns regarding the representativeness of traditional landline telephone surveys. At this time, the mobile-only population differs significantly from general population, which leads to coverage bias when using fixed-line samples only for telephone surveys. However, in many European countries the mobile-only population is not the only source of coverage bias in telephone surveys. In addition, we have to consider coverage biases caused by considerable proportions of citizens without any telephone service. Since these two groups differ from the general population with respect to differential socio-demographic categories, in our view, the negative effects of mobile-only coverage error in traditional landline telephone surveys might in fact compensate-in part-for coverage bias caused by the no-phone population. To test this hypothesis of compensating coverage biases we calculated relative coverage biases caused by the mobile-only population and relative coverage biases caused by the no-phone population in 30 European countries for two socio-demographic variables in two points in time. Results are presented for four groups of countries that differ with respect to no-phone and mobile-only rates. Results suggest that-in general-mobile-only biases and no-phone biases do not compensate to a great extent, and thus the alarming mobile-only biases cannot be neglected when using telephone surveys in the estimation of population parameters. Nevertheless, there are several countries where the bias caused by the mobile-only population is far bigger than the joint bias caused by the mobile-only population and the no-phone population. This finding suggests that biases caused by the recent mobile-only population would be even more severe if the no-phone population did not exist.
2004
The increase in mobile phone ownership is changing the sampling frame for landline telephone surveys, with a consequent impact on coverage error. This paper describes the main features of the Italian phone market -characterized by high mobile phone penetration ratesand the rising impact of mobile-phone-only (MPO) households. A survey that uses a landline sampling frame excludes MPO and no-phone households, creating a noncoverage rate of 17% in 2002. Types of phone arrangements and noncoverage vary dramatically among households: by region, household type, age, education and social class of the household. All these differences clearly introduce a non-ignorable bias in landline telephone surveys. Possible solutions are discussed from a methodological perspective. The analysis presented uses data collected in a face-to-face survey by the Italian Institute of Statistics.
Computers, Tablets, and Smart Phones: The Truth About Web-based Surveys
The exponential increase of smart phone, tablet, and laptop use places the topic of web-based surveys at the center of survey methodology discussions. As individuals now have a variety of options for taking online surveys, researchers must understand who completes their surveys through which device as it may impact completion rates and data quality. The analysis of two national online surveys (n=487 and n=1,046) revealed that individuals utilizing smart phones to complete the studies were signi cantly younger than those accessing surveys through computers, while only one study indicated a signi cant gender difference with females using smart phones more than males. Additionally, data showed that the respondent’s level of education did not signi cantly differ by device used to take the surveys.
Using mobile phones for survey research: a comparison with fixed phones
International Journal of Market Research, 2009
The increase in mobile phone penetration is stimulating a trend towards the use of mobile phones to supplement or even replace traditional telephone surveys. Despite this trend, few studies have systematically compared differences between the two modes. This paper describes a study in which both mobile and fixed phones were used to collect data on a national survey on internet and cultural practices. Findings revealed significant differences between mobile phone respondents and fixed phone respondents in terms of demographic characteristics and responses to some of the substantive items of the survey. In terms of data quality the mobile phone survey proved to be different from the fixed phone survey in two indicators: completion times and percentage of respondents with item omissions. The mobile phone survey was more difficult to implement than the fixed phone survey since much more screening was required to identify working phone numbers; in addition it yielded a lower response rate than the fixed phone survey.