Non-response biases in surveys of school children: the case of the English PISA samples (original) (raw)
Related papers
Journal of the Royal Statistical Society Series a, 2012
The Case of the English PISA Samples * We analyse response patterns to an important survey of school children, exploiting rich auxiliary information on respondents' and non-respondents' cognitive ability that is correlated both with response and the learning achievement that the survey aims to measure. The survey is the Programme for International Student Assessment (PISA), which sets response thresholds in an attempt to control data quality. We analyse the case of England for 2000 when response rates were deemed high enough by the PISA organisers to publish the results, and 2003, when response rates were a little lower and deemed of sufficient concern for the results not to be published. We construct weights that account for the pattern of nonresponse using two methods, propensity scores and the GREG estimator. There is clear evidence of biases, but there is no indication that the slightly higher response rates in 2000 were associated with higher quality data. This underlines the danger of using response rate thresholds as a guide to data quality.
1976
1 , This study is a two-part-analysis aimed at determining what differences occur in the incidence Of poverty. When diffexent definitions Of'income-are employed and when ihe time frame of analytis is changed. The first part of the .analysis concentrates , on school-aged children, while the second part studies families. Xhe study-is based on data from the Survey Research Centerli Panel Study of Income pYnamics, tor the years 1968-1972.,For the,analyses school-aged children, all children'In the panel between the ages of 5 and 18 in the, spring of 1972 were counted, resulting in s sample of .1,834 aildren. For the family analysis those families in 1972 which included a male head from 1968, a female head from 1968, or the wife' of a male head in 1968 were counted, resulting in a sample size of 4,010 families. In general, adjustmeati to annual faiily money-income to account for ceriain costs of reteiving income, and for certain non-money components of.income, result in little'change in the rank ordering in economic status of familes, and children. The results also . show.that the Ancidence of poveety is\ligher for children than far families,.by all meisUres: Although adjustmekts to annual money income can change a famill's economic posifion, especially in imoving it'out cd the poverty classification, the time period covered appears tO be a more in;lu tial factor. A crude assessment of the effects of changing the measur4 Of poverty cn the relative number counted as poor in different s b-nationalgareis-was attempted. The reSults'are inconclusive. Vario s policy implications stemming from the results 'of this stut14-are gi en. (Author/AM) Documents acquired by ERIC include many informal unpublished materials not available-from other sources, ERIC makes every .
Using logistic and multilevel logistic modelling we examine non-response at the school and pupil level to the important educational achievement survey Programme for International Student Assessment (PISA) for England. The analysis exploits unusually rich auxiliary information on all schools and pupils sampled for PISA whether responding or not, including data from two large-scale administrative sources on pupils' results in national public exams, which correlate highly with the PISA target variable. Results show that characteristics associated with non-response differ between the school and pupil levels. The findings have important implications for the survey design of education data.
Income and poverty: the rubber band theory
2008
The Economic and Social Research Council welcomes this briefing. It is timed to coincide with the launch of the new innovative panel survey, Understanding Society. The new survey will follow nearly 100,000 individuals in 40,000 households. With funding from the Large Facilities Capital Fund, ESRC's own resources, and a number of government departments, Understanding Society represents the largest ever single investment in academic social research resources in this country. The need for this new survey was identified in a recent review of longitudinal data sources, undertaken as part of the ESRC's National Data Strategy. It concluded that a new, much larger and innovative panel survey should be commissioned. This exciting project will facilitate an in-depth understanding of the lives and diversity of experiences of UK citizens, both over time and across generations. The findings from the study will help to inform and evaluate long-term policy decisions in areas as diverse as housing, health, and education. The UK has long recognised the benefits of longitudinal studies. The British Household Panel Survey (BHPS), and the birth cohort surveys of 1958, 1970 and 2000, are often referred to as the jewels in the crown of the ESRC. These studies have transformed our understanding of the complex trends affecting UK society, and have informed long-term policy making in many areas of government. This briefing provides a series of illustrations of the value of panel surveys, using BHPS findings to demonstrate how longitudinal analysis contributes to the evidence about social processes. I am grateful to the distinguished group of researchers who have combined here to endorse the ESRC's product. The BHPS sample will be incorporated into Understanding Society, and its long run of data will continue to be analysed. Not only will the new study be the largest of its kind ever undertaken across the world, it will also push the methodological boundaries of quantitative data-sets. It confirms the pre-eminence of UK social science in the international research community.
Estimating Income Poverty in the Presence of Missing Data and Measurement Error
Journal of Business & Economic Statistics, 2011
This series presents research findings based either directly on data from the German Socio-Economic Panel Study (SOEP) or using SOEP data as part of an internationally comparable data set (e.g. CNEF, ECHP, LIS, LWS, CHER/PACO). SOEP is a truly multidisciplinary household panel study covering a wide range of social and behavioral sciences: economics, sociology, psychology, survey methodology, econometrics and applied statistics, educational science, political science, public health, behavioral genetics, demography, geography, and sport science.
Evaluating socio-economic status (SES) bias in survey nonresponse
JOURNAL …, 2002
Economic characteristics of households are an important component of their socioeconomic environment. However, they are often given short shrift in surveys because survey designers believe that other measures of socioeconomic status (e.g., education) render them redundant, respondents will not answer such questions truthfully, and they may lead some potential respondents to drop out of the survey altogether. We argue that the conventional wisdom regarding survey questions on economic characteristics is wrong. We explain the conceptual distinctions among various economic measures, summarize survey methods that maximize data quality and present recommendations of specific sets of questions that gather the needed data.
Poverty analysis with missing data: alternative estimators compared
Empirical Economics, 2009
Institute for Social and Economic Research The Institute for Social and Economic Research (ISER) specialises in the production and analysis of longitudinal data. ISER incorporates the following centres: • ESRC Research Centre on Micro-social Change. Established in 1989 to identify, explain, model and forecast social change in Britain at the individual and household level, the Centre specialises in research using longitudinal data. • ESRC UK Longitudinal Centre. This national resource centre was established in October 1999 to promote the use of longitudinal data and to develop a strategy for the future of large-scale longitudinal surveys. It was responsible for the British Household Panel Survey (BHPS) and for the ESRC's interest in the National Child Development Study and the 1970 British Cohort Study • European Centre for Analysis in the Social Sciences. ECASS is an interdisciplinary research centre which hosts major research programmes and helps researchers from the EU gain access to longitudinal data and cross-national datasets from all over Europe. The British Household Panel Survey is one of the main instruments for measuring social change in Britain. The BHPS comprises a nationally representative sample of around 5,500 households and over 10,000 individuals who are reinterviewed each year. The questionnaire includes a constant core of items accompanied by a variable component in order to provide for the collection of initial conditions data and to allow for the subsequent inclusion of emerging research and policy concerns. Among the main projects in ISER's research programme are: the labour market and the division of domestic responsibilities; changes in families and households; modelling households' labour force behaviour; wealth, well-being and socioeconomic structure; resource distribution in the household; and modelling techniques and survey methodology. BHPS data provide the academic community, policymakers and private sector with a unique national resource and allow for comparative research with similar studies in Europe, the United States and Canada. BHPS data are available from the Data Archive at the University of Essex http://www.data-archive.ac.uk Further information about the BHPS and other longitudinal surveys can be obtained by telephoning +44 (0) 1206 873543.