Ruth A Childs | University of Toronto (original) (raw)

Papers by Ruth A Childs

Medical Teacher , 2018

Context: Creating a new testing program requires the development of a test blueprint that will de... more Context: Creating a new testing program requires the development of a test blueprint that will determine how the items on each test form are distributed across possible content areas and practice domains. To achieve validity, categories of a blueprint are typically based on the judgments of content experts. How experts judgments are elicited and combined is important to the quality of resulting test blueprints. Methods: Content experts in dentistry participated in a day-long faculty-wide workshop to discuss, refine, and confirm the categories and their relative weights. After reaching agreement on categories and their definitions, experts judged the relative importance between category pairs, registering their judgments anonymously using iClicker, an audience response system. Judgments were combined in two ways: a simple calculation that could be performed during the workshop and a multidimensional scaling of the judgments performed later. Results: Content experts were able to produce a set of relative weights using this approach. The multidimensional scaling yielded a three-dimensional model with the potential to provide deeper insights into the basis of the experts' judgments. Conclusion: The approach developed and demonstrated in this study can be applied across academic disciplines to elicit and combine content experts judgments for the development of test blueprints.

This case study investigated equity in teacher education admissions. Through document analysis an... more This case study investigated equity in teacher education admissions. Through document analysis and structured interviews with ten past or current members of the admissions committee in a large initial teacher education program in Ontario, we developed an understanding of equity in teacher education admissions as encompassing two foci: equity in admissions—that is, equity of access for applicants to the program—and equity through admissions— that is, equity of educational opportunity and outcomes for the children in the schools where the teachers trained by the programs will eventually teach. Our analysis illustrates the importance of recognizing both foci and the tensions between them.

BMC Public Health, 2014

Background: Few researchers have the data required to adequately understand how the school enviro... more Background: Few researchers have the data required to adequately understand how the school environment impacts youth health behaviour development over time. Methods/Design: COMPASS is a prospective cohort study designed to annually collect hierarchical longitudinal data from a sample of 90 secondary schools and the 50,000+ grade 9 to 12 students attending those schools. COMPASS uses a rigorous quasi-experimental design to evaluate how changes in school programs, policies, and/or built environment (BE) characteristics are related to changes in multiple youth health behaviours and outcomes over time. These data will allow for the quasi-experimental evaluation of natural experiments that will occur within schools over the course of COMPASS, providing a means for generating "practice based evidence" in school-based prevention programming. Discussion: COMPASS is the first study with the infrastructure to robustly evaluate the impact that changes in multiple school-level programs, policies, and BE characteristics within or surrounding a school might have on multiple youth health behaviours or outcomes over time. COMPASS will provide valuable new insight for planning, tailoring and targeting of school-based prevention initiatives where they are most likely to have impact.

This report tests these policy objectives by exploring the nature of educational pathways and the... more This report tests these policy objectives by exploring the nature of educational pathways and the links between educational pathways and the labour market in Ontario, and it compares these outcomes with Canada as a whole. Its purpose is to inform policy and practices at the departmental level within PSE institutions, at the institutional level and at the system level as a whole. It finds that the extent to which students stay within the same field of education when they undertake a second PSE qualification varies quite widely between fields of education, but that overall, pathways within fields of education are quite modest.

This presentation reports on an Ontario government funded project on educational pathways. It exp... more This presentation reports on an Ontario government funded project on educational pathways. It explores whether graduates stay within the same field of study when they undertake a second postsecondary education qualification. It examines educational pathways within fields of study between educational institutions (college to college; college to university; university to college; and university to university) and by qualification level (diploma to degree, degree to diploma, degree to post-graduate qualification etc). It compares the outcomes in Ontario with Canada overall (excluding Quebec). Preliminary findings show that: • The percentage of students who move from college to university is lower in Ontario than it is for Canada; • Within Ontario and Canada, the most common pathway consists of students who undertake a first and second qualification in university; • The extent to which students stay within the same field of study when they undertake a second PSE qualification varies. Ov...

Journal of allied health, 2009

Determining admission criteria that will predict successful student outcomes is a challenging und... more Determining admission criteria that will predict successful student outcomes is a challenging undertaking for newly established health professional programs. This study examined data from the students who entered a medical radiation sciences program in September 2002. By analyzing the correlation between undergraduate GPA, grades in undergraduate science courses, performance in program coursework, and post-graduation certification examination results, the authors determined admission criteria that were linked to successful student outcomes for radiological technology and radiation therapy students.

This study investigated the possible impacts of language and curriculum differences on the perfor... more This study investigated the possible impacts of language and curriculum differences on the performance of test items by subpopulations of students. Focusing on Measurement and Geometry items completed by students in French- and English-language schools in Ontario made it possible to explore the differences and to compare the item response theory (IRT) and Mantel-Haenszel (MH) approaches to finding items with differential item functioning (DIF). The tests in this study might have had DIF as a result of translation or curriculum. Data came from the 2001 administration of the School Achievement Indicators Program (SAIP) mathematics assessment. The dataset contained information on 793 13-year-old and 677 16-year-old students from English-language schools and 487 13-year-old and 546 16-year-old students from French-language schools. Thirteen of the 27 items studies were flagged as exhibiting DIF by the MH approach, and 6 of these 13 were also flagged by IRT. Differences in curricula and ...

Page 1. AD-A256 160 ARI Research Note 92-74 III 1|1I1 Selection and Classification Tests for ... ... more Page 1. AD-A256 160 ARI Research Note 92-74 III 1|1I1 Selection and Classification Tests for ... United States Army Research Institute for the Behavioral and Social Sciences Approvd for pub4bo rmta ; dlfltbulion Ii unlimited. Page 2. US ARMY RESEARCH INSTITUTE ...

Most provinces and states that require students to take large-scale assessments provide informati... more Most provinces and states that require students to take large-scale assessments provide information about the tests for parents; however, parents vary in their uses of this information. In this study, parents in an urban Ontario elementary school were surveyed about where they obtained information about the Ontario assessments, what they knew about the assessments, and what they believed about the

Most teacher education programs receive many more applications than they can accept. How should p... more Most teacher education programs receive many more applications than they can accept. How should programs select among applicants and how should the programs evaluate the success of their selection processes? In this article we review the criteria utilized throughout North America to select prospective teachers into education programs. The strengths and weaknesses of each criterion are discussed. We propose a

An occupational information system for the 21st century: The development of O*NET., 1999

ABSTRACT Because of the central role of occupational ratings in the Occupational Information Netw... more ABSTRACT Because of the central role of occupational ratings in the Occupational Information Network (O*NET), the authors have given careful consideration to identifying potential sources of variance in the O*NET ratings. The authors present a model of these occupational descriptor covariates and describe the steps they have taken to minimize or control the effects of each covariate on the O*NET system. They also discuss the implications of our preliminary analyses of the O*NET data for future research on the accuracy of occupational analysis data. (PsycINFO Database Record (c) 2012 APA, all rights reserved)

Educational Measurement: Issues and Practice, 2011

ABSTRACT Large-scale assessment results for schools, school boards/districts, and entire province... more ABSTRACT Large-scale assessment results for schools, school boards/districts, and entire provinces or states are commonly reported as the percentage of students achieving a standard—-that is, the percentage of students scoring above the cut score that defines the standard on the assessment scale. Recent research has shown that this method of reporting is sensitive to small changes in the cut score, especially when comparing results across years or between groups. This study builds on that work, investigating the effects of reporting group size on the stability of results. In Part 1 of this study, Grade 6 students’ results on Ontario's 2008 and 2009 Junior Assessments of Reading, Writing and Mathematics were compared, by school, for different sizes of schools. In Part 2, samples of students’ results on the 2009 assessment were randomly drawn and compared, for 10 group sizes, to estimate the variability in results due to sampling error. The results showed that the percentage of students above a cut score (PAC) was unstable for small schools and small randomly drawn groups.

Journal of allied health, 2009

Applied Psychological Measurement, 1999

This study investigates what grade-3 teachers say they would do if faced with common test adminis... more This study investigates what grade-3 teachers say they would do if faced with common test administration dilemmas - and why. Grade-3 teachers with experience administering On- tario's provincially mandated assessment were recruited through professional association newsletters and Toronto-area newspapers. They responded to an on-line questionnaire (n = 98) or an hour-long interview (n = 40). Many teachers predicted that they would not follow the test administration instructions. Their rationales included (a) supporting the students to do their best work, (b) ensuring a positive testing experience for the students, and (c) main- taining their pedagogical routines and their relationships with the students. The findings highlight the ethical dilemmas teachers may experience when proctoring tests.

Journal of Educational Administration and History, 2006

Educational Measurement: Issues and Practice, 1991

Academic Medicine, 2003

This study investigates (a) whether items within the Multiple-Choice Questions component of the M... more This study investigates (a) whether items within the Multiple-Choice Questions component of the Medical Council of Canada&amp;#39;s Qualifying Examination Part I exhibit local dependencies and (b) potential sources of such dependencies. The dimensionality of each of six discipline-based subtests was assessed based on exploratory nonlinear factor analyses. A standardized Fisher&amp;#39;s z statistic was used to test residual item correlations for local item dependencies. The characteristics of pairs of items flagged as possibly locally dependent were reviewed. Some items in the Pediatrics and Public Medicine/Community Health subtests are locally dependent; these tend to be the more difficult items on the subtests. While these results are encouraging, the possible causes and potential impacts of any local dependencies should be investigated further.

Medical Teacher , 2018

BMC Public Health, 2014

Journal of allied health, 2009

An occupational information system for the 21st century: The development of O*NET., 1999

Educational Measurement: Issues and Practice, 2011

Journal of allied health, 2009

Applied Psychological Measurement, 1999

Journal of Educational Administration and History, 2006

Educational Measurement: Issues and Practice, 1991

Academic Medicine, 2003

In L. Thomas & M. Hirschkorn (Eds.), Change and progress in Canadian teacher education (pp. 420-440). E-book published by the Canadian Association for Teacher Education., 2015

This chapter explores how changes in, to, and through the process of admitting teacher candidates... more This chapter explores how changes in, to, and through the process of admitting teacher candidates to an initial teacher education (ITE) program relate to wider program changes.
Through a review of materials from a large consecutive ITE program in Ontario, we identified seven problems that the admission process in that program has been called upon to solve. Some of the problems have been seen as the responsibility of the admission process for many years;
other problems have only recently been assigned to the admission process. These problems are related to three distinct relationships of admission process changes with wider program changes: changes in the admission process that happen as a result of other changes in the program; changes to the admission process with no attendant changes to the program; and changes through the admission process in which the admission process is intended to change the program or the teaching force.

In L. Thomas (Ed.), What is Canadian about Teacher Education in Canada? Multiple Perspectives on Canadian Teacher Education in the Twenty-First Century (pp. 172-186) . Canadian Association for Teacher Education/Association canadienne pour la formation à l’enseignement., 2013

This study investigated how applicants’ racial identity is considered when Ontario’s initial teac... more This study investigated how applicants’ racial identity is considered when Ontario’s initial teacher education programs select among applicants. In 2012, ten of the twelve publicly-funded Faculties of Education offering one-year post-Bachelor’s degree programs considered racial identity in admission. Analysis of their application instructions and forms revealed five approaches: (1) designating places in the program for applicants who self-identify as members of racially minoritized groups or communities and meet all admission requirements (three programs); (2) waiving minimum requirements (one, possibly two, programs); (3) inviting descriptions of past educational disadvantage (two programs); (4) inviting descriptions of future benefit (two programs), and (5) monitoring the admission process (one program). The assumptions underlying each approach and their relationships to employment equity goals are discussed.

In L. Thomas (Ed.), Becoming teacher: Sites for development of Canadian teacher education (pp. 174-207). E-book published by the Canadian Association for Teacher Education., 2014

This study explored the beliefs of teacher candidates around diversity and equity, and whether th... more This study explored the beliefs of teacher candidates around diversity and equity, and whether their beliefs changed during an initial teacher education program. Essay questions from the
application profile completed by applicants to an initial teacher education program were re-administered to six teacher candidates toward the end of the program. A qualitative analysis of the responses revealed little difference in the teacher candidates’ beliefs around equity and diversity between the essays written as part of the application process and the essays written after teacher candidates had completed most of the program. From these results, it is difficult to determine whether the teacher candidates’ beliefs remained constant or whether the essay questions were not sufficiently sensitive to change. Implications for future research are discussed.

In T. Falkenberg and H. Smits (Eds.), The question of evidence in research in teacher education in the context of teacher education program review in Canada (2 vols., pp.59-68). Winnipeg, MB: Faculty of Education, University of Manitoba., 2011

This chapter describes a role for quantitative research in promoting equity in initial teacher ed... more This chapter describes a role for quantitative research in promoting equity in initial teacher education. Specifically, we argue that quantitative research can (1) help programs identify patterns – especially, gaps between groups of applicants or teacher candidates, (2) push programs to articulate values and goals, and (3) make the admission process more transparent and reliable. Examples are drawn from a program of research to support equity in admissions for the Ontario Institute for Studies in Education’s Consecutive Initial Teacher Education Program.

Canadian Journal of Educational Administration and Policy, 121, 1-23., 2011

This article argues for the importance of broad and on-going research to support initial or pre-s... more This article argues for the importance of broad and on-going research to support initial or pre-service teacher education program admissions. Examples from a large initial teacher education program at one Canadian university illustrate the contributions of research to the evaluation and refinement of admission processes. These examples include anonymous surveys and confidential interviews of current pre-service teachers about their experiences of answering application questions about their social identity, how they decided to apply to and attend the program, and their expectations of teacher education and teaching. Research studies about the perspectives of and agreement among the application raters are also discussed. Finally, how the operational needs of the admission processes shape the research agenda and the emerging research findings in turn shape the admission processes is explored.

Identifying under-represented groups a challenge for college and university access programs Coll... more Identifying under-represented groups a challenge for college and university access programs

Colleges and universities across Ontario are engaged in many outreach, recruitment and retention efforts to improve access to postsecondary education for under-represented groups. However, two new reports from the Higher Education Quality Council of Ontario (HEQCO) find that defining which students are targeted for these programs and which groups institutions are attempting to serve are significant challenges to improving access. The studies found that the emphasis on student self-identification and tension between local demographic needs and broader ministry guidelines made providing equitable access and accurate program evaluation difficult.