Benjamin Piper | Research Triangle Institute International (original) (raw)

Papers by Benjamin Piper

Many successful piloted programs fail when scaled up to a national level. In Kenya, which has a l... more Many successful piloted programs fail when scaled up to a national level. In Kenya, which has a long history of particularly ineffective implementation after successful pilot programs, the Tusome national literacy program—which receives funding from the United States Agency for International Development—is a national-level scale-up of previous literacy and numeracy programs. We applied a scaling framework (Crouch & DeStefano; 2017) to examine whether Tusome's implementation was rolled out in ways that would enable government structures and officers to respond effectively to the new program. We found that Tusome was able to clarify expectations for implementation and outcomes nationally using benchmarks for Kiswahili and English learning outcomes, and that these expectations were communicated all the way down to the school level. We noted that the essential program inputs were provided fairly consistently, across the nation. In addition, our analyses showed that Kenya developed functional, if simple, accountability and feedback mechanisms to track performance against benchmark expectations. We also established that the Tusome feedback data were utilized to encourage greater levels of instructional support within Kenya's county level structures for education quality support. The results indicated that several of the key elements for successful scale-up were therefore put in place. However, we also discovered that Tusome failed to fully exploit the available classroom observational data to better target instructional support. In the context of this scaling framework, the Tusome literacy program’s external evaluation results showed program impacts of 0.6–1.0 standard
deviations on English and Kiswahili learning outcomes. The program implemented a functional classroom observational feedback system through existing government systems, although usage of those systems varied widely across Kenya. Classroom visits, even if still falling short of the desired rate, were far more frequent, were focused on instructional quality, and included basic feedback and advice to teachers. These findings are promising with respect to the ability of countries facing quality problems to implement a coherent instructional reform through government systems at scale.

RTI Press Publication No. OP-0053-1805

This report presents the results of RTI International Education’s study on teachers' guides acros... more This report presents the results of RTI International Education’s study on teachers' guides across 13 countries and 19 projects. Using quantitative and qualitative methods, we examine how teachers’ guides across the projects differ and find substantial variation in the design and structure of the documents. We develop a scripting index so that the scripting levels of the guides can be compared across projects. The impact results of the programs that use teachers’ guides show significant impacts on learning outcomes, associated with approximately an additional half year of learning, showing that structured teachers’ guides contribute to improved learning outcomes. During observations, we find that teachers make a variety of changes in their classroom instruction from how the guides are written, showing that the utilization of structured teachers’ guides do not create robotic teachers unable to use their own professional skills to teach children. Unfortunately, many changes that teachers make reduce the amount of group work and interactivity that was described in the guides, suggesting that programs should encourage teachers to more heavily utilize the instructional routines designed in the guide. The report includes a set of research-based guidelines that material developers can use to develop teachers’ guides that will support effective instructional practices and help improve learning outcomes. The key takeaway from the report is that structured teachers' guides improve learning outcomes, but that overly scripted teachers' guides are somewhat less effective than simplified teachers' guides that give specific guidance to the teacher but are not written word for word for each lesson in the guide.

Limited rigorous evidence is available from sub-Saharan Africa regarding whether children who lea... more Limited rigorous evidence is available from sub-Saharan Africa regarding whether children who learn to read in their mother tongue will have higher learning outcomes in other subjects. A randomised controlled trial of mother-tongue literacy instruction, the Primary Math and Reading (PRIMR) Initiative, was implemented in Kenya from 2013 to 2014. We compared the impacts of the PRIMR mother-tongue treatment group in two languages with those of another group that did not use mother tongue, but utilised the same instructional components. Results showed that assignment to the mother-tongue group had no additional benefits for English or Kiswahili learning outcomes beyond the non-mother-tongue group, and that the mother-tongue group had somewhat lower mathematics outcomes. Classroom observational analysis showed that assignment to the mother tongue group had only small impacts on the usage of mother tongue in other subjects. Advocates for mother tongue programmes must consider such results alongside local implementation resistance in programme design.

Since 2008, the Ministries of Education in Liberia and Kenya have undertaken transitions from sma... more Since 2008, the Ministries of Education in Liberia and Kenya have undertaken transitions from small-scale pilot programs to improve reading outcomes among primary learners to the large-scale implementation of reading interventions. The effects of the pilots on learning outcomes were significant, but questions remained regarding whether such large gains could be sustained at scale. In this article, the authors dissect the Liberian and Kenyan experiences with implementing large-scale reading programs, documenting the critical components and conditions of the program designs that affected the likelihood of successfully transitioning from pilot to scale. They also review the design, deployment, and effectiveness of each pilot program and the scale, design, duration, enabling conditions, and initial effectiveness results of the scaled programs in each country. The implications of these results for the design of both pilot and large-scale reading programs are discussed in light of the experiences of both the Liberian and Kenyan programs.

Reading research suggests that script type and writing systems have a relationship with children’... more Reading research suggests that script type and writing systems have a
relationship with children’s ability to recognise letters, syllables and
words. In Ethiopia, the scripts used for writing language differ by
visual complexity and the psycholinguistic grain size of the script. The
Ge’ez-script languages have alphasyllabic-based writing systems,
while the Latin-script languages have phoneme-based writing
systems. These differences in script and the differences in aspects of
the writing system influence early reading acquisition. We exploited a
large, regionally representative data set assessing a variety of early
reading tasks in six Ethiopian languages to estimate the impact of
script and aspects of writing system differences on early reading
outcomes in the areas of letter identification, word reading, non-word
decoding and story reading. We made comparisons between
language outcomes using Ge’ez and Latin scripts, controlling for
student background and school socio-economic status (SES).
Additional analyses compared across-script and writing system
differences within regions and gender. We found that Ethiopian script
and writing system differences have implications for instructional
methods for letter identification and word decoding.

Research in sub-Saharan Africa investigating the effect of mother tongue (MT) literacy instructio... more Research in sub-Saharan Africa investigating the effect of mother tongue (MT) literacy instruction at medium scale is limited. A randomized controlled trial of MT literacy instruction was implemented in 2013 and 2014 as part of the Primary Math and Reading (PRIMR) Initiative in Kenya. We compare the effect of two treatment groups—the base PRIMR program teaching literacy in English and Kiswahili and the PRIMR-MT program, which taught literacy in English, Kiswahili, and mother tongue—in two different language environments. Implementation of the MT program faced challenges because many educators were not speakers of the languages, some communities resisted mother tongue instruction, and some areas were more language heterogeneous. Effect sizes on MT literacy averaged between 0.3 and 0.6 standard deviations. The base PRIMR program also increased MT learning outcomes in some measures but had smaller effects than the PRIMR-MT program in oral reading fluency and comprehension.

Education policymakers are investing in information and communications technology (ICT) without a... more Education policymakers are investing in information and communications technology (ICT) without a research base on how ICT improves outcomes. There is limited research on the effects of different types of ICT investments on outcomes. The Kenya Primary Math and Reading (PRIMR) study implemented a randomized controlled trial comparing the effects and cost of three interventions – e-readers for students, tablets for teachers, and the base PRIMR program with tablets for instructional supervisors. The results show that the ICT investments do not improve literacy outcomes significantly more than the base non-ICT instructional program. Our findings show that cost considerations should be paramount in selecting ICT investments in the education sector.

To improve learning outcomes, an intervention in Kenya called the Primary Math and Reading (PRIMR... more To improve learning outcomes, an intervention in Kenya called the Primary Math and Reading (PRIMR) Initiative provided pupil learning materials, teachers’ guides and modest teacher professional development in mathematics. This paper presents the causal impact of PRIMR’s mathematics intervention on pupil achievement indices for procedural and conceptual numeracy, using a differences-in-differences analytic strategy. The mathematics intervention produced modest, statistically
significant results: generally similar results for males and females, a larger impact in grade 2 than grade 1, a larger impact in nongovernment schools than public schools, and smaller outcomes in mathematics than for English or Kiswahili. These findings have relevant policy implications in Kenya given an impending national mathematics programme.

Ubuntu is an African philosophy of human kindness; applying it in the Global South would fundamen... more Ubuntu is an African philosophy of human kindness; applying it in the Global South would fundamentally alter the design of the education sector. This essay argues, however, that the field of international educational development is not, in fact, structured to support an education influenced by ubuntu ideals. Specifically, the educational development milieu includes donors, implementers and academicians who do not sufficiently question the power dynamics which underpin education development. This creates a field where the power imbalances between donors and host governments are not interrogated, where development workers place too much faith in their own knowledge rather than that of local education experts, and where development practitioners rarely appreciate the privilege of working in countries which are not their own. An ubuntu education would alter the educational development field in myriad critical ways, a few of which are suggested in this essay. Educational development programmes in universities and intake programmes for implementers and donors should teach officers humility, appreciating existing local talent and expertise. Donor programmes should incentivise reflective practice which formally embeds appreciation for local culture and expertise, thereby supporting structures which help educational development experts to review their metacognitive processes. The field should also dramatically increase the numbers of local, minority and female educational development practitioners and provide more avenues for advancement for those groups. These are activities which are critical to supporting the education development field, but require a fundamental change of attitude by practitioners to ensure the right kind of relationships between the West and the Global South.

International Review of Education, 2015

Children from low-income families are at risk of learning outcome difficulties, particularly in l... more Children from low-income families are at risk of learning outcome difficulties, particularly in literacy. Various studies link poor literacy results with performance later in primary and secondary school, and suggest that poverty, literacy skills and weak instructional methods combine to drastically limit the educational opportunities for many poor children. The Primary Math and Reading (PRIMR) Initiative was designed to support the learning gains of Class 1 and 2 pupils in seven counties across Kenya. PRIMR uses a randomised controlled trial design
to establish the effect of its intervention and employs basic literacy measures to estimate causal effects. This study shows that PRIMR has been effective for children from low-income families and that early literacy interventions can mitigate socio-economic effects. The findings suggest that efforts to improve
literacy outcomes for the poor should begin early in primary school. Strategies for ensuring that instruction is equitable across socio-economic status are advocated.

Kenya is investing in information and communication technology (ICT) to improve children’s learni... more Kenya is investing in information and communication technology (ICT) to improve children’s learning
outcomes. However, the literature on ICT is pessimistic about the ability of ICT alone to improve outcomes,
and few ICT programs have created the instructional change necessary to increase learning. The Primary Math
and Reading (PRIMR) Initiative implemented a randomized controlled trial of three ICT interventions to
enhance learning outcomes: tablets for instructional supervisors, tablets for teachers, and e-readers for
students. All three showed significant impacts in English and Kiswahili above the results of the control group.
The impacts of the three interventions were not statistically significantly different from each other. Based on
the findings, we recommend that Kenyan policy makers embed ICT interventions in a larger instructional
reform, using ICT to support particular instructional improvement challenges. We also suggest that policy
makers incorporate empirically derived cost-effectiveness analysis into investment decisions, to ensure that
ICT provides value for money.

Despite rapid growth in literacy-related programmes and evaluation in sub-Saharan Africa, little ... more Despite rapid growth in literacy-related programmes and evaluation in sub-Saharan Africa, little critical attention has been paid to the relevance of assumptions that underlie existing assessment methods. This study focuses on the issue of timing in the assessment of oral reading fluency, a critical component of successful reading (Chard, Vaughn, & Tyler, 2002; National Institute of Child Health and Human Development (NICHD), 2000; Pikulski & Chard, 2005). Within the context
of the Primary Math and Reading Initiative, a randomized controlled trial of several instructional interventions in Kenya, timed and untimed Early Grade Reading Assessment (EGRA) oral reading fluency and reading comprehension tasks were administered to 4385 students in 95 government and 125 informal schools. Using the data from the EGRA – whose administration has expanded within sub-Saharan Africa recently – we found that students did not perform significantly better on the assessments when they had more time. This pattern largely held when we examined the effects disaggregated over student ability level. This suggests that timed assessments, which are faster to administer and logistically easier, are appropriate for use in Kenya.

Reading research has shown that variable relationships exist between measures of oral reading flu... more Reading research has shown that variable relationships exist between measures of
oral reading fluency and reading comprehension, depending on whether the
language of the text is the reader’s first language or an additional language. This
paper explores this phenomenon, using reading assessment data for 2,000 Kenyan
children in two or three languages: English, Kiswahili and one of two mother
tongues, Dholuo or Gikuyu. The assessment data allowed us to compare reading
and comprehension rates across languages. The data indicated that many children
could read English words more easily than words in Kiswahili or their mother
tongue; nevertheless, their reading comprehension was significantly lower in
English than in Kiswahili, Dholuo or Gikuyu. The paper concludes that
emphasising English reading fluency is an inefficient route to gaining reading
comprehension skills because pupils are actually attaining minimal oral reading
fluency in English and only modest comprehension skills in their own languages.
The evidence also demonstrates that Kenya’s national language policy of mother
tongue as a medium of instruction in the early primary grades is consistently
ignored in practice.

Instructional coaching has improved student outcomes in the United States, and may help to solve ... more Instructional coaching has improved student outcomes in the United States, and may help to solve
Kenya's literacy problems. Coaching is costly, however, and evidence is lacking regarding the most costefficient
teacher-to-coach ratio. We used student literacy outcome data from more than 8000 students
participating in the Kenya Primary Math and Reading Initiativeda randomized controlled trial of
instructional interventions in public and nonformal schoolsdto fill this gap. Coaches in larger public
zones made fewer visits per teacher, and teacher-coach ratio and student performance were negatively
associated. Using causal methods, we concluded that lower ratios might improve nonformal school
outcomes.

International Journal of Educational Development, 2014

In recent years, the Education for All movement has focused more intensely on the quality of educ... more In recent years, the Education for All movement has focused more
intensely on the quality of education, rather than simply provision. Many recent and
current education quality interventions focus on literacy, which is the core skill
required for further academic success. Despite this focus on the quality of literacy
instruction in developing countries, little rigorous research has been conducted on
critical issues of assessment. This analysis, which uses data from the Primary Math
and Reading Initiative (PRIMR) in Kenya, aims to begin filling this gap by addressing
a key assessment issue – should literacy assessments in Kenya be administered
orally or silently? The authors compared second-grade students’ scores
on oral and silent reading tasks of the Early Grade Reading Assessment (EGRA) in
Kiswahili and English, and found no statistically significant differences in either
language. They did, however, find oral reading rates to be more strongly related to
reading comprehension scores. Oral assessment has another benefit for programme
evaluators – it allows for the collection of data on student errors, and therefore the
calculation of words read correctly per minute, as opposed to simply words read per
minute. The authors therefore recommend that, in Kenya and in similar contexts,
student reading fluency be assessed via oral rather than silent assessment.

Language policy is generally seen as a national-level decision regarding which languages the stat... more Language policy is generally seen as a national-level decision regarding which
languages the state will support, and in which public domains. However, the reality is
that language policy plays out at regional and local levels as well. In fact, it could be
argued that the most important instantiations of language policy are those which
directly determine local-language behaviors in institutions such as schools,
government and civil society. Using data drawn from Kenya, this article examines the
formulation and implementation of language policy as it plays out in the primary
classroom environment. The relationships between language policy implementation at
the classroom level and students’ early literacy outcomes are explored, giving insight
into how the degree of adherence to language policy in the classroom intersects with
student achievement. The article presents findings using language use as a predictor,
school and student-level economic status as control variables and student
achievement as the outcomes. The country-level differences in language policy
implementation between Kenya and Uganda, and the impact of those differences on
student achievement in the two countries, are also examined. The article has
implications for the establishment of a learning environment in the multi-language
primary classroom, and demonstrates the extent to which choices about language
policy implementation can present a serious challenge to effective education.

Research in Comparative and International Education, Sep 2013

The rise and widespread adoption of the early grade reading assessment (EGRA) has produced an amp... more The rise and widespread adoption of the early grade reading assessment (EGRA) has produced an ample
supply of critics and converts. This paper seeks to clarify the purpose of EGRA and its limitations. EGRA
was created to inform education systems and programmes and alone, is not an intervention. Designed to
measure some of the foundational literacy skills that readers need for beginning reading, EGRA is a
collection of subtasks, each with a specific purpose. This paper includes a description and rationale for
each subtask, as well as the conceptual framework that underpins the assessment. Key results from
multiple surveys provide informative, grounded examples of how the assessment results are being used
to inform both classroom practice and system-level policy. We conclude with a brief discussion on the
potential uses of EGRA and similar oral assessments of early learning for informing the monitoring of the
post-2015 education indicators.

Many successful piloted programs fail when scaled up to a national level. In Kenya, which has a l... more Many successful piloted programs fail when scaled up to a national level. In Kenya, which has a long history of particularly ineffective implementation after successful pilot programs, the Tusome national literacy program—which receives funding from the United States Agency for International Development—is a national-level scale-up of previous literacy and numeracy programs. We applied a scaling framework (Crouch & DeStefano; 2017) to examine whether Tusome's implementation was rolled out in ways that would enable government structures and officers to respond effectively to the new program. We found that Tusome was able to clarify expectations for implementation and outcomes nationally using benchmarks for Kiswahili and English learning outcomes, and that these expectations were communicated all the way down to the school level. We noted that the essential program inputs were provided fairly consistently, across the nation. In addition, our analyses showed that Kenya developed functional, if simple, accountability and feedback mechanisms to track performance against benchmark expectations. We also established that the Tusome feedback data were utilized to encourage greater levels of instructional support within Kenya's county level structures for education quality support. The results indicated that several of the key elements for successful scale-up were therefore put in place. However, we also discovered that Tusome failed to fully exploit the available classroom observational data to better target instructional support. In the context of this scaling framework, the Tusome literacy program’s external evaluation results showed program impacts of 0.6–1.0 standard
deviations on English and Kiswahili learning outcomes. The program implemented a functional classroom observational feedback system through existing government systems, although usage of those systems varied widely across Kenya. Classroom visits, even if still falling short of the desired rate, were far more frequent, were focused on instructional quality, and included basic feedback and advice to teachers. These findings are promising with respect to the ability of countries facing quality problems to implement a coherent instructional reform through government systems at scale.

RTI Press Publication No. OP-0053-1805

This report presents the results of RTI International Education’s study on teachers' guides acros... more This report presents the results of RTI International Education’s study on teachers' guides across 13 countries and 19 projects. Using quantitative and qualitative methods, we examine how teachers’ guides across the projects differ and find substantial variation in the design and structure of the documents. We develop a scripting index so that the scripting levels of the guides can be compared across projects. The impact results of the programs that use teachers’ guides show significant impacts on learning outcomes, associated with approximately an additional half year of learning, showing that structured teachers’ guides contribute to improved learning outcomes. During observations, we find that teachers make a variety of changes in their classroom instruction from how the guides are written, showing that the utilization of structured teachers’ guides do not create robotic teachers unable to use their own professional skills to teach children. Unfortunately, many changes that teachers make reduce the amount of group work and interactivity that was described in the guides, suggesting that programs should encourage teachers to more heavily utilize the instructional routines designed in the guide. The report includes a set of research-based guidelines that material developers can use to develop teachers’ guides that will support effective instructional practices and help improve learning outcomes. The key takeaway from the report is that structured teachers' guides improve learning outcomes, but that overly scripted teachers' guides are somewhat less effective than simplified teachers' guides that give specific guidance to the teacher but are not written word for word for each lesson in the guide.

Limited rigorous evidence is available from sub-Saharan Africa regarding whether children who lea... more Limited rigorous evidence is available from sub-Saharan Africa regarding whether children who learn to read in their mother tongue will have higher learning outcomes in other subjects. A randomised controlled trial of mother-tongue literacy instruction, the Primary Math and Reading (PRIMR) Initiative, was implemented in Kenya from 2013 to 2014. We compared the impacts of the PRIMR mother-tongue treatment group in two languages with those of another group that did not use mother tongue, but utilised the same instructional components. Results showed that assignment to the mother-tongue group had no additional benefits for English or Kiswahili learning outcomes beyond the non-mother-tongue group, and that the mother-tongue group had somewhat lower mathematics outcomes. Classroom observational analysis showed that assignment to the mother tongue group had only small impacts on the usage of mother tongue in other subjects. Advocates for mother tongue programmes must consider such results alongside local implementation resistance in programme design.

Since 2008, the Ministries of Education in Liberia and Kenya have undertaken transitions from sma... more Since 2008, the Ministries of Education in Liberia and Kenya have undertaken transitions from small-scale pilot programs to improve reading outcomes among primary learners to the large-scale implementation of reading interventions. The effects of the pilots on learning outcomes were significant, but questions remained regarding whether such large gains could be sustained at scale. In this article, the authors dissect the Liberian and Kenyan experiences with implementing large-scale reading programs, documenting the critical components and conditions of the program designs that affected the likelihood of successfully transitioning from pilot to scale. They also review the design, deployment, and effectiveness of each pilot program and the scale, design, duration, enabling conditions, and initial effectiveness results of the scaled programs in each country. The implications of these results for the design of both pilot and large-scale reading programs are discussed in light of the experiences of both the Liberian and Kenyan programs.

Reading research suggests that script type and writing systems have a relationship with children’... more Reading research suggests that script type and writing systems have a
relationship with children’s ability to recognise letters, syllables and
words. In Ethiopia, the scripts used for writing language differ by
visual complexity and the psycholinguistic grain size of the script. The
Ge’ez-script languages have alphasyllabic-based writing systems,
while the Latin-script languages have phoneme-based writing
systems. These differences in script and the differences in aspects of
the writing system influence early reading acquisition. We exploited a
large, regionally representative data set assessing a variety of early
reading tasks in six Ethiopian languages to estimate the impact of
script and aspects of writing system differences on early reading
outcomes in the areas of letter identification, word reading, non-word
decoding and story reading. We made comparisons between
language outcomes using Ge’ez and Latin scripts, controlling for
student background and school socio-economic status (SES).
Additional analyses compared across-script and writing system
differences within regions and gender. We found that Ethiopian script
and writing system differences have implications for instructional
methods for letter identification and word decoding.

Research in sub-Saharan Africa investigating the effect of mother tongue (MT) literacy instructio... more Research in sub-Saharan Africa investigating the effect of mother tongue (MT) literacy instruction at medium scale is limited. A randomized controlled trial of MT literacy instruction was implemented in 2013 and 2014 as part of the Primary Math and Reading (PRIMR) Initiative in Kenya. We compare the effect of two treatment groups—the base PRIMR program teaching literacy in English and Kiswahili and the PRIMR-MT program, which taught literacy in English, Kiswahili, and mother tongue—in two different language environments. Implementation of the MT program faced challenges because many educators were not speakers of the languages, some communities resisted mother tongue instruction, and some areas were more language heterogeneous. Effect sizes on MT literacy averaged between 0.3 and 0.6 standard deviations. The base PRIMR program also increased MT learning outcomes in some measures but had smaller effects than the PRIMR-MT program in oral reading fluency and comprehension.

Education policymakers are investing in information and communications technology (ICT) without a... more Education policymakers are investing in information and communications technology (ICT) without a research base on how ICT improves outcomes. There is limited research on the effects of different types of ICT investments on outcomes. The Kenya Primary Math and Reading (PRIMR) study implemented a randomized controlled trial comparing the effects and cost of three interventions – e-readers for students, tablets for teachers, and the base PRIMR program with tablets for instructional supervisors. The results show that the ICT investments do not improve literacy outcomes significantly more than the base non-ICT instructional program. Our findings show that cost considerations should be paramount in selecting ICT investments in the education sector.

To improve learning outcomes, an intervention in Kenya called the Primary Math and Reading (PRIMR... more To improve learning outcomes, an intervention in Kenya called the Primary Math and Reading (PRIMR) Initiative provided pupil learning materials, teachers’ guides and modest teacher professional development in mathematics. This paper presents the causal impact of PRIMR’s mathematics intervention on pupil achievement indices for procedural and conceptual numeracy, using a differences-in-differences analytic strategy. The mathematics intervention produced modest, statistically
significant results: generally similar results for males and females, a larger impact in grade 2 than grade 1, a larger impact in nongovernment schools than public schools, and smaller outcomes in mathematics than for English or Kiswahili. These findings have relevant policy implications in Kenya given an impending national mathematics programme.

Ubuntu is an African philosophy of human kindness; applying it in the Global South would fundamen... more Ubuntu is an African philosophy of human kindness; applying it in the Global South would fundamentally alter the design of the education sector. This essay argues, however, that the field of international educational development is not, in fact, structured to support an education influenced by ubuntu ideals. Specifically, the educational development milieu includes donors, implementers and academicians who do not sufficiently question the power dynamics which underpin education development. This creates a field where the power imbalances between donors and host governments are not interrogated, where development workers place too much faith in their own knowledge rather than that of local education experts, and where development practitioners rarely appreciate the privilege of working in countries which are not their own. An ubuntu education would alter the educational development field in myriad critical ways, a few of which are suggested in this essay. Educational development programmes in universities and intake programmes for implementers and donors should teach officers humility, appreciating existing local talent and expertise. Donor programmes should incentivise reflective practice which formally embeds appreciation for local culture and expertise, thereby supporting structures which help educational development experts to review their metacognitive processes. The field should also dramatically increase the numbers of local, minority and female educational development practitioners and provide more avenues for advancement for those groups. These are activities which are critical to supporting the education development field, but require a fundamental change of attitude by practitioners to ensure the right kind of relationships between the West and the Global South.

International Review of Education, 2015

Children from low-income families are at risk of learning outcome difficulties, particularly in l... more Children from low-income families are at risk of learning outcome difficulties, particularly in literacy. Various studies link poor literacy results with performance later in primary and secondary school, and suggest that poverty, literacy skills and weak instructional methods combine to drastically limit the educational opportunities for many poor children. The Primary Math and Reading (PRIMR) Initiative was designed to support the learning gains of Class 1 and 2 pupils in seven counties across Kenya. PRIMR uses a randomised controlled trial design
to establish the effect of its intervention and employs basic literacy measures to estimate causal effects. This study shows that PRIMR has been effective for children from low-income families and that early literacy interventions can mitigate socio-economic effects. The findings suggest that efforts to improve
literacy outcomes for the poor should begin early in primary school. Strategies for ensuring that instruction is equitable across socio-economic status are advocated.

Kenya is investing in information and communication technology (ICT) to improve children’s learni... more Kenya is investing in information and communication technology (ICT) to improve children’s learning
outcomes. However, the literature on ICT is pessimistic about the ability of ICT alone to improve outcomes,
and few ICT programs have created the instructional change necessary to increase learning. The Primary Math
and Reading (PRIMR) Initiative implemented a randomized controlled trial of three ICT interventions to
enhance learning outcomes: tablets for instructional supervisors, tablets for teachers, and e-readers for
students. All three showed significant impacts in English and Kiswahili above the results of the control group.
The impacts of the three interventions were not statistically significantly different from each other. Based on
the findings, we recommend that Kenyan policy makers embed ICT interventions in a larger instructional
reform, using ICT to support particular instructional improvement challenges. We also suggest that policy
makers incorporate empirically derived cost-effectiveness analysis into investment decisions, to ensure that
ICT provides value for money.

Despite rapid growth in literacy-related programmes and evaluation in sub-Saharan Africa, little ... more Despite rapid growth in literacy-related programmes and evaluation in sub-Saharan Africa, little critical attention has been paid to the relevance of assumptions that underlie existing assessment methods. This study focuses on the issue of timing in the assessment of oral reading fluency, a critical component of successful reading (Chard, Vaughn, & Tyler, 2002; National Institute of Child Health and Human Development (NICHD), 2000; Pikulski & Chard, 2005). Within the context
of the Primary Math and Reading Initiative, a randomized controlled trial of several instructional interventions in Kenya, timed and untimed Early Grade Reading Assessment (EGRA) oral reading fluency and reading comprehension tasks were administered to 4385 students in 95 government and 125 informal schools. Using the data from the EGRA – whose administration has expanded within sub-Saharan Africa recently – we found that students did not perform significantly better on the assessments when they had more time. This pattern largely held when we examined the effects disaggregated over student ability level. This suggests that timed assessments, which are faster to administer and logistically easier, are appropriate for use in Kenya.

Reading research has shown that variable relationships exist between measures of oral reading flu... more Reading research has shown that variable relationships exist between measures of
oral reading fluency and reading comprehension, depending on whether the
language of the text is the reader’s first language or an additional language. This
paper explores this phenomenon, using reading assessment data for 2,000 Kenyan
children in two or three languages: English, Kiswahili and one of two mother
tongues, Dholuo or Gikuyu. The assessment data allowed us to compare reading
and comprehension rates across languages. The data indicated that many children
could read English words more easily than words in Kiswahili or their mother
tongue; nevertheless, their reading comprehension was significantly lower in
English than in Kiswahili, Dholuo or Gikuyu. The paper concludes that
emphasising English reading fluency is an inefficient route to gaining reading
comprehension skills because pupils are actually attaining minimal oral reading
fluency in English and only modest comprehension skills in their own languages.
The evidence also demonstrates that Kenya’s national language policy of mother
tongue as a medium of instruction in the early primary grades is consistently
ignored in practice.

Instructional coaching has improved student outcomes in the United States, and may help to solve ... more Instructional coaching has improved student outcomes in the United States, and may help to solve
Kenya's literacy problems. Coaching is costly, however, and evidence is lacking regarding the most costefficient
teacher-to-coach ratio. We used student literacy outcome data from more than 8000 students
participating in the Kenya Primary Math and Reading Initiativeda randomized controlled trial of
instructional interventions in public and nonformal schoolsdto fill this gap. Coaches in larger public
zones made fewer visits per teacher, and teacher-coach ratio and student performance were negatively
associated. Using causal methods, we concluded that lower ratios might improve nonformal school
outcomes.

International Journal of Educational Development, 2014

In recent years, the Education for All movement has focused more intensely on the quality of educ... more In recent years, the Education for All movement has focused more
intensely on the quality of education, rather than simply provision. Many recent and
current education quality interventions focus on literacy, which is the core skill
required for further academic success. Despite this focus on the quality of literacy
instruction in developing countries, little rigorous research has been conducted on
critical issues of assessment. This analysis, which uses data from the Primary Math
and Reading Initiative (PRIMR) in Kenya, aims to begin filling this gap by addressing
a key assessment issue – should literacy assessments in Kenya be administered
orally or silently? The authors compared second-grade students’ scores
on oral and silent reading tasks of the Early Grade Reading Assessment (EGRA) in
Kiswahili and English, and found no statistically significant differences in either
language. They did, however, find oral reading rates to be more strongly related to
reading comprehension scores. Oral assessment has another benefit for programme
evaluators – it allows for the collection of data on student errors, and therefore the
calculation of words read correctly per minute, as opposed to simply words read per
minute. The authors therefore recommend that, in Kenya and in similar contexts,
student reading fluency be assessed via oral rather than silent assessment.

Language policy is generally seen as a national-level decision regarding which languages the stat... more Language policy is generally seen as a national-level decision regarding which
languages the state will support, and in which public domains. However, the reality is
that language policy plays out at regional and local levels as well. In fact, it could be
argued that the most important instantiations of language policy are those which
directly determine local-language behaviors in institutions such as schools,
government and civil society. Using data drawn from Kenya, this article examines the
formulation and implementation of language policy as it plays out in the primary
classroom environment. The relationships between language policy implementation at
the classroom level and students’ early literacy outcomes are explored, giving insight
into how the degree of adherence to language policy in the classroom intersects with
student achievement. The article presents findings using language use as a predictor,
school and student-level economic status as control variables and student
achievement as the outcomes. The country-level differences in language policy
implementation between Kenya and Uganda, and the impact of those differences on
student achievement in the two countries, are also examined. The article has
implications for the establishment of a learning environment in the multi-language
primary classroom, and demonstrates the extent to which choices about language
policy implementation can present a serious challenge to effective education.

Research in Comparative and International Education, Sep 2013

The rise and widespread adoption of the early grade reading assessment (EGRA) has produced an amp... more The rise and widespread adoption of the early grade reading assessment (EGRA) has produced an ample
supply of critics and converts. This paper seeks to clarify the purpose of EGRA and its limitations. EGRA
was created to inform education systems and programmes and alone, is not an intervention. Designed to
measure some of the foundational literacy skills that readers need for beginning reading, EGRA is a
collection of subtasks, each with a specific purpose. This paper includes a description and rationale for
each subtask, as well as the conceptual framework that underpins the assessment. Key results from
multiple surveys provide informative, grounded examples of how the assessment results are being used
to inform both classroom practice and system-level policy. We conclude with a brief discussion on the
potential uses of EGRA and similar oral assessments of early learning for informing the monitoring of the
post-2015 education indicators.