Resetting targets: Examining large effect sizes and disappointing benchmark progress

Abstract

This paper uses recent evidence from international early grade reading programs to provide guidance about how best to create appropriate targets and more effectively identify improved program outcomes. Recent results show that World Bank and US Agency for International Development–funded large-scale international education interventions in low- and middle–income countries tend to produce larger impacts than do interventions in the United States, as measured by effect sizes. However, these effect sizes rarely translate into large gains in mean oral reading fluency scores and are associated with only small increases in the proportion of students meeting country-level reading benchmarks. The limited impact of these low- and middle–income countries’ reading programs on the proportion of students meeting reading benchmarks is in large part caused by right-skewed distributions of student reading scores. In other words, modest impacts on the proportion of students meeting benchmarks are caused by low mean scores and large proportions of nonreaders at baseline. It is essential to take these factors into consideration when setting program targets for reading fluency and comprehension. We recommend that program designers in lower-performing countries use baseline assessment data to develop benchmarks based on multiple performance categories that allow for more ambitious targets focused on reducing nonreaders and increasing beginning readers, with more modest targets aimed at improving oral reading fluency scores and increasing the percentage of proficient readers.