Evaluation Engine

Using existing student data to evaluate the effectiveness of education interventions

Schools frequently try new programs, efforts, or interventions to improve educational outcomes. These interventions might include, for example, smaller class sizes, new curricula, teacher training, remediation programs, or educational technology.

Measuring whether those interventions provide the desired results can be a challenge. Though substantial student and school data exist, experimental designs and impact evaluations take time, require advanced statistical experience, and can be extremely costly to implement. Additionally, evaluators need to navigate the Family Educational Rights and Privacy Act (FERPA), which protects the privacy of student data.

User-Friendly Tool Enables Rigorous Evaluation of Program Outcomes

RTI’s Evaluation Engine is a convenient, secure, and affordable tool to help schools use existing data to evaluate the quality and efficacy of their educational interventions.

The Evaluation Engine—developed in partnership with The Bill & Melinda Gates Foundation and statisticians from the University of Michigan—makes use of Statewide Longitudinal Data Systems (SLDS) to assess the impact of educational interventions.

The tool enables state research and accountability officers, district researchers, partners in research, or other approved stakeholders to easily conduct rigorous evaluations without having an advanced degree in statistics or large research budgets that span several years.

Creating Valid Comparisons While Protecting the Privacy of Student Data

Users identify students participating in a program of interest, and the Evaluation Engine combs the data to create an appropriate comparison group using propensity score matching. This technique identifies groups whose observable characteristics—such as student demographics, grade levels, and prior test results—closely match those of program participants.

Using the comparison group, Evaluation Engine conducts a thorough analysis of the outcomes and compiles web and PDF reports that explain the findings, as well as technical and methodological information. The tool analyzes outcomes to see if the two groups performed differently, providing evidence to help educators and policy makers assess the impact of their efforts.

Because the Evaluation Engine separates student identities from the data, student information is never exposed, meaning the Evaluation Engine makes it easy to comply with FERPA requirements.

Lowering the Burden of Analyzing Educational Outcomes and Impact

The Evaluation Engine helps educators and education policymakers save money, time, and energy by dramatically reducing the effort needed to respond to requests from funders and researchers, and quickly and clearly showing whether educational interventions are realizing their promise to improve outcomes for students.