When randomization meets reality: An impact evaluation in the Republic of Georgia

Abstract

The plastic bin was shaken seven times while the rattling of the plastic capsules inside it reverberated throughout the room. Hushed anticipation was palpable as a single capsule was drawn, the name inside it read aloud. The winner—an older Georgian farmer—rushed to the front of the room as the audience burst into applause. He spread his arms wide, and, in broken but determined English, shouted, “Thank you, America!”

The prize? A grant to expand his poultry operation. His effusive show of gratitude toward the United States was due to the grant being part of an agribusiness development project—known as ADA— funded by the Millennium Challenge Corporation (MCC), an American foreign aid agency that focuses on reducing poverty through economic growth.

The demand for ADA funds is far beyond the 250 grants (over four years) that the program will be able to issue. Recognizing a rigorous program evaluation opportunity as a result of this over-subscription, MCC decided to sponsor an experimental impact evaluation, conducted by NORC, to track the performance of program participants (the “treatment group”) against a statistically similar group of farmers (the “control group”). The treatment and control groups were selected through a randomization process.