Comparison group selection in the presence of rolling entry for health services research

Abstract

OBJECTIVE: To demonstrate rolling entry matching (REM), a new statistical method, for comparison group selection in the context of staggered nonuniform participant entry in nonrandomized interventions.

STUDY SETTING: Four Health Care Innovation Award (HCIA) interventions between 2012 and 2016.

STUDY DESIGN: Center for Medicare and Medicaid Innovation HCIA participants entering these interventions over time were matched with nonparticipants who exhibited a similar pattern of health care use and expenditures during each participant's baseline period.

DATA EXTRACTION METHODS: Medicare fee-for-service claims data were used to identify nonparticipating, fee-for-service beneficiaries as a potential comparison group and conduct REM.

PRINCIPAL FINDINGS: Rolling entry matching achieved conventionally-accepted levels of balance on observed characteristics between participants and nonparticipants. The method overcame difficulties associated with a small number of intervention entrants.

CONCLUSIONS: In nonrandomized interventions, valid inference regarding intervention effects relies on the suitability of the comparison group to act as the counterfactual case for the intervention group. When participants enter over time, comparison group selection is complicated. Rolling entry matching is a possible solution for comparison group selection in rolling entry interventions that is particularly useful with small sample sizes and merits further investigation in a variety of contexts.