The Costs and Benefits of Improving Response Rates in the CAHPS Medicare Fee for Service Survey

Abstract

Survey researchers agree that nonresponse can bias survey data because nonrespondents may differ in some systematic way from respondents. For this reason, many researchers have gone to extraordinary expense to achieve what many would agree are marginal increases in response at best. Recent research has begun to call into question the "higher response rates at any cost" approach by asking whether it might not be a better use of research dollars to shift resources into more effective and efficient ways to maximize the quality of the data (Keeter, Miller, Kohut, Groves and Presser (2000); Teitler, Reichman and Sprachman (2003). Using data collected for the annual CAHPS Medicare Fee for Service Survey, this paper expands on this earlier work by examining the effect of the incremental response gained during the nonresponse follow-up effort on the CAHPS satisfaction measures, self-reported general health and self- reported mental health questions. The CAHPS Medicare Fee for Service survey is an annual survey of 178,000 beneficiaries of the original Medicare plan. The survey data are collected using the standard mail methodology including a lead letter, two waves of mailing and reminder postcards all sent at the appropriate intervals. The study design also includes a telephone follow-up with a third wave overnight mailing to sample members without known telephone numbers. Discussion will focus on an analysis of the relative costs and benefits of the resources allocated to the telephone non-response follow-up.