Determining the optimal number of interview waves in a panel survey with application to the National Crime Victimization Survey

Abstract

Panel surveys need to balance the benefits of repeated measurements (e.g., bounded interview, reduced cost, increased response rates) with the drawbacks that may eventually occur (e.g., respondent fatigue, mode effect). The optimal number of interview waves for a panel survey needs to maximize the advantages while minimizing the potential for bias due to incorporating sampling units for too many interview waves. In this paper, we develop cost models for two potential constraints: (1) keeping the number of interviews constant across designs, and (2) keeping the cost constant across designs. These models are applied to the National Crime Victimization Survey (NCVS). The NCVS currently uses a seven-wave or time-in-sample (TIS) design. In an effort to maintain or reduce costs and improve data quality, the Bureau of Justice Statistics commissioned a Panel Design Study to evaluate the effects of changing the NCVS from a 7-TIS design to a 5-TIS, 4-TIS, 3-TIS, or 1-TIS design. The study used a set of simulations to mimic different panel designs. The simulation assumptions were constructed using NCVS data from 1999 to 2011, and included assumptions about sample sizes, costs, response rates, household replacement, type of interview, demographics, and victimization propensities. Samples were simulated with different panel designs and summary victimization propensities, and standard errors were computed for key estimates. Simulations considered both keeping the cost constant and keeping the number of interviews constant across the different panel design options. In this paper, we show the impact of changing the number of panel TISs on property and violent victimization rates in terms of point estimates, variability, sample sizes, and costs, by several population characteristics. Simulation results found that a 4-TIS design is optimal for the NCVS.