Redesign Recommendations for the National Crime Victimization Survey

Methodological research to lower the cost of a large-scale household survey while maintaining data quality

Accurate data on crime is vital to a wide range of public-policy decisions in the United States, but it is difficult to collect. Because not all victims report crimes to police, law enforcement data sources alone don’t tell the entire story.

Since 1972, the federal Bureau of Justice Statistics (BJS) has conducted an in-depth study on crimes affecting individuals and households. The National Crime Victimization Survey (NCVS) involves approximately 160,000 people per year and is the preeminent survey on crime in the U.S. Interviewers visit some 90,000 households, surveying every resident 12 and older about crimes affecting individuals as well as the household. Selected households remain in the sample for 3 years and eligible persons in these households are interviewed every 6 months, for a total of 7 interviews. The result is a valuable trove of information that enables BJS to estimate the likelihood of rape, sexual assault, robbery, aggravated and simple assault, theft, household burglary, and motor vehicle theft for the population as a whole as well as for different demographic segments.

Collecting such in-depth data, however, is costly. In 2008, an expert panel study by the National Research Council of the National Academy of Sciences recommended BJS explore redesign options for the NCVS—including survey methods that would reduce costs while minimizing the impact on standard errors. BJS hired RTI and other organizations to evaluate potential study design changes to see if they could save time and money without sacrificing data quality.

Survey Technologies and Incentives Could Reduce Cost while Raising Response Rates

Our work for BJS involved field testing traditionally lower-cost, self-administered survey modes as complements to the interviewer-based data collection methods in the NCVS. We experimented with using Web-based surveys and inbound computer-assisted telephone interviews, or inbound CATI, to assess their potential to increase survey participation while maintaining affordable costs and quality. These modes can eliminate or reduce interviewer labor and travel costs.

We also tested the impact of a nominal incentive on survey participation when self-administered modes are used. We found that both Web and inbound CATI modes have the potential to yield significant cost savings in the out waves of the survey—that is, after rapport had been established with the household during the initial survey wave. In other words, a significant percentage of Wave 1 survey participants were willing to complete their follow-up survey via the Web or inbound CATI six months later.

We also tested the effectiveness of a $10 promised incentive—something NCVS had never previously used—when offered in conjunction with the Web and inbound CATI modes. In Wave 1 of our field test, we found that the cost per completed interview was lower for households offered the incentive. In other words, the incentives were paid for by savings in interviewer labor. We also found the offer of an incentive produced higher response rates among households contacted initially by telephone.

Recommending Best Options and Questions for Further Study

Our research led us to make several recommendations to BJS in 2014 regarding the NCVS study design. We recommended BJS consider using both Web and inbound CATI modes to reduce data collection costs in subsequent waves of the study, once initial rapport with a household has been established. However, we noted that further investigation was needed to assess the feasibility of Web for youth participants and in late waves of the survey.

Given the positive impact of incentives on response rates and survey costs during the field test, we also recommended BJS consider further research to determine the optimal incentive amount for survey participants. BJS is considering RTI’s findings, along with those of other organizations involved in the NCVS redesign work, to determine how best to maintain the quality and viability of this critically important survey in the years ahead.