Census Geocoding for Nonresponse Bias Evaluation in Telephone Surveys An Assessment of the Error Properties

Abstract

The threat of nonresponse bias has been increasing with the precipitous decline of survey response rates, particularly in random-digit-dialed (RDD) telephone surveys. Often, researchers have only geographic information for RDD nonresponse cases. To compensate for nonresponse in landline samples, census demographic information can be appended at varying levels of geographic aggregation for both respondents and nonrespondents. The effectiveness of this approach depends on the error properties of the census geocoding (CG) process; however, to date, this process has never been thoroughly evaluated. In extreme situations, errors in the CG process can do more harm than good for survey estimates. If components of the process can be identified as more susceptible to error, then improvements can be made to enhance the process and its ability to reduce nonresponse bias. Using parameters from an RDD survey, we imposed nonresponse on a face-to-face survey with a much higher response rate in order to evaluate the error in the CG process. This approach provides a gold standard for respondents and nonrespondents, as well as for listed and unlisted telephone numbers. Preliminary findings show that although for some of the demographic variables incorrect matches of unlisted numbers contribute the most to bias in the CG process, surprisingly, bias is also relatively large for correctly matched telephone numbers. This article concludes with a warning for the use of the CG method for nonresponse bias evaluations in surveys, particularly, but not limited to, RDD surveys