Using Call-Level Interviewer Observations to Improve Response Propensity Models

Abstract

Response propensities are increasingly being used during data collection for responsive survey design applications. Better propensity models, with more predictive power, are needed to guide responsive interventions during data collection. However, available data on both respondents and nonrespondents to improve these models are limited. This analysis investigates the usefulness of a new type of paradata, in the form of an interviewer observation taken at the call level, to increase the predictive power of propensity models. The interviewer observation investigated is the interviewer’s assessment of the likelihood that a given respondent will participate in the survey, collected at each contact of a telephone study. We evaluate whether these ratings improve the fit and discrimination of “classic” response propensity models that include call record data and interviewer characteristics. Then, the performance of the interviewer observation is tested in a simulated responsive survey design, where propensity models are generated at several dates during data collection, to determine whether the observations improve the accuracy of the predictions made at these time points over using classic propensity models. The interviewer ratings not only contribute to the predictive power of the classic models overall but, on their own, appear to perform better than the classic model at predicting high probability cases in the responsive survey design context.