Randomized experiments for web-mail surveys conducted using address-based samples of the general population

Abstract

Web-mail surveys conducted using address-based samples exploit the advantages of web surveys with respect to cost, timeliness, measurement flexibility, reduced respondent burden, interactive question-answer processes, and the advantages of address-based samples with respect to coverage of a household population. The studies described here investigate the use of concurrent and web-intensive strategies in a general population web-mail survey with voluntary participation. A series of randomized experiments divided samples from the University of Michigan’s Surveys of Consumers (SOC) into groups that received either a concurrent request to complete the survey by mail or web or a web-intensive request to complete the survey by web before offering a mail alternative. These approaches were compared in terms of response rates, process measures, sample composition, and key substantive measures. Although differences in web-intensive vs. concurrent method respondent demographic characteristics were in expected directions, there were not clear differences in overall response rates or substantive findings. The web-intensive approach shifts mail respondents to the web mode but does not attract different subgroups of people to participate in the web survey who would not have participated in a mail survey. The percentages of completions by web in the concurrent or web-intensive methods were not high enough to realize the advantages of cost and timeliness in the overall survey. Further development in encouraging the use of web among respondents is needed to exploit these advantages.