Wary Public Puts Strain on Survey Industry

The American Association for Public Opinion Research (AAPOR) is the leading association of public opinion and survey research professionals, including many survey statisticians like me. AAPOR’s annual conference, held this May in Toronto, brought together producers and users of survey data from a variety of disciplines—election polling, market research, statistics, research methodology, health related data collection, education, and government.

As David Dutwin highlighted in his presidential address, survey science is under a lot of stress. David was mostly referring to the delegitimizing of the results of polls and public opinion research. But he also mentioned that official statistics, even results of the census, are being questioned for political purposes. He concludes that survey practitioners need to publicly advocate for the legitimacy of our field.

“We have to learn to professionalize our communication and advocate for our members and our field,” he said. “There are no such thing as sidelines anymore. We must do our part to defend survey science, polling, and the very role of public opinion in a functioning democracy.”

For me, publicly advocating for the legitimacy of statistics is best left to others. I am more interested in the day-to-day problems of survey implementation in the context of population health surveillance. From my perspective, the largest problem facing survey research is that the people we sample refuse to participate more often than they used to. For phone surveys, this trend is accelerating. Phone, mail and e-mail telemarketing has increased dramatically. Telemarketers often use practices that would be considered unethical or deceitful by the data collection standards practiced at RTI and advocated by AAPOR. The members of the public do not always distinguish between legitimate surveys and telemarketers. Consequently, many people are wary of being contacted by strangers and refuse to participate in public health surveillance research.

At the conference, I was unsurprised to learn that the rising cost and declining response rates of telephone and ABS surveys are not unique to RTI; the entire field is grappling with the same issues. An AAPOR task force produced a report on telephone spam blocking and its impact on survey research. Unfortunately, AAPOR specifically, and the field of survey research in general, has very little power to exert influence in a rapidly changing telecommunications environment.

Many in our field are looking for solutions to these problems. There is a general trend to switch telephone surveys to some other method, such as ABS with a mail contact. Probability and nonprobability web panels are another alternative. I have been working on an emerging nonprobability sampling methodology called redirected inbound call sampling, or RICS, where callers who misdial idle toll-free numbers are redirected to a data collection system.

Another interesting topic this year was the use of nonprobability data. I learned about interesting advances in this nascent field, especially during a session dedicated to hybrid estimation with probability and nonprobability data. RTI’s Marcus Berzofsky described how he supplemented a probability sample with a sample from Twitter to get a hard-to-reach LGBTQ population. David Dutwin described an evaluation of different calibration methodology to reduce bias when combining data from a probability survey with a nonprobability web panel. Michael Yang and Andrew Mercer also presented on methodology for combining probability and nonprobability samples and evaluation of the quality of the estimates.  

While there are other professional organizations for statisticians, AAPOR has the highest proportion of members that specialize in survey statistics and survey methodology. Having RTI researchers as active members of AAPOR for 60 years keeps RTI on the forefront of the newest innovations.

Predictions are difficult. But one prediction is bound to be true: survey statistics is changing rapidly, and the future will look different than the present. AAPOR conferences are the best place to get the first look at the future of survey statistics.