Systems and processes for detecting interviewer falsification and assuring data collection quality

Abstract

Falsification in survey data exists at many stages of data collection and presents serious consequences to the stakeholders. Quality assurance teams, charged with confirming the authenticity of interviewer-collected survey data, encounter ever-changing challenges in performing this task. This environment calls for multiple methods that aid the monitoring of data collectors and review of data after receipt. This paper describes techniques used at RTI and examples of falsification detectable by those means. When feasible, use of computer audio-recorded interviewing (CARI) allows detailed oversight of in-person surveys that otherwise lack direct observation, and it can augment live monitoring. Collection of global positioning system (GPS) coordinates can confirm the location of interviews conducted by field staff using mobile devices. In addition, we follow standard practices for quality management; we conduct systematic review of paradata and response data for item non-response, timing outliers, duplicate records and high levels of CARI refusals, and our survey supervisors hold quality-review discussions on a regular basis. The combination of these techniques provides an adaptable, multi-sourced, evidence-based process for quality assurance and control.