Survey Research Innovations
Survey Methods and Data Collection Management
Statistics and Data Analysis
Survey Methods and Data Collection Management
Handheld Devices
We were an early adopter of handheld computers for collecting survey data. As part of the 1999 National Household Survey of Drug Abuse (now the National Survey of Drug Use and Health), we collected screener data from over 100,000 households using the Apple Newton. Since then, we have incorporated handheld computers into dozens of studies, and we have recently developed an ACASI application for handheld computers as well.
Cell Phone Surveys
With coverage of the U.S. adult population rapidly decreasing in random-digit-dialed landline telephone surveys, methods are needed to implement multiple sampling frames that improve coverage, while avoiding the introduction of new sources of error. Dual frame landline and cell phone surveys have been conducted at RTI and research is continually being conducted of the development of improved methods that correctly identify selection probabilities, address measurement differences, combine estimates, and reduce total survey error.
Contact: Lisa Carley-Baxter
Smartphone Surveys
With the advancement and proliferation of communication technologies, new opportunities arise for probability-based surveys. We are conducting experiments to evaluate and develop methods for the use of phones with Web capabilities.
Virtual Call Centers
The Research Operations Center (ROC) developed a virtual call center (VCC) which allows telephone interviewers to work at home. The VCC infrastructure allows RTI to support "virtual" (at home) interviewers and affords unlimited capacity and interviewer recruitment capabilities. Virtual interviewer's work at home using data collection systems and applications managed and monitored from our Raleigh, North Carolina call center. The VCC employs the latest VoIP/TDM hybrid technology while accessing the RTI-developed Web-based computer telephone integrated (CTI) system converging our CATI applications, case management system, and telephony application. All are hosted on our secure call center and corporate network infrastructure into a unified contact/call center.
Contact: Richard Heman-Ackah
Call Center NIST Compliance
RTI used National Institute of Science and Technology (NIST) Special Publication (SP) 800-37, Guide for the Security Certification and Accreditation of Federal Information Systems, as the basis of an assessment conducted by an independent certified security consulting firm. Using the Federal Information Processing Standards (FIPS) 199 guidelines, the security impact level for each of the three security objectives of confidentiality, integrity, and availability was determined to be Low, and the Call Center LAN was granted an Authority to Operate (ATO). Then, in response to increased security concerns and the needs of our clients, RTI implemented the Enhanced Security Network (ESN) in October 2008 with controls to ensure that data from all projects collecting personally identifying information (PII) and personal health information (PHI) are secured in a NIST Moderate environment. Our CATI systems on the Call Center Private LAN can provide either NIST Low or Moderate data security as required.
Contact: Richard Heman-Ackah
DocMan
We developed the DocMan system to manage all of the documents used by field (CAPI) studies in a secure manner. DocMan is a custom document repository that supports the scanning, encryption, verification, and transmission of paper-based forms. The software is used to track paper-based consent forms, case notes, and other scanned documents. The DocMan system includes the capability of capturing digital signatures on consent and other forms. The digital signatures are validated by an electronic tablet and the validation is CFR-part 11 compliant. In addition, RTI's DocMan can provide case history information (including tracing and locating data) to the field interviewers through a password-protected, unprintable, and unpasteable PDF document. All of the documents collected for a case are zipped, encrypted, and then transmitted back to RTI. This technique, in addition to making the documents secure, returns the forms much faster than mailing can.
Contact: R. Suresh
iLearning
We have developed and introduced multimedia training applications for use by both field interviewers and virtual call center interviewers. First developed in 2005 and launched in 2006 for the National Survey on Drug Use and Health (NSDUH) and later expanded for training of virtual call center interviewers, our iLearning applications enable interviewers to view standardized, high-quality training courses on various topics at their own pace. The applications include multimedia training tools such as video, audio, complex graphics, and interactive assessment questions.
Bilingual Interviewer Certification
Our language methodologists developed a protocol to evaluate the language proficiency of field and telephone interviewers. Spanish-speaking interviewers are required to pass a language proficiency assessment before being allowed to interview in Spanish. The protocol is easily adaptable for other languages as well.
Contact: Marjorie Hinsdale-Shouse
Computer-Assisted Interviewing
We have long been committed to the use of computer-assisted interviewing. As early as 1981, RTI converted a national youth attitude tracking survey for the Department of Defense from paper and pencil to computer-assisted telephone interviewing (CATI), requiring a major evaluation of the questionnaire and survey methods to maximize the benefits of this new technology.
Contact: Rachel Caspar
Audio Computer-Assisted Self-Interviewing
We pioneered the development of audio computer-assisted self-interviewing (ACASI) for administering sensitive topic surveys. In ACASI, the respondent uses headphones connected to a laptop computer to listen to prerecorded questions and then keys his/her responses directly into the computer. In 2002, we received the AAPOR Innovators Award for the development of ACASI and the 57th Annual Conference of the American Association for Public Opinion Research (AAPOR). The Cycle V National Survey on Family Growth, conducted by RTI, was the first national study to use this technology.
Contact: Rachel Caspar
Computer Audio-Recorded Interviewing
Computer audio-recorded interviewing (CARI) is a laptop computer application developed by us in 1999 for audio recording of field interviews. It provides a means for assessing the authenticity and quality of the field interview, including the behavior for the field interviewer during the interview and the reactions of the respondent to survey questions. CARI was pioneered on the National Survey of Child and Adolescent Well-Being.
Contact: Rita Thissen
Telephone Audio Computer-Assisted Self-Interviewing
We developed an extension of ACASI for use with sensitive questions on telephone interviews, called T-ACASI. Telephone respondents are transferred from a live telephone interviewer to the T-ACASI system where they listen to prerecorded questions and key their answers directly into their telephone keypad.
Contact: Lisa Carley-Baxter
Cognitive Lab-Based Survey Design
We collaborated with other researchers in the initial study of designing survey questionnaires using cognitive laboratory methods. We were one of the first organizations to establish a cognitive laboratory and to advocate for its use as a regular component of instrument development. We currently operate cognitive laboratories in both Research Triangle Park, NC and in Washington, D.C. The labs have been outfitted to support usability testing as well.
RoboCAI
This software was created to facilitate the development and testing of specifications for the programming of computer-assisted survey instruments. RoboCAI allows the instrument developer to test the instrument using predefined testing scripts that are executed automatically, reducing the burden typically associated with manual testing of complex instruments. Results from the RoboCAI log allow the developer to identify and correct errors in the flow logic.
Questionnaire Appraisal System
The questionnaire appraisal system (QAS), originally developed by our researchers for use by the CDC, allows researchers to evaluate survey questions for potential problems that will adversely affect data quality using a structured, standardized methodology. More recently, the QAS was updated (QAS-04) to incorporate codes to identify potential problems associated with cross-cultural and translation issues.
Second Life
RTI maintains a research facility in Second Life. Second Life is an online virtual world with a 3-D graphical interface. In Second Life, residents design avatars (animated visual representations) for themselves and visit the world for socializing, world-building, and participating in an in-world virtual economy that depends on the sale and purchase of user-created content. As a global phenomenon (approximately 60 percent of Second Life residents live outside the United States), Second Life offers researchers the ability to reach a broad international population.
Open to the public, our building in Second Life includes a reception area, two small interviewing rooms designed for one-on-one interviews, and two large interviewing rooms designed for focus groups and other meetings. Our survey methods research in Second Life has focused initially on the logistics related to recruiting interview respondents, conducting self-administered surveys via virtual kiosk, and conducting in-avatar interviews.
Contact: Elizabeth Dean (Frances Meriman in SL), Research Survey Methodologist, 919-541-7445.
Statistics and Data Analysis
General Exponential Modeling for Weight Calibration
We have developed a general exponential modeling (GEM) program that can simultaneously apply weight adjustments for nonresponse and poststratification under predetermined restrictions. This allows a reduction in the additional variance induced in survey estimates from weights while preserving the effect of weight adjustments on point estimates such as means and proportions.
More Information: Weighting
Small Area Estimation
Small area estimation (SAE) is the process of using statistical models to link survey outcome variables to a set of predictor variables known for small domains or areas (such as sub-populations or small geographic areas) in order to predict domain-level estimates. SAE methodologies are particularly beneficial for policy makers and researchers who are interested in estimates for small domains but who lack adequate funds for a large-scale survey that could produce precise, direct survey estimates for the small domains. For example, SAE can be used to produce state-level estimates from national data with much greater precision than one would observe in state-level, direct survey estimates.
More Information: Small Area Estimation
GeoFrame
GeoFrame is a method of household field enumeration that has been developed and used successfully at RTI. It employs ground-level capture of both a digital photograph and geospatial coordinates of a dwelling unit to replace the typical address, unit description, and hand-drawn maps of traditional field enumeration. In most cases, GeoFrame allows listers to enumerate dwelling units as fast as they can take pictures. The pictures and coordinates are used to create maps that interviewers use to locate dwelling units selected for interview. The GeoFrame process can dramatically shorten the time between enumeration and the beginning of interviewing by efficiently producing paper, electronic, and web-based maps to direct interviewers accurately to sampled dwelling units.
Contact: Joseph McMichael
Sampling Frames Based on U.S. Postal Address Lists
With increasing pressure on survey resources, there is a need to develop less costly methods for various stages of the survey process while maintaining the properties of survey estimates. In area-probability surveys, a large cost is associated with the construction and maintenance of the sampling frame, traditionally with listers or interviewers listing all households in selected geographic segments. We have been developing and evaluating methods to use frequently updated U.S. postal address lists in the construction of sampling frames.
More Information: Sampling
SUDAAN
SUDAAN was originated by RTI researchers in 1972 and has evolved into a leading software package for analyzing correlated and complex survey data. SUDAAN is used to analyze clustered data arising in many applications, including complex sample surveys, randomized experiments, and epidemiological studies. SUDAAN offers procedures for descriptive statistics and for statistical modeling. The modeling procedures include both linear and multinomial-logistic models and a procedure for fitting proportional hazards (Cox) regression models for time-to-event data. SUDAAN fits marginal or population-averaged models using generalized estimating equations (GEE). Robust variance estimates are computed that fully account for intracluster correlation, unequal weighting, stratification, and without-replacement sampling.
More Information: SUDAAN
Identity Protection
We developed a statistical model for protecting the identity of study subjects in public use or other data sets.
More Information: Human Research Protection
RTI Analysis
RTI Analysis is a Web-based tool that allows real-time analysis of complex survey data through a point-and-click interface.
Latent Class Modeling
We use latent class modeling innovatively to produce survey estimates with less measurement error when multiple reports are available. This also helps our researchers reduce nonresponse bias by classifying respondents into latent classes based on their likelihood to respond.
