The Validity of Survey Responses: Insights from Interviews of Married Couples in a Survey of Soviet Emigrants

Abstract

Survey researchers have given little attention to the reliability of respondents as informants about others in the household. Data from the Soviet Interview Project General Survey, a study based on recent emigrants from the Soviet Union to the United States, provide a special opportunity to test for the accuracy of recall and of reporting about others by measuring the level of agreement between the answers by 163 married couples who lived in the same household in the USSR. It also tests for the response effects of two sources of potential 'contamination' in the interview setting: having multiple respondents in a family who are interviewed at different times and who thus could discuss the survey; and having third parties present during the interview. The analysis shows a very high level of agreement between spouses on measures of household material status in the USSR (income, wealth, housing space), probably stemming from the high saliency of these measures as status indicators. Contrary to expectations, neither the agreement on objective questions nor on subjective questions is affected by possible contamination of the interviews. A low level of agreement on answers to some ostensibly objective questions occurs, however, because the behavior was variant over time, definitions were ambiguous, or the question was not properly administered. The analysis also shows that studying divergent answers by associated pairs of respondents to questions to which there should be substantial agreement can reveal unanticipated ambiguities in question wording and thus can be an aid in debugging and interpreting survey data