A general approach for estimating scale score reliability for panel survey data

Abstract

Scale score measures are ubiquitous in the psychological literature and can be used as both dependent and independent variables in data analysis. Poor reliability of scale score measures leads to inflated standard errors and/or biased estimates, particularly in multivariate analysis. Reliability estimation is usually an integral step to assess data quality in the analysis of scale score data. Cronbach's alpha is a widely used indicator of reliability but, due to its rather strong assumptions, can be a poor estimator (L. J. Cronbach, 1951). For longitudinal data, an alternative approach is the simplex method; however, it too requires assumptions that may not hold in practice. One effective approach is an alternative estimator of reliability that relaxes the assumptions of both Cronbach's alpha and the simplex estimator and thus generalizes both estimators. Using data from a large-scale panel survey, the benefits of the statistical properties of this estimator are investigated, and its use is illustrated and compared with the more traditional estimators of reliability