Multivariate tests for phase capacity

Abstract

To combat the potentially detrimental effects of nonresponse, most surveys repeatedly followup with nonrespondents, often targeting a response rate or predetermined number of completes. Each additional recruitment attempt generally brings in a new wave of data, but returns gradually diminish over the course of a static data collection protocol. Consequently, (nonresponse-adjusted) point estimates calculated from the accumulating data begin to stabilize. This is the notion of phase capacity, suggesting some form of design change is warranted, such as switching modes, increasing the incentive, or simply discontinuing nonrespondent follow-up.Phase capacity testing methods that have appeared in the literature to date are generally only applicable to a single point estimate. It is unclear how to proceed if conflicting results are obtained following independent tests on two or more point estimates. The purpose of this paper is to introduce two multivariate phase capacity tests, one referred to as the Wald chi-square method and another referred to as the non-zero trajectory method. Both methods are designed to provide a universal, yes-or-no phase capacity determination for a battery of point estimates. The two competing methods' performance is compared via simulation and application using data from the 2011 Federal Employee Viewpoint Survey. All else equal, the Wald chi-square method is found to detect phase capacity sooner than the non-zero trajectory method.