RTI uses cookies to offer you the best experience online. By clicking “accept” on this website, you opt in and you agree to the use of cookies. If you would like to know more about how RTI uses cookies and how to manage them please view our Privacy Policy here. You can “opt out” or change your mind by visiting: http://optout.aboutads.info/. Click “accept” to agree.
The moderating effects of population characteristics
A potential biasing factor when employing non-random samples to conduct experimental research
Silver, I. A., & Kelsay, J. D. (2023). The moderating effects of population characteristics: A potential biasing factor when employing non-random samples to conduct experimental research. Journal of Experimental Criminology, 19(1), 107-118. https://doi.org/10.1007/s11292-021-09478-7
Objectives: Due to time and financial limitations, most randomized controlled trials (RCTs) are conducted employing non-random sampling techniques. Although valuable, when the unique characteristics of a non-random sample unknowingly interact with the treatment, the results of the RCT could become biased. Nevertheless, the amount of bias remains unexamined. Methods: The current study evaluated if non-random sampling techniques could bias the slope coefficients of an RCT when an interaction exists between the treatment and a characteristic in the population using two simulation analyses. Results: The results suggested that the sampling distributions of slope coefficients from an RCT — across random specifications — expand drastically when (1) an interaction between the treatment and a characteristic in the population exists and (2) the non-random sample has unique scores on that characteristic. Conclusions: Considering these findings, four recommendations are made for scholars currently or intending to conduct a RCT employing non-random sampling techniques.
