Assessing validity of the Fitbit indicators for US public health surveillance

Abstract

Personally generated health data are increasingly used to report on population prevalence and trends, providing a new avenue for public health surveillance.1 Documentation of acceptable measurement properties to ensure correct interpretations should precede their use. One common source of personally generated health data comes from activity trackers, self-worn devices that provide feedback and long-term tracking on physical activity–related metrics.2 Activity trackers are relatively unobtrusive and low cost, with 12.5% of U.S. adults reporting wearing one in 2015.3 Companies selling activity trackers already report on data acquired by their users.4,5 In 2015, the U.S. Fitbit Health and Activity Index was launched (and updated in 2017), providing a suite of metrics including (1) prevalence of five indicators (steps, active minutes, resting heart rate, sleep, BMI), (2) popular Fitbit activities, and (3) time trends in activities. Using company-provided online tools, users can cross-tabulate three Fitbit indicators (steps, active minutes, resting heart rate) with diabetes, obesity, or cardiovascular disease (from the 2014 Behavioral Risk Factor Surveillance System [BRFSS]). An expert panel recommended assessing the psychometric properties of instruments for surveillance,6 but the validity of these Fitbit indicators is unknown. Thus, this study explored whether the Fitbit indicators of physical activity (steps, active minutes), resting heart rate, and BMI provided evidence for validity for use as a surveillance tool. Methods The Fitbit company evaluated aggregated data from >10 million users between June 2015 and June 2016 and published results in 2017. In February 2017, average steps/day, active minutes/day, resting heart rate, and BMI were abstracted by state or district from their website (www.fitbit.com/activity-index). All measures except BMI were Fitbit-assessed. Height and weight were entered typically at account set up. These data were compared to state- or district-based data from the 2015 BRFSS (www.cdc.gov/brfss/). The BRFSS is an ongoing, state-based random-digit dialed telephone survey of noninstitutionalized adults aged ≥18 years. Participants self-reported about physical activity or exercise in the past month, including the type, duration, and frequency of up to two activities. Physical activities were summed in minutes/week for both total and vigorous intensity.7 Estimated maximal oxygen uptake (VO2) was age–gender specific.7 BMI was derived in kg/m2 using self-reported height and weight. Spearman rank correlation coefficients provided associations between BRFSS and Fitbit indicators. As a guide, these ratings indicated agreement level8: 0–0.2 poor, 0.2–0.4 fair, 0.4–0.6 moderate, 0.6–0.8 substantial, and 0.8–less than 1.0 almost perfect. Bland–Altman plot for BMI indicated direction of bias.9 Analyses were conducted using SAS, version 9.3, and data from both sources were deidentified and publicly available. Results Both steps and active minutes Fitbit indicators showed a poor association with VO2 and a fair association with vigorous activity (Table 1). The resting heart rate Fitbit indicator showed a poor association with VO2 and total physical activity, and a fair association with vigorous activity. The BMI Fitbit indicator showed a fair association with BMI.