GMS Medizinische Informatik, Biometrie und Epidemiologie (Aug 2020)
Comparison of accuracy of activity measurements with wearable activity trackers in wheelchair users: a preliminary evaluation
Abstract
Background: Central nervous system diseases or injuries such as spinal cord injury (SCI) are often associated with a severe impairment of ambulatory function and result in wheelchair dependency. Typical long-term complications of wheelchair users include pressure injuries, spasticity, musculoskeletal pain and psychological issues. It is hypothesized that the occurrence of these complications is related to the level of physical activity (PA). While a low level of PA often results in skin or cardiovascular problems, a very high PA level results in shoulder pain and muscular fatigue. However, current evidence for this hypothesis is based on qualitative data from patient interviews. To investigate the relation between PA and health complications in a more objective manner, we propose the use of wearable activity trackers. As a first step, accuracy of common trackers – Apple Watch Series 4 and Fitbit Flex 2 – for quantification of wheelchair pushes was determined. For the Apple Watch, two different conditions are compared: out of the box usage and usage with GPS calibration.Methods: We used a 200 m outdoor test course. Healthy subjects were asked to propel with a wheelchair along the course while ground truth was captured by manually counting the wheelchair pushes. This procedure was conducted twice: once with the GPS-calibrated Apple Watch and Flex 2 in parallel and once with the non-calibrated Apple Watch. We analyzed the reproducibility of the ground truth measurement method by calculating the interrater reliability for two rater roles. To compare for accuracy of the activity trackers, we calculated differences between trackers and ground truth and analyzed them in Bland-Altman plots. To conclude on the need for GPS calibration of the Apple Watch in future studies, we did an equivalence test.Results: Twenty subjects without motor impairments participated in driving the test course. We found a reproducibility of the ground truth measurement of ICC(2,1)=0.981 (CI=0.96<ICC<0.99). The percentage of error for the calibrated Apple Watch is 13.9%, for the uncalibrated Apple Watch 22.8% and for the Flex 2 59.7%. Bland-Altman plots indicate a tendency for a higher error for test series with a higher number of pushes for the Flex 2. The equivalence test was significant for the defined equivalence boundaries of 15%.Conclusions: The ICC of the ground truth measurements indicates a high reproducibility of manual counting. Percentages of error show the highest accuracy for the calibrated Apple Watch. However, the results of the significant equivalence test suggest that the Apple Watch can be used without complex and time-consuming calibration. Due to extremely high error, PA tracking by Fitbit Flex 2 cannot be recommended for wheelchair users. This work is intended to serve as a basis for a future evaluation study of an activity tracker, for which we recommend using the uncalibrated Apple Watch in chronic wheelchair users. The evaluated tracker is intended to be used in a larger healthcare registry for follow-up of SCI patients.
Keywords
