Device-based measurement of physical activity in pre-schoolers: Comparison of machine learning and cut point methods

Abstract

Introduction Machine learning (ML) accelerometer data processing methods have potential to improve the accuracy of device-based assessments of physical activity (PA) in young children. Yet the uptake of ML methods by health researchers has been minimal and the use of cut-points (CP) continues to be the norm, despite evidence of significant misclassification error. The lack of studies demonstrating a relative advantage for ML approaches over CP methods maybe a key contributing factor. Purpose The current study compared the accuracy of PA intensity predictions provided by ML classification models and previously published CPs for preschool-aged children. Methods In a free-living study, 31 preschool-aged children (mean age = 4.0 ± 0.9 y) wore wrist and hip ActiGraph GT3X+ accelerometers while completing a video recorded 20-minute free play session. Ground truth PA intensity was coded continuously using the Children’s Activity Rating Scale (CARS). Accelerometer data was classified as sedentary (SED), light intensity (LPA), or moderate-to-vigorous intensity (MVPA) using ML random forest PA classifiers and published CPs for preschool-aged children. Performance differences were evaluated in a hold-out sample by comparing weighted kappa statistics, classification accuracy for each intensity band, and equivalence testing. Results ML classification models (hip: κ = 0.76; wrist: κ = 0.72) exhibited significantly higher agreement with ground truth PA intensity than CP methods (hip: κ = 0.38–0.49; wrist: κ = 0.31–0.44). For the ML models, classification accuracy for SED and LPA ranged from 83% - 88%, while classification accuracy for MVPA ranged from 68% - 78%. For the CP’s, classification accuracy ranged from 50% - 94% for SED, 19% - 75% for LPA, and 44% - 76.1% for MVPA. ML classification models showed equivalence (within ± 0.5 SD) with directly observed time in SED, LPA, and MVPA. None of the CP’s exhibited evidence of equivalence. Conclusions Under free living conditions, ML classification models for hip or wrist accelerometer data provide more accurate assessments of PA intensity in young children than CP methods. The results demonstrate the relative advantage of ML methods over threshold-based approaches and adds to a growing evidence base supporting the feasibility and accuracy of ML accelerometer data processing methods.