Nature and Science of Sleep (Apr 2022)
Multi-Night at-Home Evaluation of Improved Sleep Detection and Classification with a Memory-Enhanced Consumer Sleep Tracker
Abstract
Shohreh Ghorbani,* Hosein Aghayan Golkashani,* Nicholas IYN Chee, Teck Boon Teo, Andrew Roshan Dicom, Gizem Yilmaz, Ruth LF Leong, Ju Lynn Ong, Michael WL Chee Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, Singapore*These authors contributed equally to this workCorrespondence: Michael WL Chee, Centre for Sleep and Cognition, Yong Loo Lin School of Medicine, National University of Singapore, MD1 Level 13 Rm 05B, 117549, Singapore, Tel +65 66013199, Email [email protected]: To evaluate the benefits of applying an improved sleep detection and staging algorithm on minimally processed multi-sensor wearable data collected from older generation hardware.Patients and Methods: 58 healthy, East Asian adults aged 23– 69 years (M = 37.10, SD = 13.03, 32 males), each underwent 3 nights of PSG at home, wearing 2nd Generation Oura Rings equipped with additional memory to store raw data from accelerometer, infra-red photoplethysmography and temperature sensors. 2-stage and 4-stage sleep classifications using a new machine-learning algorithm (Gen3) trained on a diverse and independent dataset were compared to the existing consumer algorithm (Gen2) for whole-night and epoch-by-epoch metrics.Results: Gen 3 outperformed its predecessor with a mean (SD) accuracy of 92.6% (0.04), sensitivity of 94.9% (0.03), and specificity of 78.5% (0.11); corresponding to a 3%, 2.8% and 6.2% improvement from Gen2 across the three nights, with Cohen’s d values > 0.39, t values > 2.69, and p values < 0.01. Notably, Gen 3 showed robust performance comparable to PSG in its assessment of sleep latency, light sleep, rapid eye movement (REM), and wake after sleep onset (WASO) duration. Participants < 40 years of age benefited more from the upgrade with less measurement bias for total sleep time (TST), WASO, light sleep and sleep efficiency compared to those ≥ 40 years. Males showed greater improvements on TST and REM sleep measurement bias compared to females, while females benefitted more for deep sleep measures compared to males.Conclusion: These results affirm the benefits of applying machine learning and a diverse training dataset to improve sleep measurement of a consumer wearable device. Importantly, collecting raw data with appropriate hardware allows for future advancements in algorithm development or sleep physiology to be retrospectively applied to enhance the value of longitudinal sleep studies.Keywords: consumer wearable, sleep tracking, validation, sleep staging, sleep detection
