Design of a deep learning model for automatic scoring of periodic and non-periodic leg movements during sleep validated against multiple human experts
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
Design of a deep learning model for automatic scoring of periodic and non-periodic leg movements during sleep validated against multiple human experts. / Carvelli, Lorenzo; Olesen, Alexander N.; Brink-Kjær, Andreas; Leary, Eileen B.; Peppard, Paul E.; Mignot, Emmanuel; Sørensen, Helge B.D.; Jennum, Poul.
In: Sleep Medicine, Vol. 69, 2020, p. 109-119.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Design of a deep learning model for automatic scoring of periodic and non-periodic leg movements during sleep validated against multiple human experts
AU - Carvelli, Lorenzo
AU - Olesen, Alexander N.
AU - Brink-Kjær, Andreas
AU - Leary, Eileen B.
AU - Peppard, Paul E.
AU - Mignot, Emmanuel
AU - Sørensen, Helge B.D.
AU - Jennum, Poul
PY - 2020
Y1 - 2020
N2 - Objective: Currently, manual scoring is the gold standard of leg movement scoring (LMs) and periodic LMs (PLMS) in overnight polysomnography (PSG) studies, which is subject to inter-scorer variability. The objective of this study is to design and validate an end-to-end deep learning system for the automatic scoring of LMs and PLMS in sleep. Methods: The deep learning system was developed, validated and tested, with respect to manual annotations by expert technicians on 800 overnight PSGs using a leg electromyography channel. The study includes data from three cohorts, namely, the Wisconsin Sleep Cohort (WSC), Stanford Sleep Cohort (SSC) and MrOS Sleep Study. The performance of the system was further compared against individual expert technicians and existing PLM detectors. Results: The system achieved an F1 score of 0.83, 0.71, and 0.77 for the WSC, SSC, and an ancillary study (Osteoporotic Fractures in Men Study, MrOS) cohorts, respectively. In a total of 60 PSGs from the WSC and the SSC scored by nine expert technicians, the system performed better than two and comparable to seven of the individual scorers with respect to a majority-voting consensus of the remaining scorers. In 60 PSGs from the WSC scored accurately for PLMS, the system outperformed four previous PLM detectors, which were all evaluated on the same data, with an F1 score of 0.85. Conclusions: The proposed system performs better or comparable to individual expert technicians while outperforming previous automatic detectors. Thereby, the study validates fully automatic methods for scoring LMs in sleep.
AB - Objective: Currently, manual scoring is the gold standard of leg movement scoring (LMs) and periodic LMs (PLMS) in overnight polysomnography (PSG) studies, which is subject to inter-scorer variability. The objective of this study is to design and validate an end-to-end deep learning system for the automatic scoring of LMs and PLMS in sleep. Methods: The deep learning system was developed, validated and tested, with respect to manual annotations by expert technicians on 800 overnight PSGs using a leg electromyography channel. The study includes data from three cohorts, namely, the Wisconsin Sleep Cohort (WSC), Stanford Sleep Cohort (SSC) and MrOS Sleep Study. The performance of the system was further compared against individual expert technicians and existing PLM detectors. Results: The system achieved an F1 score of 0.83, 0.71, and 0.77 for the WSC, SSC, and an ancillary study (Osteoporotic Fractures in Men Study, MrOS) cohorts, respectively. In a total of 60 PSGs from the WSC and the SSC scored by nine expert technicians, the system performed better than two and comparable to seven of the individual scorers with respect to a majority-voting consensus of the remaining scorers. In 60 PSGs from the WSC scored accurately for PLMS, the system outperformed four previous PLM detectors, which were all evaluated on the same data, with an F1 score of 0.85. Conclusions: The proposed system performs better or comparable to individual expert technicians while outperforming previous automatic detectors. Thereby, the study validates fully automatic methods for scoring LMs in sleep.
KW - Automatic event detection
KW - Leg movements during sleep
KW - Manual scoring of polysomnography
KW - Periodic leg movements during sleep
KW - Polysomnography
U2 - 10.1016/j.sleep.2019.12.032
DO - 10.1016/j.sleep.2019.12.032
M3 - Journal article
C2 - 32062037
AN - SCOPUS:85079175170
VL - 69
SP - 109
EP - 119
JO - Sleep Medicine
JF - Sleep Medicine
SN - 1389-9457
ER -
ID: 260998995