(S-016) Verification of an algorithm for detection of pronation-supination movements in Parkinson’s disease.

Joshua Abrams, Ph.D. – Quantitative Medicine – Critical Path Institute; Grace Lee, Ph.D. – Quantitative Medicine – Critical Path Institute; Roopal Bhatnagar, M.S. – Quantitative Medicine – Critical Path Institute; Grace Erhart – Critical Path Institute; Shasta Jorgensen – Critical Path Institute; Shu Chin Ma, Ph.D. – Quantitative Medicine – Critical Path Institute; Diane Stephenson, Ph.D. – Critical Path Institute; Klaus Romero, M.D., M.S., F.C.P. – Critical Path Institute; Jamie Adams, M.D. – University of Rochester; Ray Dorsey, M.D. – University of Rochester; Sakshi Sardar, Ph.D. – Quantitative Medicine – Critical Path Institute; Martijn Muller, Ph.D. – CPP – Critical Path Institute

Objectives: To verify a pronation-supination-detection algorithm of inertial measurement unit (IMU) data collected in parallel with item 3.6 (Pronation-Supination of the Hands) of the MDS-UPDRS in comparison to visually annotated data by two independent raters.

Methods: The Wearable Assessments in the Clinic and at Home (WATCH-PD) dataset contains accelerometry data (256 Hz) collected in parallel with traditional clinical assessments from 40 normative and 73 individuals with early PD, who performed the MDS-UPDRS Part-3 in-clinic at a baseline visit and 5 follow-up visits spanning 12 months [1]. We analyzed repeated instrumented pronation-supination task data collected from Opal sensors by Clario (powered by APDM) placed on both wrists at the baseline visit.
A graphical user interface (GUI) was developed to inspect and annotate raw IMU recordings of the pronation-supination task across all study participants. The visualized data included timeseries of acceleration, gyroscope, and magnetometer data. Two independent raters used this GUI to inspect and annotate the data to obtain start- and end-times of the task.
Raters were instructed to identify an oscillatory signal in the data indicative of the pronation-supination task. Annotations were inspected for quality assurance before assessing inter-rater reliability. A Jaccard Index (JI)-based filtration process was implemented over all annotations to determine if annotations from each rater corresponded to the same activity. If the 2-way JI > 0, raters’ annotations referred to the same activity. For each recording, we computed confusion matrices of raters to evaluate agreement of where the activity does and does not occur. Construct validity was established by noting how many pronation-supination activities were commonly identified and computing the average total agreement between raters over all recordings from the baseline visit.
A detection algorithm was developed to identify pronation-supination events from accelerometry data and compared against human rater annotations. The algorithm assumes that the power (energy transferred per unit time, expressed in Watt) of the movement of the activity is orders of magnitude larger than the noise generated from the medical device [3]. This algorithm takes the short time Fourier transform (STFT) of the input signal, integrates it over all frequencies to obtain a time-dependent power signal and applies a global threshold parameter set to 10 W.

Results: Across 111 recordings inspected, there were 121 commonly identified pronation-supination movements between the raters resulting in a 98.81% average agreement of where the activity occurs across all files. The algorithm also identified the 121 pronation-supination movements achieving an average total agreement of 96.85% with the human raters of where the activity occurs.

Conclusions: Our results suggest the algorithm assumption captures the rater instructions and accurately detects when the activity occurs. The development of this algorithm provides a tool to objectively mark activities, which will accelerate an otherwise manually intensive step. Verification of this algorithm provides a critical step in the development of clinically relevant features derived from digital data.

Citations: [1] Adams, J.L., Kangarloo, T., Tracey, B. et al. Using a smartwatch and smartphone to assess early Parkinson’s disease in the WATCH-PD study. npj Parkinsons Dis. 9, 64 (2023).
[2] Goldsack, J.C., Coravos, A., Bakker, J.P. et al. Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). npj Digit. Med. 3, 55 (2020). https://doi.org/10.1038/s41746-020-0260-4
[3] MacDougall HG, Moore ST. Marching to the beat of the same drummer: the spontaneous tempo of human locomotion. J Appl Physiol (1985). 2005 Sep;99(3):1164-73. doi: 10.1152/japplphysiol.00138.2005. Epub 2005 May 12. PMID: 15890757.

Keywords: Digital Health Technology, Verification, Digital Biomarker