MCS Thesis Examination: Mohanad Arafe

Abstract

    Machine Learning (ML) techniques are growing in popularity for analyzing T1-weighted magnetic resonance imaging (MRI) as it is a promising source of Parkinson’s disease (PD) biomarkers. However, there is growing concern within the scientific community regarding the reproducibility of research findings. This reproducibility crisis suggests that a significant proportion of studies may not be reliable which impacts the validity of results in a preclinical setting.

    The objective of this paper is to reproduce and replicate the findings of a study by (Shu et al., 2020) that uses ML techniques to predict the progression of PD using conventional MRI and radiomic biomarkers in whole-brain white matter. We aim to assess the reproducibility and replicability of (Shu et al., 2020)’s predictive capabilities using open-source tools. We used the Parkinson’s Progression Markers Initiative (PPMI) dataset, the same dataset used by (Shu et al., 2020) and similar analyses to assess the reproducibility of the findings. While we attempted to follow the methods outlined in (Shu et al., 2020) as closely as possible, some details were unclear and we made educated guesses. We introduced variations in the methodological methods, including different cohorts, feature sets, ML algorithms, and evaluation techniques, to assess the replicability of the findings. Our study could not reproduce nor replicate the predictive capabilities of (Shu et al., 2020). The lack of reproducibility and replicability in this paper highlights the importance of adopting open science practices to ensure that proposed biomarkers are robust.

Examining Committee

• Dr. Yang Wang (Chair) 
• Dr. Tristan Glatard & Jean-Baptiste Poline (Supervisor)
  
• Dr. Marta Kersten-Oertel (Examiner)
• Dr. Yang Wang (Examiner)