TY - JOUR

T1 - Reproducibility of findings in modern PET neuroimaging

T2 - insight from the NRM2018 grand challenge

AU - Veronese, Mattia

AU - Rizzo, Gaia

AU - Belzunce, Martin

AU - Schubert, Julia

AU - Searle, Graham

AU - Whittington, Alex

AU - Mansur, Ayla

AU - Dunn, Joel

AU - Reader, Andrew

AU - Gunn, Roger N.

AU - Albrecht, Daniel S.

AU - Mandeville, Joseph B.

AU - Sander, Christin Y.

AU - Price, Julie

AU - Levine, Michael A.

AU - Rullmann, Michael

AU - Becker, Georg Alexander

AU - Barthel, Henryk

AU - Hesse, Swen

AU - Sattler, Bernhard

AU - Sabri, Osama

AU - Zanderigo, Francesca

AU - Rubin-Falcone, Harry

AU - Ogden, Todd

AU - Johansson, Jarkko

AU - Jonasson, Lars

AU - Grill, Filip

AU - Karalija, Nina

AU - Rieckmann, Anna

AU - Boellaard, Ronald

AU - Golla, Sandeep

AU - Yaqub, Maqsood

AU - Erlandsson, Kjell

AU - Thomas, Benjamin A.

AU - Kr€amer, Stefanie D.

AU - Lawson, Lucas Narciso

AU - Lawson, Udunna Anazodo

AU - Norgaard, Martin

AU - Ganz, Melanie

AU - Schain, Martin

AU - Svarer, Claus

AU - Hansen, Hanne D.

AU - Knudsen, Gitte M.

AU - Smith, Christopher T.

AU - Jonasson, My

AU - Lubberink, Mark

AU - Tonietto, Matteo

AU - and the Grand Challenge Participants#

PY - 2021/10

Y1 - 2021/10

N2 - The reproducibility of findings is a compelling methodological problem that the neuroimaging community is facing these days. The lack of standardized pipelines for image processing, quantification and statistics plays a major role in the variability and interpretation of results, even when the same data are analysed. This problem is well-known in MRI studies, where the indisputable value of the method has been complicated by a number of studies that produce discrepant results. However, any research domain with complex data and flexible analytical procedures can experience a similar lack of reproducibility. In this paper we investigate this issue for brain PET imaging. During the 2018 NeuroReceptor Mapping conference, the brain PET community was challenged with a computational contest involving a simulated neurotransmitter release experiment. Fourteen international teams analysed the same imaging dataset, for which the ground-truth was known. Despite a plurality of methods, the solutions were consistent across participants, although not identical. These results should create awareness that the increased sharing of PET data alone will only be one component of enhancing confidence in neuroimaging results and that it will be important to complement this with full details of the analysis pipelines and procedures that have been used to quantify data.

AB - The reproducibility of findings is a compelling methodological problem that the neuroimaging community is facing these days. The lack of standardized pipelines for image processing, quantification and statistics plays a major role in the variability and interpretation of results, even when the same data are analysed. This problem is well-known in MRI studies, where the indisputable value of the method has been complicated by a number of studies that produce discrepant results. However, any research domain with complex data and flexible analytical procedures can experience a similar lack of reproducibility. In this paper we investigate this issue for brain PET imaging. During the 2018 NeuroReceptor Mapping conference, the brain PET community was challenged with a computational contest involving a simulated neurotransmitter release experiment. Fourteen international teams analysed the same imaging dataset, for which the ground-truth was known. Despite a plurality of methods, the solutions were consistent across participants, although not identical. These results should create awareness that the increased sharing of PET data alone will only be one component of enhancing confidence in neuroimaging results and that it will be important to complement this with full details of the analysis pipelines and procedures that have been used to quantify data.

KW - data analysis

KW - data sharing

KW - PET

KW - reproducibility crisis

KW - “NRM2018 PET Grand Challenge”

UR - http://www.scopus.com/inward/record.url?scp=85111182284&partnerID=8YFLogxK

U2 - 10.1177/0271678X211015101

DO - 10.1177/0271678X211015101

M3 - Journal article

C2 - 33993794

AN - SCOPUS:85111182284

VL - 41

SP - 2778

EP - 2796

JO - Journal of Cerebral Blood Flow and Metabolism

JF - Journal of Cerebral Blood Flow and Metabolism

SN - 0271-678X

IS - 10

ER -