Ground Truth Or Dare: Factors Affecting The Creation Of Medical Datasets For Training AI
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Ground Truth Or Dare : Factors Affecting The Creation Of Medical Datasets For Training AI. / Zajac, Hubert Dariusz; Avlona, Rozalia Natalia; Andersen, Tariq Osman; Kensing, Finn; Shklovski, Irina.
AIES ’23, August 8–10, 2023, Montréal, QC, Canada. Association for Computing Machinery, 2023. p. 351–362.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Ground Truth Or Dare
T2 - 2023 AAAI/ACM Conference on AI, Ethics, and Society - AIES '23
AU - Zajac, Hubert Dariusz
AU - Avlona, Rozalia Natalia
AU - Andersen, Tariq Osman
AU - Kensing, Finn
AU - Shklovski, Irina
PY - 2023
Y1 - 2023
N2 - One of the core goals of responsible AI development is ensuring high-quality training datasets. Many researchers have pointed to the importance of the annotation step in the creation of high-quality data, but less attention has been paid to the work that enables data annotation. We define this work as the design of ground truth schema and explore the challenges involved in the creation of datasets in the medical domain even before any annotations are made. Based on extensive work in three health-tech organisations, we describe five external and internal factors that condition medical dataset creation processes. Three external factors include regulatory constraints, the context of creation and use, and commercial and operational pressures. These factors condition medical data collection and shape the ground truth schema design. Two internal factors include epistemic differences and limits of labelling. These directly shape the design of the ground truth schema. Discussions of what constitutes high-quality data need to pay attention to the factors that shape and constrain what is possible to be created, to ensure responsible AI design.
AB - One of the core goals of responsible AI development is ensuring high-quality training datasets. Many researchers have pointed to the importance of the annotation step in the creation of high-quality data, but less attention has been paid to the work that enables data annotation. We define this work as the design of ground truth schema and explore the challenges involved in the creation of datasets in the medical domain even before any annotations are made. Based on extensive work in three health-tech organisations, we describe five external and internal factors that condition medical dataset creation processes. Three external factors include regulatory constraints, the context of creation and use, and commercial and operational pressures. These factors condition medical data collection and shape the ground truth schema design. Two internal factors include epistemic differences and limits of labelling. These directly shape the design of the ground truth schema. Discussions of what constitutes high-quality data need to pay attention to the factors that shape and constrain what is possible to be created, to ensure responsible AI design.
U2 - 10.1145/3600211.3604766
DO - 10.1145/3600211.3604766
M3 - Article in proceedings
SP - 351
EP - 362
BT - AIES ’23, August 8–10, 2023, Montréal, QC, Canada
PB - Association for Computing Machinery
Y2 - 8 August 2023 through 10 August 2023
ER -
ID: 362452948