Artificial Intelligence (AI) in healthcare, especially based on Machine Learning (ML) techniques, holds significant promise for the field. These techniques have been extensively applied to address various clinical challenges, including pathology detection in X-rays, CT and MRI scans, mammography, skin cancer detection, diabetic retinopathy identification, and predicting readmissions and post-surgery complications. However, despite these advancements, AI-based systems remain notably absent in current clinical practice, limiting their clinical impact. One key factor contributing to this gap is the prevalent technology-centric approach to AI innovation, which often results in the limited clinical usefulness of AI-based support systems.
Through this thesis, comprising this kappa and four of my publications, I address the problem of innovating, i.e., designing, developing, and integrating AI-based systems considered useful by medical professionals in practice. The research presented in this thesis was conducted within the framework of the AI4XRAY project (2020-2025) - an interdisciplinary project aimed at creating a chest X-ray support tool for radiologists in both Denmark and Kenya. I used a combination of literature review, ethnographic work, and design work to investigate the clinical usefulness of AI-based systems in healthcare. The literature review aimed to identify the challenges of realising AI in clinical practice, while the ethnographic work involved in-situ observations and interviews with medical professionals and AI engineers in Denmark and Kenya. The design work consisted of grounded envisioning and design interventions to explore the opportunities for AI support in chest X-ray practice and configurations affordances of AI support for chest X-ray practice. I analysed the collected data using grounded theory and thematic analysis methods. This thesis presents five key contributions that contribute to the understanding of clinical usefulness and inform the realisation of clinically useful AI-based systems. Moreover, this thesis emphasises the interdisciplinary nature of clinical AI innovation, making it relevant to practitioners and researchers in Human-Computer Interaction (HCI), AI, and healthcare domains. First, I enrich the conceptualisation of clinical usefulness with four novel perspectives. I demonstrate how the end-users’ expectation for real-world performance depends on the intended use. Additionally, I show how the pre-labelling work on medical datasets for training AI conditions real-world performance, organisational acceptance, and clinical efficacy. Moreover, I highlight how broadening the AI design space enables organisational acceptance and clinical efficacy. I explore how configurable AI boosts organisational acceptance and clinical efficacy. Importantly, these dependencies are not exhaustive, and further research may expand them.Second, based on a systematic literature review, I find that challenges afflicting the realisation of clinical AI in practice stem not from a single issue but rather from sociotechnical interdependencies present when introducing AI into a clinical context. I conceptualise five challenges spanning three technical (training data & ML model, system integration & data used, and the user interface) and three social (user & system use, workflow & organisation, and healthcare institution & political arenas) aspects. I argue that addressing these challenges necessitates close collaboration among stakeholders with expertise in HCI, AI, and healthcare throughout the innovation processes. Third, I underscore the importance of attending to the pre-labelling phase in dataset creation. Particularly, I highlight how external and internal factors: regulatory constraints, the context of creation and use, commercial and operational pressures, epistemic differences, and limits of labelling condition the type of data that could be collected, the purpose for which it could be used, and the design of the ground truth schemas, i.e., the selection of labels and additional metrics annotated on the collected data. These fundamental decisions have consequences for shaping the design space of future AI-based systems that use such datasets. Fourth, I propose five visions for AI support grounded in practical challenges of chest X-ray practice faced across clinical contexts. The visions include distributing examinations by user’s expertise, detecting medical emergencies, providing decision support on subtle and difficult cases, measuring visual features and comparing changes across historical examinations, and double-checking reports against radiographs for missed or misinterpreted findings. These visions transcend functionalities traditionally emerging from technology-centred innovation processes and offer nuanced insights into potential AI applications in radiology. Finally, I delineate how AI-based systems should be configured both before and in use to realise previous visions in practice. The purpose of the configuration is to align the technical dimensions of AIbased systems with clinical needs that depend on social dimensions of clinical practice. The social dimensions span medical knowledge, clinic type, user expertise level, patient context, and user situation. The technical dimensions of AI comprise medical focus, functionality, decision threshold, and explainability methods. By ensuring alignment between these dimensions, AI-based systems can deliver value in concrete situations for concrete medical professionals in clinical practice. I advocate for ongoing consideration of these dependencies to ensure that the AI-based systems undergo necessary configuration before use and include necessary configurability options in use.