Leicester researchers are urging caution before artificial intelligence (AI) models such as ChatGPT are used in healthcare for ethnic minority populations.
Writing in the Journal of the Royal Society of Medicine, epidemiologists at the University of Leicester and University of Cambridge say that existing inequalities for ethnic minorities may become more entrenched due to systemic biases in the data used by healthcare AI tools.
AI models need to be ‘trained’ using data scraped from different sources such as healthcare websites and scientific research. However, evidence shows that ethnicity data are often missing from healthcare research. Ethnic minorities are also less represented in research trials.
Mohammad Ali, PhD Fellow in Epidemiology at the College of Life Sciences, University of Leicester, said: “This disproportionately lower representation of ethnic minorities in research has evidence of causing harm, for example by creating ineffective drug treatments or treatment guidelines which could be regarded as racist.
“If the published literature already contains biases and less precision, it is logical that future AI models will maintain and further exacerbate them.”
The researchers are also concerned that health inequalities could worsen in low- and middle-income countries (LMICs). AI models are primarily developed in wealthier nations like the USA and Europe and a significant disparity in research and development exists between high- and low-income countries.
The researchers point out that most published research does not prioritise the needs of those in the LMICs with their unique health challenges, particularly around healthcare provision. AI models, they say, may provide advised based on data on populations wholly different from those in LMICs.
While crucial to acknowledge these potential difficulties, say the researchers, it is equally important to focus on solutions. Mr Ali said: “We must exercise caution, acknowledging we cannot and should not stem the flow of progress.”
The researchers suggest ways to overcome potentially exacerbating health inequalities, starting with the need for AI models to clearly describe the data used in their development. They also say work is needed to address ethnic health inequalities in research, including improving recruitment and recording of ethnicity information. Data used to train AI models should be adequately representative, with key actors such as ethnicity, age, sex and socioeconomic factors considered. Further research is also required to understand the use of AI models in the context of ethnically diverse populations.
By addressing these considerations, say the researchers, the power of AI models can be harnessed to drive positive change in healthcare while promoting fairness and inclusivity.
As well as Mr Ali, the research team also included Professor Kamlesh Khunti CBE, Director of NIHR ARC East Midlands, Director of the Centre for Ethnic Health Research and Professor of Primary Care Diabetes and Vascular Medicine at the University of Leicester. And, also Dr Claire Lawson, Associate Professor at the University of Leicester, and Professor Angela Wood, Professor of Health Data Science at the University of Cambridge.
To access the study, ‘Addressing ethnic and global health inequalities in the era of artificial intelligence healthcare models: a call for responsible implementation’, click here.
Published on: 20 Jul 2023