Is AI in Healthcare Unfair? What Does the HEAL Framework Promise?
The HEAL framework developed by Google Research aims to measure performance disparities of machine learning models across different patient groups. So why has this been overlooked until now?
When it was announced that an AI model performed better than doctors at diagnosing skin cancer, everyone was excited. However, there was a rarely discussed detail: this model was much more successful on light-skinned patients compared to dark-skinned ones. A new method is now on the agenda precisely to detect such inequities.
The Google Research team announced they have developed a framework called HEAL. Standing for 'Health Equity Assessment of Machine Learning Performance', this tool, simply put, promises to measure whether algorithms used in healthcare work equally well for everyone.
Why Has It Been So Difficult Until Now?
The issue is actually not limited to skin color. Consider that a heart disease prediction model can provide much more accurate results for male patients compared to females. Or a model trained on data from affluent patients can become almost useless for low-income groups. However, existing evaluation methods mostly focused on overall success rates. Inequities hidden behind averages were easily overlooked.
So how exactly does HEAL work? The system, in addition to measuring model performance with traditional metrics (like accuracy, precision), breaks down this performance according to patients' demographic characteristics. It's like looking not only at the overall grade average of a class but also at each student's individual success. This makes it possible to get clear answers to questions like, "How successful is this model for elderly patients compared to younger ones?"
A Scenario from the Real World
Consider an AI tool used for diabetic retinopathy screening. This disease, which can show different symptoms in individuals of South Asian origin compared to those of European origin, can be missed by standard models. The HEAL framework offers a nuanced evaluation precisely at this point by comparing the model's 'false negative' rates across different ethnic groups. What's noteworthy is that this tool not only reveals the problem but also helps find the source of the inequity. Is the problem due to a lack of representation in the training data, or does it stem from a structural issue in the algorithm itself?
Recently, especially with the pandemic, automation in health technologies has accelerated. However, research from Boston revealed that emergency room triage (prioritization) algorithms tend to assign black patients a lower priority level compared to white patients presenting with the same complaint. Such cases show that the issue is not just a technical matter but also has ethical and social dimensions.
How long will it take for the HEAL framework to be adopted in practice? Experts say it requires a cultural shift. Developers can no longer be content with just saying 'our model works with 95% accuracy'. Instead, they will have to adopt a more transparent and fair language, such as: 'our model works with 95% accuracy, but this rate drops to 87% for patients over 65 and to 82% for those living in rural areas.' This will naturally bring along better and more inclusive data collection processes and more careful model training.
Perhaps the real question is this: Will technology be a tool that reinforces existing biases, or a lever that helps eliminate them? Initiatives like HEAL are part of the effort to choose the second path. Because equity in health is not the same treatment for everyone, but everyone having access to the treatment they need. For now, it is us humans who determine where AI will stand in this equation.
