Reducing Demographic Bias in Clinical AI Models With Color Space Augmentation

Artificial intelligence (AI) is reshaping clinical diagnostics, but one outstanding limitation is its inherent bias with available training data. In skin disease detection, most AI models rely on datasets dominated by lighter skin tones—leaving patients with darker skin at risk of misdiagnosis.

Nawsabah Noor, MBBS, assistant professor of medicine at Popular Medical College in Bangladesh, and colleagues are working on mitigating racial bias in AI models, specifically in identifying skin lesions in mpox in different skin colors. Noor presented initial findings of the AI model and app they are developing at the American College of Physicians (ACP) Internal Medicine (IM) Meeting 2025, held April 3-5, in New Orleans, Louisiana. HCPLive spoke with Noor to learn more about how the team worked to address the issue of racially biased training data and future applications for such a model and app.

HCPLive: Can you give me a little bit of background on the research and why it's important?

Nawsabah Noor, MBBS: As you know, we use AI very often in our day-to-day lives. But how does this AI actually work? AI learns by exposing itself to large datasets. It sees images, then develops a mathematical formula and comes to a conclusion—“This image represents this,” or “That image represents that.” So, as you can guess, it depends on the quality of the data. But if the data is biased—say, you want to develop an AI model for facial recognition, but most of your dataset includes only white-skinned individuals—it won’t work well for people with brown or dark skin tones, like me. This raises issues of racial bias. These are also referred to as demographic biases, and it’s a hot topic in AI right now.

Most public datasets available for disease detection—especially for skin-related issues—are based mostly on images of lighter-skinned individuals. If you search online, most of what you find are images of white skin. So, AI performs well on white people, but not as well on others. This issue has even been labeled "the white guy problem" in AI and has been well studied.

That’s one issue—racial bias in AI. The other issue is mpox. In 2022, we saw a large outbreak of mpox that rapidly spread outside of Africa to more than 40 countries. I’m from Bangladesh, and we were very cautious because countries nearby—like India and Pakistan—were also affected. Bangladesh is densely populated, and mpox is highly contagious, so we were concerned.

Early detection of mpox is difficult because the rash looks very similar to other conditions like chickenpox, measles, and hand, foot, and mouth disease. Diagnosing it early requires close inspection of the rash, plus awareness of other clinical symptoms. For example, mpox is more common among homosexual populations. The rashes can appear throughout the body, including the anus, and patients often report anorectal bleeding and pain. These are key distinguishing features.

Another issue is the availability of confirmatory testing. Diagnosis depends on PCR testing, which is not widely accessible in low-resource settings like ours. So, we wanted to develop a smart solution—an app powered by AI. Let’s say you develop a rash and are unsure whether it might be mpox. You could take a picture, upload it into the app, and the app would ask questions like: Do you have a fever? Do you have anal pain or bleeding? Are you heterosexual or homosexual? These questions help assess risk and support early prediction.

That’s how we came up with the idea. But when we started developing the AI model, we encountered racial bias. Most of the mpox training data we found was from African countries, meaning images featured dark-pigmented skin. When we trained our model with that data, it didn’t work well on skin like ours—not white, but also not as dark. So we asked, how can we reduce this racial bias?

Our innovation in the paper was proposing color space augmentation. You may know from photo editing apps that you can adjust hue, saturation, and brightness to create different versions of the same image. We applied that idea—taking an image of a rash on dark skin, for instance, and generating many variants by changing its hue and saturation. We ended up with thousands of augmented images, some with green, pink, or orange skin tones. By using these distorted images, the AI learns to focus only on the rash itself, not the skin color. It stops depending on surrounding skin features and instead targets the rash patterns. That’s the main idea.

We also used something called the Fitzpatrick skin tone scale, developed by a dermatologist, which classifies all skin tones into six types. Type I is ivory (very light), and Type VI is deeply pigmented brown skin. We changed the skin tones of our images across all six types to diversify the training data. After applying this color space augmentation, we saw our model’s accuracy improve to about 83%—a very promising result. We’re continuing to validate the model with more robust training and more collaboration. We’ve partnered with Johns Hopkins University and another research team in Congo to help upgrade the app and AI algorithm. That’s essentially our research.

Because it involves AI training, we physicians aren't typically experts in this area. We understand the clinical side—the rashes and symptoms—but we’re not always strong in algorithm development. That’s why this has been a collaboration between physicians, dermatologists, and biomedical engineers. We’re working closely with a group of engineers to develop the formulas. We hope this interdisciplinary collaboration will continue and have a positive impact worldwide.

Do you envision future applications beyond mpox—just reducing racial bias in AI in general?

Noor: Yes, absolutely. When we began this work, we also looked at other datasets. One of the dermatologists on our team raised a great point: If this app can detect mpox, maybe it could help detect other rashes too—like hand, foot, and mouth disease. We’re actually working on that now. It’s still early, but our vision is to develop a broader app or solution that can detect common rashes using AI. Some rashes, like sun-related skin irritation, are harmless. But others—like mpox—are contagious and serious. So being able to quickly distinguish between them would be really valuable.

Is there anything else you want to add for the audience?

Noor: Our app is still in the prototype phase, and we’ll be starting our pilot study soon. One limitation is that we don’t yet have enough data across all skin types. Thankfully, mpox hasn’t spread like COVID, but that means there’s less image data. With more diverse data, we can keep improving the app. As collaborations continue, I believe we’ll launch a much better version of the app soon—something that has a meaningful impact globally.

For physicians, I want to say: AI is coming. It can help us in many ways. We should embrace AI, think about how it can support clinical practice, and work with engineering teams to develop practical solutions. It may sound simple, but tools like this can make a real difference. We need to incorporate AI not just in our daily practice, but in our research as well.

This content has been edited for clarity.