For both the builders and users of clinical AI, bias remains one of the single greatest issues keeping the industry up at night. The prevailing anxiety revolves around one urgent question: how do we counteract it? When we hear the word “bias” in this space, we instantly frame this through a derogatory lens. It is almost universally treated as a synonym for unfairness, an algorithmic flaw that inherently discriminates against specific groups of people.
We saw exactly why this fear is justified when researchers discovered that a widely used algorithm was drastically reducing the number of Black patients flagged for high-risk care management. The model wasn’t explicitly programmed to harm anyone; it simply equated past health care costs with actual health needs. It was blind to the reality that marginalized populations historically spend less on health care due to systemic barriers to accessing care.
In the wake of findings like that, the instinct is immediate: eliminate the bias. Strip out race. Make the algorithm blind to demographics, and the problem disappears.
But removing race does not remove the racial signal. It removes the label. The imprint of inequity remains in ZIP codes, insurance status, prior utilization, referral patterns, and who can return for follow-up care. These variables quietly carry the same information. Masking demographics can make bias less visible, but it does not make it disappear.
When we train models on past spending, hospitalizations, or referral rates, we are training them on decisions made inside an inequitable health care system. The result is a model that excels at pattern recognition but cannot distinguish between true clinical need and barriers to access. Focusing narrowly on eliminating or masking race in the name of fairness can degrade both accuracy and equity, while giving the illusion that the problem has been solved. Bias in clinical AI is not a contaminant that can be scrubbed away by deleting a column. It demands ongoing measurement, transparency, and governance. If we want these systems to serve patients equitably, we must confront the reality that they are learning from a system that has never been neutral.
The reality of clinical AI is more complicated. What we often label as “bias” sometimes requires a different framing: intentional calibration. We do not need models that pretend every patient has equal or equitable access to care. We need models capable of recognizing disparities and responding to them.
Consider maternal health. In the United States, the chances of Black women dying during pregnancy or childbirth are more than three times higher than those of white women. If an AI model designed to monitor pregnant patients ignores this stark reality in the name of demographic neutrality, it fails its most vulnerable patients. Instead, developers can design the model to account for this disparity, for example, by lowering alert thresholds when patients face well-documented elevated risks. This is not unfair discrimination; it is a deliberate, compensatory measure designed to close a known and deadly care gap.
The same principle applies to diagnostic tools. If a health system deploys an AI model to detect skin cancer, training that algorithm predominantly on images of lighter skin tones guarantees it will miss potentially fatal melanomas in patients of color. An equitable model cannot rely on a generic data set; it must be intentionally engineered and heavily weighted with diverse images to ensure it performs accurately across all skin tones.
Furthermore, clinical AI must account for the logistical realities of access. An algorithm that optimizes clinic scheduling or patient outreach solely based on geographic distance may be mathematically efficient but clinically inequitable. To serve and reach historically marginalized populations, these tools must account for the compounding barriers, such as a lack of transportation and other socioeconomic factors, that determine whether a patient can access care.
In other words, clinical AI must weigh both what a patient needs and what stands in the way of meeting those needs. The goal is not to discriminate in favor of or against certain groups. The goal is to present an unflinching picture of the barriers that limit care for marginalized populations and design systems capable of navigating them.
Intentional calibration, however, cannot be improvised. It requires rigorous evaluation, transparency, and independent oversight. Good intentions are not enough. Health care needs clear standards for measuring equity, consistent reporting of model performance across populations, and third-party validation to ensure that adjustments meant to close gaps do not introduce new harm.
This is where independent accreditation becomes essential. Just as hospitals and health plans are evaluated against established quality standards, clinical AI systems require structured frameworks to assess safety, fairness, and governance before and after deployment. Emerging accreditation programs offer one pathway for evaluating how models are built, tested, monitored, and updated over time. The aim is not to stifle innovation but to create guardrails that make innovation trustworthy.
Building intentional equity into AI is not only about improving care; it is about preserving trust. According to recent Pew Research Center research, 60% of U.S. adults say they would feel uncomfortable if their health care provider relied on AI for their medical care. Patients are rightfully wary.
If we cling to the illusion that demographic blindness equals fairness, clinical AI will entrench the very disparities it promises to solve. But if we embrace the complexity of intentional calibration, backed by transparent standards and independent accountability, we can build tools that see the whole patient and help move a fragmented system closer to equity.
Photo: FotografiaBasica, Getty Images
Dr. Shakira J. Grant is a physician, entrepreneur, and healthcare executive working at the intersection of clinical medicine, health policy, and artificial intelligence. She is the founder of CROSS Global Research & Strategy, LLC, which provides end-to-end solutions to support responsible AI adoption in healthcare, with a focus on clinical workflow integration, equity, policy, AI literacy, and community access through its CARE AI program.
Board-certified in Internal Medicine, Geriatric Medicine, Hematology, and Medical Oncology, Dr. Grant brings nearly two decades of patient care experience to her work, ensuring AI solutions are clinically relevant, practical, and equitable. Her expertise also includes health policy, having served as an advisor to the U.S. House Ways and Means Committee. With more than 35 peer-reviewed publications focused on marginalized communities, she is recognized as a Healio Next-Gen Disruptive Innovator and is pursuing formal AI training through IBM certification coursework.
This post appears through the MedCity Influencers program. Anyone can publish their perspective on business and innovation in healthcare on MedCity News through MedCity Influencers. Click here to find out how.
