State of the Art and Science
Mar 2021

Piloting and Scaling a Good Health Equity Evidence Base From Big Data

Stephen Lockhart, MD, PhD
AMA J Ethics. 2021;23(3):E252-257. doi: 10.1001/amajethics.2021.252.


Eliminating racial inequity in health outcomes has historically been complicated by a lack of clear methods to quantify the problems and study interventions’ effects. Health care organizations’ investment in electronic health record systems for millions of patients, however, presents opportunities to use data to research health inequity and respond to it. One health system’s development and validation of a measure to identify and quantify outcomes inequity across patient groups demonstrates an approach that could be nationally scalable.

Need for Data

In the United States, Black women are 3.15 times as likely to die from pregnancy-related causes as their White counterparts,1 and Black newborns are more than twice as likely to die as White infants.2 These racial disparities persist even when controlling for maternal income2 and education level.1,2 This kind of inequity underscores how both direct racism and unconscious racial bias create dangerous—even deadly—variation in health service delivery. As a Black American, my own life experiences validate how racism and unconscious racial bias can undermine health.

I was born at home in segregated St Louis, Missouri, in 1958. It was an era when expectant Black mothers were denied a hospital birth unless a cesarean section was required or they were in mortal danger. My mother and I benefitted from an uncomplicated delivery. Other Black women and their babies were not so lucky. I am an anesthesiologist, a specialty that helped reduce anesthesia-related mortality 10-fold from the 1970s to the 1990s3 by studying systems and acting upon outcomes data. Additionally, I hold degrees in biostatistics and economics. I am a numbers-driven guy.

As chief medical officer of one of California’s leading health care organizations, Sutter Health, I had a unique opportunity. Within our research institutes, experienced scientists and health data statisticians leveraged our investment in an integrated electronic health record (EHR) covering 3.5 million patients to explore how data could be used to help mitigate the persistence of racial inequity in health outcomes. Three years ago, Sutter Health created an Advancing Health Equity team, which I lead, to study and address health inequity with a data-driven, scientific approach.

Measuring Inequity

Although there are state-level data in California on disease prevalence, outcomes, and mortality rates by race, these data are not as detailed as EHR data. The Advancing Health Equity team wanted more than geographically based metrics, such as the social vulnerability index or the area deprivation index. Thus, the first step was to create a novel metric—a health equity index (HEI)—to identify and quantify inequity within health care organizations and develop targeted interventions to enhance equity.4 Sutter Health’s HEI is the first health equity metric to be implemented that uses real-time EHR data combined with external demographic, prevalence, and utilization statistics to identify inequitable outcomes, reveal their underlying causes, and help illuminate interventions. Applied initially to ambulatory care-sensitive conditions, the HEI has been deployed and studied at Sutter Health for 3 years. The HEI allows the Advancing Health Equity team to implement tailored interventions and measure their efficacy, and it informs the next steps and strategy.

The team is under no illusions that measuring health outcomes inequity will of itself eliminate negative health consequences of racism. Racism in health care and in our larger society has a long history that cannot be undone overnight. However, developing quantitative measures and applying data science to health equity presents opportunities to take possibly the most important next steps since hospital desegregation in 1966.

One of many challenges organizations face is identifying racism in its many forms. Sutter Health’s Advancing Health Equity Team identified at least 3 forms of racism that contribute to health care inequity. First, social racism can define some patients’ lived experiences and lead to a lifetime of stress, negative interactions, and limited access to quality health care services. For example, social racism can be a contributing factor in Black women’s health risks during pregnancy, such as high blood pressure and premature delivery.5 A second form of racism is clinician bias, which can be unconscious or conscious. For example, research on prescribing practices at Sutter Health revealed unexplained differences by race in physicians’ pain relief prescriptions for patients with long-bone fractures. Minority patients were prescribed less potent opioids on average than non-Hispanic White patients, potentially reflecting bias.6 Third, systemic racism resulting from deeply entrenched historically situated patterns of thinking and acting is often least apparent. Bias in computer algorithms, for example, can influence clinical or research decision making in ways that can disproportionately negatively affect people of color.

Bias in computer algorithms can influence clinical or research decision making. 

In 2019, the Advancing Health Equity Team conducted a thorough self-analysis of 18 quality measures, stratified by race and ethnicity, across the care continuum to identify and quantify inequity throughout Sutter Health and published its findings.7 By reporting publicly, Sutter Health accepts accountability for these data and models for others the importance of knowledge sharing to promote transparency and equity. Sutter Health’s goal is to motivate creation of a national community of organizations and clinicians who acknowledge the existence of racism and its roles in patients’ and communities’ health.

Scaling to Promote Equity Nationally

Sutter Health shared the HEI with other health systems to promote collaboration, validate the tool’s usefulness on a broader scale, and generate best practice models that can be implemented nationally. In April 2019, Sutter Health partnered with Salinas Valley Memorial Healthcare System in Salinas, California, to apply the HEI to its patient population. The HEI identified that, in Salinas, Hispanic people ages 20 to 44 have higher-than-expected emergency department (ED) utilization rates for diabetes. This finding, which improved understanding of the impact of social racism in the community, enabled Salinas to target outreach efforts to hire bilingual-certified diabetic educators and improve initial diabetic education referrals for newly diagnosed (typically younger) patients at higher risk of showing up in the ED. Salinas is studying the HEI over time to see whether opening a diabetes clinic staffed with bilingual educators influences ED utilization. In this case, the HEI provided insight into inequity experienced by community members and informed decisions about limited resource expenditure.

A newly created Sutter Health Institute for Advancing Health Equity is working to deploy the HEI in a cohort of organizations across the state to study the HEI’s general applicability and provide a precise geographic and demographic model of health inequity across California. More widespread use of the index will help identify opportunities for collaboration among provider organizations to motivate health equity in American communities.

Inequity and Social Racism

Asthma. After studying Sutter Health’s own EHR data, as well as state data, the Advancing Health Equity team identified Black American patients as having disproportionately higher rates of emergency department (ED) visits for asthma. Using the HEI, the team identified and targeted specific regions or demographic subgroups with large numbers of patients experiencing the greatest outcomes inequity. In 2017, Sutter Health’s study found that 72% of Black patients drove up to 8 miles to a hospital ED to access care, even though they lived within a mile of a primary care clinic. Interviews and surveys revealed that they did so due to a lack of culturally competent primary care, so Sutter Health designed a program to address this specific problem. Today, the Sutter Health Advancing Health Equity Adult Asthma Program provides Black patients in East Oakland and Berkeley race concordant, culturally competent care, with group classes, home visits, virtual access, and tools for medication and disease management (eg, stoplight tools, peak-flow measurements, and action plans). Nearly 600 Black American patients suffering from asthma have participated in the asthma program and very few have returned to the ED.8

COVID-19. The recent COVID-19 pandemic also demonstrated that the asthma program, by accounting for racism, can promote trusted outreach, education, and treatment during a public health crisis.9 As the pandemic spreads throughout the United States, racial and ethnic minorities and socioeconomically disadvantaged groups bear a disproportionate burden of illness and death. In California, Black Americans compose about 6% of the state’s population but, as of May 12, 2020, contribute 10.3% of COVID-19 deaths where race and ethnicity are documented.10 To better understand COVID-19’s impact on Sutter Health patients and to develop solutions, Sutter Health’s Advancing Health Equity team performed data analyses that revealed that Black patients are nearly 3 times more likely to be hospitalized for COVID-19 than non-Hispanic White patients and access care when they are sicker and more likely to require hospitalization and intensive care.11 As a result, Sutter Health is working with a community-based organization to enhance SARS-CoV-2 testing in the Black community.

Inequity and Clinician Bias

In addition to studying social racism, the Advancing Health Equity team studied clinician bias.  Clinician bias as it relates to maternal outcomes has received national attention.12,13,14 The team examined Sutter Health’s rates of nulliparous term singleton vertex births by cesarean section. Maintaining appropriately low rates of cesarean deliveries is a nationally recognized measure of obstetrical quality.15 Sutter Health network facilities deliver approximately 32 000 babies per year, allowing for collection and analysis of a large sample of data over a multiyear period. As a member of the California Maternal Quality Care Collaborative, Sutter Health has developed and implemented programs to ensure that it has among the lowest cesarean delivery rates in California for mothers of all races and ethnicities.7 Sutter Health learned that racial discordance between patients and staff is one factor that increases the likelihood of cesarean delivery.16 Because staff diversity and increased awareness of and training in bias is key to reducing cesarean delivery rates, in August 2020, Sutter Health began training obstetric and gynecologic teams to cultivate awareness of bias and promote behavior change.

Inequity and Structural Racism

Racism is not just the expression of bias that we humans display toward each other. It is built into the policy and practice structures of care and service delivery. For example, as we build clinical decision support and artificial intelligence (AI) tools into EHRs, we must not unwittingly build bias into algorithms by using incomplete or biased source data. Although AI holds tremendous potential for improving health outcomes, algorithmic bias amplifies inequity: some widely used health care risk assessment algorithms express anti-Black bias, in particular.17 Sutter Health has emphasized studying and addressing bias in AI, starting with review of palliative care, sepsis, and hospital readmission risk algorithms in its EHRs. Even if I, as a clinician, can perfectly reign in my biases, I might still provide care that perpetuates systemic racism if I rely on decision support systems that embed bias. Systemic racism can be harder to identify and rectify than social or clinician bias because it can be easily masked by the presumed objectivity of data and algorithms.

Data for Equity Now

All forms of bias require vigilance, especially when we care for vulnerable ill or injured people. George Floyd’s killing has pushed our country to a tipping point. Across the nation, conversations about systemic racial injustice—which for too long were stigmatized and sidelined—are now more central in public discourse. We are at a turning point in the history of heath care: big data and advanced analytic capacity enable precision approaches to improving health outcomes for everyone. Sutter Health’s HEI is a first-generation attempt to use data to measure, report, and respond to health inequity. As noted by the Institute of Medicine,18 health equity is a measure of quality. If Black mothers like mine are not enjoying the best outcomes on par with other mothers, organizations have work to do.


  1. Petersen EE, Davis NL, Goodman D, et al. Racial/ethnic disparities in pregnancy-related deaths—United States, 2007-2016. MMWR Morb Mortal Wkly Rep. 2019;68(35):762-765.
  2. Matthew DB, Rodrigue E, Reeves RV. Time for justice: tackling race inequalities in health and housing. Brookings. October 19, 2016. Accessed July 14, 2020.

  3. Li G, Warner M, Lang BH, Huang L, Sun LS. Epidemiology of anesthesia-related mortality in the United States, 1999-2005. Anesthesiology. 2009;110(4):759-765.
  4. Pressman A, Lockhart S, Petersen J, Robinson S, Moreno M, Azar KMJ. Measuring health equity for ambulatory care sensitive conditions in a large integrated health care system: the development of an index. Health Equity. 2019;3(1):92-98.
  5. Latendresse G. The interaction between chronic stress and pregnancy: preterm birth from a biobehavioral perspective. J Midwifery Womens Health. 2009;54(1):8-17.
  6. Romanelli RJ, Shen Z, Szwerinski N, Scott A, Lockhart S, Pressman AR. Racial and ethnic disparities in opioid prescribing for long bone fractures at discharge from the emergency department: a cross-sectional analysis of 22 centers from a health care delivery system in Northern California. Ann Emerg Med. 2019;74(5):622-631.
  7. Sutter Health. Advancing Health Equity in an Integrated Healthcare Network. Sutter Health; 2019.

  8. Advancing Health Equity Adult Asthma Program Database. Sutter Health; 2019.

  9. Eligon J. For urban poor, the coronavirus complicates existing health risks. New York Times. March 7, 2020. Accessed July 14, 2020.

  10. California Department of Public Health. COVID-19 race and ethnicity data. Updated May 12, 2020. Accessed November 24, 2020.

  11. Azar KMJ, Shen Z, Romanelli RJ, et al. Disparities in outcomes among COVID-19 patients in a large health care system in California. Health Aff (Millwood). 2020;39(7):1253-1262.
  12. Committee on Health Care for Underserved Women. ACOG Committee Opinion No. 649: racial and ethnic disparities in obstetrics and gynecology. Obstet Gynecol. 2015;126(6):e130-e134.
  13. California Dignity in Pregnancy and Childbirth Act, S 464, 2019-2020 Session (Ca 2019). Accessed February 17, 2021.

  14. Sen. Harris introduces bill aimed at reducing racial disparities in maternal mortality. News release. Office of Senator Kamala D. Harris; August 22, 2018. Accessed November 24, 2020.

  15. Vadnais MA, Hacker MR, Shah NT, et al. Quality improvement initiatives lead to reduction in nulliparous term singleton vertex cesarean delivery rate. Jt Comm J Qual Patient Saf. 2017;43(2):53-61.
  16. Bridges CJ. Maternal and Infant Health in the United States: An Investigation of Pregnancy Outcomes. Bachelor’s thesis. Princeton University; 2019.

  17. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453.
  18. Smedley BD, Stith AY, Nelson AR, eds; Institute of Medicine. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. National Academies Press; 2003.


AMA J Ethics. 2021;23(3):E252-257.




The author gratefully acknowledges the contributions of Kelly Smits, MS.

Conflict of Interest Disclosure

The author(s) had no conflicts of interest to disclose.

The viewpoints expressed in this article are those of the author(s) and do not necessarily reflect the views and policies of the AMA.