AMA Journal of Ethics^®

Abstract

Approaches to responding to racial and ethnic health inequity in the United States have had limited impact over the past 40 years. Efforts to increase the number of medical students of color are undermined by hyperfocus and overreliance on and misinterpretation and misuse of standardized examination scores. Structural racism and persistence of deficit-focused interventions undermine appreciation of the value that students and physicians with minoritized identities bring to medicine and to US health care’s systemic capacity to motivate equity.

Diversity Motivates Equity

Although the US health care system is characterized by high technology and high resource investment, the United States has the lowest life expectancy and highest infant mortality of 11 of the highest-income nations.¹ Health inequity contributes to this lag, and increasing the number of physicians in training from groups underrepresented in medicine (URiM), such as African Americans/Blacks, Latinos, and Native Americans, is key to promoting health equity.^2,3,4 Our nation cannot possibly achieve its potential without responding to health needs in underserved communities, addressing social and political determinants of health (eg, structural racism), and increasing physician workforce diversity.

Over the past 40 years, several initiatives have been undertaken to increase the number and proportion of URiM physicians (and physicians in training), including—but not limited to—targeted recruitment, summer support programs, general academic support, and holistic admissions processes.⁵ In this article, we will focus on the limitations posed by the current approach to recruiting, training, and assessing medical students and physicians based on overreliance on and misinterpretation of standardized tests and a corresponding lack of appreciation for the importance of diverse views in addressing the health of our nation.

Overreliance on Testing

Medical students and physicians in training must master a constantly expanding knowledge base in order to be effective practitioners. Theoretically, standardized examinations, such as the Scholastic Aptitude Test^® (SAT) and the Medical College Admissions Test^® (MCAT), are intended to assess students’ capacity and readiness for the next stage of education. The MCAT, in particular, aims to test both knowledge and critical thinking, which many have identified as foundational to medical education success.⁶ In reality, however, standardized tests’ results perpetuate inequity and bias. Although standardized testing was developed to expand the applicant pool beyond the financially privileged,^7,8 family income remains the best predictor of SAT scores.⁹ Income and privilege lead to better access to education, guidance, and preparation materials and also afford more study time, all of which are key to higher scores on standardized tests. The impact of income disparities on test scores highlights that standardized tests are one feature of structural racism that systematically disadvantages minoritized students and students with low income. Academic performance metrics (eg, scores on standardized tests) have been recognized as barriers to the recruitment, matriculation, and progress of URiM learners. Data from the Association of American Medical Colleges (AAMC), for example, show that average MCAT scores for URiM applicants and matriculants during the academic year 2020-2021 were lower than those from groups already well represented in medicine.¹⁰

It seems foolhardy for any medical school admissions committee to overemphasize MCAT performance.

The use of successive tests to predict performance—the SAT or American College Tests for college, the MCAT for medical school, the United States Medical Licensing Examination^® (USMLE) Step 1 and Step 2 clinical knowledge (CK) for residency and Steps 1 to 3 for licensure, and written subspecialty exams for board certification—could very well be misguided. Careful reviews of revisions to the MCAT, for example, demonstrate that MCAT scores are the single best predictor of performance on subsequent standardized tests, especially Step 1,^11,12,13,14 and that Step 1 scores, in turn, are the single best predictor of performance on subsequent examinations, including Step 2 CK, Step 3, residency in-service examinations, and board certification examinations.¹³ Nevertheless, in these studies, in which correlations between scores on the tests range from 0.5 to 0.6, any one predictor accounts for only about 25% to 30% of the variance in scores on another test. Test scores are thus globally but not precisely predictive of success in medicine, and overemphasizing their importance is unjust. Data from studies of MCAT scores are revealing. While first-time fails on Step 1 are rare for those with higher MCAT scores,¹³ the majority of Step 1 test takers with MCAT scores in the 9th percentile or above pass.¹³ Moreover, as discussed later, MCAT scores do not predict physician excellence.

Better Clinicians Is the Goal, Not Higher Scores

Tests have an unintended side effect of diverting energies of learners and faculty into “chasing the numbers” rather than focusing on foundational concepts and competencies needed by practicing physicians. Mounting evidence suggests that competitive entry into US medical schools prompts a “game of scores” in which information is learned merely to earn scores that open up the next opportunity, not because it is germane to the goal of serving patients well. As the number of medical school applicants per seat has grown, the average entering student MCAT score has increased.¹² Yet 42% of medical school applicants from 2018 to 2020 had MCAT scores and grade point averages high enough to be accepted into medical school.¹³

Since establishment of the USMLE licensure sequence 30 years ago, there has been intense inflationary pressure on examinees’ scores. Initially, the mean of each step was set at 200, with a standard deviation of 20; a minimum passing score was 167. Over time, passing scores for all parts of the examination have been periodically reassessed, with the current minimum passing score for Step 2 CK being 209, which exceeds the mean set 30 years ago (see Figure). This inflationary trend means there are physicians practicing today whose examination score is no longer regarded as a passing score. Ideally, a score on a high-stakes test should be meaningful as a measure of mastery of knowledge and not in relation to scores of other examinees.

Figure. USMLE Step 1 and Step 2 CK Mean and Minimum Passing Scores, 1993-2017^a

^a Data from US Medical Licensing Examination, 1993-2017.

In his book, Are We Getting Smarter? Rising IQ in the Twenty-First Century,⁸ James Flynn argues that we are getting better at “teaching to the test,” with the result that average scores on standardized tests of abstract problem-solving skills are rising without a corresponding increase in intrinsic intelligence. Some medical schools, recognizing the impact that cultural and educational environments have on scores on standardized tests, have instituted reforms that provide a cautionary lesson about underestimating students’ readiness or capacity for learning or for assuming professional responsibility on the basis of test scores. For example, after the Morehouse School of Medicine instituted an inclusive and supportive learning environment, the range of 2009-2014 graduates’ USMLE Step 1 scores “shift[ed] a full standard deviation compared with the predicted range based on their MCAT scores.”¹⁵ This finding challenges the view of MCAT scores as a fixed and linear predictor of academic performance. Because many minoritized students lack supportive learning environments, structural racism could explain the observed differences between mean scores of minoritized (excluding Asian) applicants and White applicants.¹³

It is difficult to measure the relationship between standardized test scores and physician quality (however that is defined) due in part to the lack of metrics for physician quality and the fact that some desirable outcomes, such as specialty choice and site of practice, are not predicted by available metrics. As a result, medical schools are excluding some applicants on the basis of performance on exams that has not been shown to be related to desired final outcomes—such as rural practice or primary care.¹⁶ Thus it seems foolhardy for any medical school admissions committee to overemphasize MCAT performance.

Addressing Underrepresentation

In 40 years of only partial success in increasing the number of URiM students and physicians, many recruitment and retention programs have focused on narrowing score gaps rather than on recognizing and valuing the insights, experiences, wisdom, and skills that physicians with minoritized identities bring to medicine. In its blindness to structural racism, American academic medicine has failed to identify, value, and incorporate the knowledge, perspectives, and ways of being in the world by a sufficiently diverse array of people who have much to offer professions and patients. Interventions to increase representation of URiM groups have not included a broad-based understanding of characteristics that not only position medical students for success in school but also contribute to good health outcomes. We know how to address structural racism¹⁷; certainly, awareness and recognition of it are key first steps. Holistic admissions processes are important.^18,19,20Addressing structural racism at multiple levels—including eliminating barriers to educational achievement throughout the educational pipeline and shifting attention from “deficits” to recognizing each learner’s gifts and strengths—is vital. We also must expand the set of tools we use to assess physician service and performance in responding to individuals’ and communities’ health needs.