Medical Education
Dec 2021

Three Things Schools Should Do to Make Advancement Assessment Just

Marjorie Westervelt, PhD, MPH, Darius Billingsley, MD, Maya London, and Tonya Fancher, MD, MPH
AMA J Ethics. 2021;23(12):E937-945. doi: 10.1001/amajethics.2021.937.


This article considers how student advancement assessment in American medical schools undermines equity. Although much attention is paid to admissions processes’ capacity to diversify the physician workforce, students’ advancement has been neglected as the next key step along their journeys toward graduation and residency training. This article canvasses common ways advancement undermines equity and suggests 3 areas of focus. In particular, it suggests that retention, student progression, and career advancement milestones are at least as important as admissions-based efforts to promote justice in medical education opportunity.


In Regents of the University of California v Bakke (1978), the US Supreme Court acknowledged the educational benefits of diversity, writing that while racial quotas were unconstitutional, race-conscious admissions policies were legal if race was one of many factors considered.1 Over 40 years later, we—affiliates of the medical school whose rejection of Bakke’s application spurred the case—revisit the concept of diversifying the physician workforce and ensuring that medical education systems’ support of diverse learners does not end with recruitment but continues throughout the educational continuum.2

Defining and Measuring Diversity

Medical schools’ social missions are measured by 3 indicators: the percentage of graduates practicing primary care, the percentage of graduates practicing in health professional shortage areas, and the percentage of graduates from backgrounds underrepresented in medicine (URiM).3 The historically Black colleges and universities dominate the social mission rankings by educating the vast majority of Black physicians in the United States and delivering a curriculum that inspires graduates to practice in locations and specialties with physician shortages.3 The importance of this work is magnified by recent evidence of improved health outcomes when Black patients are cared for by Black physicians.4,5 There is a need for other schools to adopt similar strategies to promote diversity more broadly.

In 2003, the Association of American Medical Colleges (AAMC) broadened its definition of URiM to include any “racial and ethnic populations that are underrepresented in the medical profession relative to their numbers in the general population” to permit schools to define underrepresentation based on their region6 and to include nonracial and ethnic identities, such as sexual orientation, disability,7 rural origin,8 growing up in a low-income household, and first generation to attend college.9 Prior to 2003, underrepresented minority was the term the AAMC used to refer to “Blacks, Mexican-Americans, Native Americans (that is, American Indians, Alaska Natives, and Native Hawaiians), and mainland Puerto Ricans.”6 The shift from “underrepresented minority” to URiM helped medical schools to think more broadly and more regionally about workforce diversity.

How progress in diversity is measured is shaped by the mission of an institution and its ideal student population. In interviews with medical school admissions officers across the United States, Ko et al (unpublished data, 2019-2020) found that they take advantage of the local approach by creating their own definitions of diversity with easy-to-meet thresholds. For example, institutions can use the demographics of their surrounding community or state as diversity benchmarks. Thus, in states with less racial and ethnic diversity, the target number of URiM matriculants will be lower. However, if national—or even global—population data were used, the benchmark for diversity at any given institution would be higher.

Diversity benchmarks can be used to promote not only parity10,11,12,13 or equality, whereby everyone receives the same thing regardless of their background, but also a culture of equity, which ensures that all students receive what they need to be successful and that all aspects of medical education are just. A culture of equity, however, cannot be limited to recruitment, admissions, and selection but must encompass curriculum, assessment, and career advancement milestones. Much of the focus in diversifying the physician workforce is on pathway programs to expand the applicant pool14,15,16 and admissions processes such as multiple mini-interviews,17 trainings to mitigate bias,18,19 and holistic review20 based on applicants’ experiences, attributes, and metrics. Less research focuses on how to ensure a supportive and equitable learning environment for learners once they matriculate. We contend that progress in equity must be measured in terms of not only recruitment to medical school but also success in medical school and beyond.

Equity in Assessment

Although holistic admissions practices have greatly increased the number of students from minoritized groups attending medical school,21 the medical education system is lagging in developing and implementing strategies that ensure student success. This lag is not due to lack of motivation or intention; in fact, medical schools across the country have invested significant resources in providing academic support to students from minoritized groups who tend to matriculate with lower metrics (eg, grade-point average, Medical College Admission Test® [MCAT]  scores).22 However, it may be unreasonable to expect the same performance from students with diverse educational opportunities and experiences. Moving away from traditional performance expectations (which were set in place by historically dominant groups) and redefining success is the only way to achieve equity in assessment in medical education.

How progress in diversity is measured is shaped by the mission of an institution and its ideal student population.

Lucey et al classified equity in medical assessment as a “wicked problem,” that is, a problem that is immeasurably complex and nearly impossible to solve.23 The authors identified 3 requirements for equity in assessment: intrinsic equity (unbiased assessments), contextual equity (a fair learning environment), and instrumental equity (the use of assessment data in advancement and selection).23 Layering these equity requirements on a competency-based medical education (CBME) framework provides an opportunity to create a system for equitable advancement and progression. Per the first requirement, assessments in a medical school employing a CBME framework would use criterion-based measures of performance, which compare performance to a predetermined standard or performance level and provide students as much time as necessary to achieve them. The decision of the United States Medical Licensing Examination® (USMLE) to move from a numeric to a pass/fail scoring system for Step 1, effective no earlier than January 1, 2022, is a timely example of aligning assessment with the principles of CBME.24 This change might reduce the USMLE’s negative effects (eg, isolation, anxiety, and the misuse of Step 1 scores for residency selection or measurement of competence) on URiM students.25

Standardized assessments (eg, USMLE Step 1 and Step 2 Clinical Knowledge) are one, albeit a predominant, traditional metric of success in undergraduate medical education (UME). Other “measures” of success include clerkship grading and clinical skills assessment—both of which can be extremely subjective and biased—and matching into a residency program. In a multi-institutional study of over 600 medical students, Bullock et al found that only 44% of medical students believed clerkship grading to be fair.26 Taking a different approach, Teherani et al conducted semi-structured interviews with 20 senior medical students and residents to identify what they perceived as equitable assessment practices.27 The analysis identified a number of possible improvements related to clinical assessment, including, but not limited to, shifting the focus from grades to patient care and removing peer comparisons. These findings once again reinforce the benefits of moving to a CBME model: focusing on the ultimate goal of the educational experience (ie, safe and effective patient care) and using criterion-based rather than normative standards. In response to this feedback, some schools decided to move to a pass/fail grading system for clerkships. However, as of the 2019-2020 academic year, only 11 of 153 schools had adopted this strategy.28 A major challenge to widespread adoption of CBME is residency selection: most residency programs rely on clerkship grades to identify medical students worthy of consideration for their specialty. Without clerkship grades and USMLE Step 1 scores, residency programs would need to find a more holistic way to review applicants, which might improve the representation of URiM residents (and eventually faculty) across all specialties.29,30,31

Standardized exams and residency selection are part of the larger medical education system that must be examined in order to achieve equity. Some students from URiM groups enter medical school having had less access to academic preparation and having underperformed on standardized tests due to the tests’ inherent biases.32,33,34 This inequity can be traced throughout the K-12 and undergraduate education systems. Expecting students to make up 16 years of disadvantages in 2 years (or less in some medical schools) while learning and retaining all the new knowledge presented to them in the preclerkship curriculum is unrealistic. However, students who may (understandably) need more time face extreme scrutiny by student progress committees through repeated reviews of their academic progress and the need to justify delays, and viewing this extra time as a delay or falling behind can have a negative impact on their well-being.35 Many URiM students also have fewer financial resources upon matriculation and leave medical school with more debt than their peers from higher socioeconomic backgrounds.36,37 A more equitable CBME system would have a “flat rate” for the MD degree, allowing students as much time and resources as necessary to move through the curriculum. In this system, success would be defined as every student finishing medical school, regardless of the amount of time it takes.

Where Can We Go?

As gatekeepers to the profession, US medical schools should embrace their role in creating an equitable medical education system and in driving the representiveness and diversity of the workforce30,38,39,40,41,42 that will address health needs around the globe, following the examples below.

Recruitment with retention. While several US medical schools (as well as graduate medical education [GME] programs29,43) have implemented holistic review and multiple mini-interviews to recruit a diverse student body,44,45,46,47 some have also added GME to UME admissions, reenvisioning admission to medical school as admission to UME and to GME. For example, Oregon Health & Science University (OHSU) and the University of California, Davis (UC Davis), with grant support from an American Medical Association Accelerating Change in Medical Education initiative, established a collaborative known as COMPADRE (California Oregon Medical Partnership to Address Disparities in Rural Education and Health). OHSU and UC Davis, along with regional residency programs, co-recruit and train the physicians needed in the rural, tribal, and urban communities residing between Portland, Oregon, and Sacramento, California.48 Other organizations have approached UME/GME joint recruitment through a time-variable approach. For example, the Education in Pediatrics Across the Continuum Project bases advancement on the achievement of competency rather than time-based milestones across the UME-GME continuum.49

Retention and advancement. The 2021 Coalition for Physician Accountability’s 42 recommendations to improve the UME-to-GME transition include a call to action for UME and GME programs to eliminate systemic biases in their grading and awards structures.50 The profession’s collective overreliance on metrics to assess student performance has been dispelled by the Morehouse School of Medicine. The Step 1 scores of students graduating between 2009 and 2014, who received interventions designed to facilitate success, exceeded those expected based on their MCAT scores.22 The school creates the right milieu for learning as well as mentoring opportunities, aligns the structure and content of its curriculum to its mission, and uses a robust system to monitor student performance and retention. Similarly, the University of Michigan School of Medicine is leading the nation in providing an inclusive environment for students with a physical disability or functional limitation7 and in intentionally aiming to reduce barriers for learners to promote equity in access and education. Although not directly supporting retention, the AAMC’s application to medical school offers prospective students the option of specifying gender identity and preferred pronouns,51 and the Accreditation Council for Graduate Medical Education awards52 includes awards for diversity and inclusion. These are 2 major steps in recognizing the diverse identities of medical learners and in promoting inclusion.

Career advancement. Exemplars of equity in access to medical specialty careers are harder to identify. The Indiana University School of Medicine publishes a diversity dashboard fact sheet— including data for UME students, staff, faculty, GME trainees, and the state of Indiana—which is a step toward accountability (though the current dashboard does not include data by specialty).53 Both Ohio University Heritage College of Osteopathic Medicine and the University of North Carolina (UNC) offer programs that include post-GME retention at the point of entry to medical school. The Heritage College’s Transformative Care Continuum is an accelerated UME-to-GME program that includes a contract with the Cleveland Clinic upon residency completion.54 The UNC Fully Integrated Readiness for Service Training program is a UME-to-GME program that includes 3 years of post-GME service in rural and underserved North Carolina.55


We propose that defining diversification goals at individual institutions demands that those institutions honor regional needs to provide the best care,4,5 advance health equity,56 and optimize the educational benefit for all students.1,57 We encourage schools to embrace recruiting a health care workforce that is diverse with respect to race and ethnicity but also to consider identities such as disability, sexual orientation, socioeconomic status, and first generation to attend college, and the intersection of these identities. Once students have been recruited, schools must support them throughout their education, remove barriers to equitable advancement, encourage them to explore all specialties, and continue to support them as they transition to careers.


  1. Regents of Univ of Cal v Bakke, 438 US 265 (1978).

  2. UC Davis School of Medicine. Class of 2025 matriculant demographics. Accessed October 9, 2021.

  3. Mullan F, Chen C, Petterson S, Kolsky G, Spagnola M. The social mission of medical education: ranking the schools. Ann Intern Med. 2010;152(12):804-811.
  4. Alsan M, Garrick O, Graziani GC. Does diversity matter for health? Experimental evidence from Oakland. National Bureau of Economic Research working paper 24787. June 2018. Accessed October 9, 2021.

  5. Greenwood BN, Hardeman RR, Huang L, Sojourner A. Physician-patient racial concordance and disparities in birthing mortality for newborns. Proc Natl Acad Sci U S A. 2020;117(35):21194-2200.
  6. Underrepresented in medicine definition. Association of American Medical Colleges. Accessed October 9, 2021.

  7. Vloet K. DisAbility. Medicine at Michigan. Winter 2019. Accessed September 7, 2021.

  8. Shipman SA, Wendling A, Jones KC, Kovar-Gough I, Orlowski JM, Phillips J. The decline in rural medical students: a growing gap in geographic diversity threatens the rural physician workforce. Health Aff (Millwood). 2019;38(12):2011-2018.
  9. Page KR, Castillo-Page L, Poll-Hunter N, Garrison G, Wright SM. Assessing the evolving definition of underrepresented minority and its application in academic medicine. Acad Med. 2013;88(1):67-72.
  10. Raj A, Kumra T, Darmstadt GL, Freund KM. Achieving gender and social equality: more than gender parity is needed. Acad Med. 2019;94(11):1658-1664.
  11. Fisler N, Sweitzer BJ, Wurz J, Kleiman AM, Stueber F, Luedi MM. Achieving gender parity in acute care medicine requires a multidimensional perspective and a committed plan of action. Anesth Analg. 2019;129(6):1778-1783.
  12. Ludmerer KM. Seeking parity for women in academic medicine: a historical perspective. Acad Med. 2020;95(10):1485-1487.
  13. Rowell TR, Redd RA, Neuberg DS, Walensky LD. Mind the gap: expediting gender parity in MD-PhD admissions. JCI Insight. 2020;5(4):e136037.

  14. Wheat JR, Leeper JD. Pipeline programs can support reforms in medical education: a cohort study of Alabama’s rural health leaders pipeline to engage community leaders. J Rural Health. 2021;37(4):745-754.
  15. Martos AJ, Piracha YS, Oladele M, Erves JG, Dorn J, Friedman E. An innovative educational pipeline programme for under-represented youth: the Sophie Davis Biomedical Education/CUNY School of Medicine model. Educ Prim Care. 2017;28(5):282-287.
  16. Acosta D, Olsen P. Meeting the needs of regional minority groups: the University of Washington’s programs to increase the American Indian and Alaskan Native physician workforce. Acad Med. 2006;81(10):863-870.
  17. Henderson MC, Kelly CJ, Griffin E, et al. Medical school applicant characteristics associated with performance in multiple mini-interviews versus traditional interviews: a multi-institutional study. Acad Med. 2018;93(7):1029-1034.
  18. Capers Q. How clinicians and educators can mitigate implicit bias in patient care and candidate selection in medical education. ATS Sch. 2020;1(3):211-217.
  19. Capers Q, McDougle L, Clinchot DM. Strategies for achieving diversity through medical school admissions. J Health Care Poor Underserved. 2018;29(1):9-18.
  20. Addams AN, Bletzinger RB, Songheimer HM, White SE, Johnson LM. Roadmap to diversity: integrating holistic review practices into medical school admissions processes. Association of American Medical Colleges; 2010. Accessed September 7, 2021.

  21. Witzburg RA, Sondheimer HM. Holistic review—shaping the medical profession one applicant at a time. N Engl J Med. 2013;368(17):1565-1567.
  22. Elks ML, Herbert-Carter J, Smith M, Klement B, Knight BB, Anachebe NF. Shifting the curve: fostering academic success in a diverse student body. Acad Med. 2018;93(1):66-70.
  23. Lucey CR, Hauer KE, Boatright D, Fernandez A. Medical education’s wicked problem: achieving equity in assessment for medical learners. Acad Med. 2020;95(12)(suppl):S98-S108.
  24. Moving forward: the Step 1 P/F score reporting policy change and revitalization of Step 2 Clinical Skills Examination. United States Medical Licensing Examination®. November 2, 2020. Accessed January 11th, 2021.

  25. McDade W, Vela MB, Sánchez JP. Anticipating the impact of the USMLE Step 1 pass/fail scoring decision on underrepresented-in-medicine students. Acad Med. 2020;95(9):1318-1321.
  26. Bullock JL, Lai CJ, Lockspeiser T, et al. In pursuit of honors: a multi-institutional study of students’ perceptions of clerkship evaluation and grading. Acad Med. 2019;94(11)(suppl):S48-S56.
  27. Teherani A, Perez S, Muller-Juge V, Lupton K, Hauer KE. A narrative study of equity in clinical assessment through the antideficit lens. Acad Med. 2020;95(12)(suppl):S121-S130.
  28. Curriculum reports: grading systems use by US medical schools. Association of American Medical Colleges. Accessed June 2021.

  29. Aibana O, Swails JL, Flores RJ, Love L. Bridging the gap: holistic review to increase diversity in graduate medical education. Acad Med. 2019;94(8):1137-1141.
  30. Nieblas-Bedolla E, Williams JR, Christophers B, Kweon CY, Williams EJ, Jimenez N. Trends in race/ethnicity among applicants and matriculants to US surgical specialties, 2010-2018. JAMA Netw Open. 2020;3(11):e2023509.

  31. McDonald TC, Drake LC, Replogle WH, Graves ML, Brooks JT. Barriers to increasing diversity in orthopaedics: the residency program perspective. JB JS Open Access. 2020;5(2):e0007.

  32. Pershing S, Co JPT, Katznelson L. The new USMLE Step 1 paradigm: an opportunity to cultivate diversity of excellence. Acad Med. 2020;95(9):1325-1328.
  33. Edmond MB, Deschenes JL, Eckler M, Wenzel RP. Racial bias in using USMLE Step 1 scores to grant internal medicine residency interviews. Acad Med. 2001;76(12):1253-1256.
  34. Rubright JD, Jodoin M, Barone MA. Examining demographics, prior academic performance, and United States Medical Licensing Examination scores. Acad Med. 2019;94(3):364-370.
  35. Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student perspectives on the “Step 1 climate” in preclinical medical education. Acad Med. 2019;94(3):302-304.
  36. Greysen SR, Chen C, Mullan F. A history of medical student debt: observations and implications for the future of medical education. Acad Med. 2011;86(7):840-845.
  37. Toretsky C, Mutha S, Coffman J. Reducing educational debt among underrepresented physicians and dentists. UCSF Healthforce Center; August 2019. Accessed September 7, 2021.

  38. Diversity in medicine: facts and figures. Table 13: practice specialty, males by race/ethnicity, 2018. Association of American Medical Colleges. Accessed October 9, 2021.

  39. Diversity in medicine: facts and figures. Table 12: practice specialty, females by race/ethnicity, 2018. Association of American Medical Colleges. Accessed October 9, 2021.

  40. Lett LA, Orji WU, Sebro R. Declining racial and ethnic representation in clinical academic medicine: a longitudinal study of 16 US medical specialties. PLoS One. 2018;13(11):e0207274.

  41. Akhiyat S, Cardwell L, Sokumbi O. Why dermatology is the second least diverse specialty in medicine: how did we get here? Clin Dermatol. 2020;38(3):310-315.

  42. Sanchez AN, Martinez CI, Stampas A, et al. Ethnic and racial diversity in academic physical medicine and rehabilitation compared with all other medical specialties. Am J Phys Med Rehabil. 2021;100(2)(suppl):S12-S16.
  43. Bates T, Mutha S, Coffman J. Practicing holistic review in medical education. UCSF Healthforce Center; July 2020. Accessed October 9, 2021.

  44. Jerant A, Fancher T, Fenton JJ, et al. How medical school applicant race, ethnicity, and socioeconomic status relate to multiple mini-interview-based admissions outcomes: findings from one medical school. Acad Med. 2015;90(12):1667-1674.
  45. Grbic D, Morrison E, Sondheimer HM, Conrad SS, Milem JF. The association between a holistic review in admissions workshop and the diversity of accepted applicants and students matriculating to medical school. Acad Med. 2019;94(3):396-403.
  46. Harrison LE. Using holistic review to form a diverse interview pool for selection to medical school. Proc (Bayl Univ Med Cent). 2019;32(2):218-221.
  47. Grabowski CJ. Impact of holistic review on student interview pool diversity. Adv Health Sci Educ Theory Pract. 2018;23(3):487-498.
  48. COMPADRE: five-year, $1.8M award aims to transform health care in rural California and Oregon. UC Davis Office of Research. June 5, 2019. Accessed October 9, 2021.

  49. Andrews JS, Bale JF Jr, Soep JB, et al; EPAC Study Group. Education in pediatrics across the continuum (EPAC): first steps toward realizing the dream of competency-based education. Acad Med. 2018;93(3):414-420.

  50. Coalition for Physician Accountability. Initial summary report and preliminary recommendations of the Undergraduate Medical Education to Graduate Medical Education Review Commmittee (UGRC). April 2021. Accessed October 9, 2021.

  51. Association of American Medical Colleges. 2020 AMCAS® applicant guide. 2019. Accessed October 9, 2021.

  52. Barbara Ross-Lee, DO Diversity, Equity, and Inclusion Award. Accreditation Council for Graduate Medical Education. Accessed June 14, 2021.

  53. DEI dashboard. School of Education, Indiana University Bloomington. Accessed June 14, 2021.

  54. Transformative Care Continuum. Heritage College of Osteopathic Medicine, Ohio University. Accessed June 14, 2021.

  55. Coe CL, Baker HM, Byerley JS, Page CP. Fully Integrated Readiness for Service Training (FIRST): an accelerated medical training program for rural and underserved North Carolina.  Acad Med. 2021;96(10):1436-1440.
  56. Marrast LM, Zallman L, Woolhandler S, Bor DH, McCormick D. Minority physicians’ role in the care of underserved patients: diversifying the physician workforce may be key in addressing health disparities. JAMA Intern Med. 2014;174(2):289-291.
  57. Page S. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton University Press; 2007.


AMA J Ethics. 2021;23(12):E937-945.




This work was supported by grant UH1HP29965 from the Health Resources and Services Administration of the US Department of Health and Human Services (Dr Fancher) and with additional funding from an American Medical Association Accelerating Change in Medical Education Innovation grant (Dr Fancher).

Conflict of Interest Disclosure

The author(s) had no conflicts of interest to disclose. 

This article is the sole responsibility of the author(s) and does not necessarily represent the views of the Health Resources and Services Administration, the Department of Health and Human Services, or the US government. The viewpoints expressed in this article are those of the author(s) and do not necessarily reflect the views and policies of the AMA.