AMA Journal of Ethics^®

Professional competence for physicians, as defined by Epstein and Hundert [1], is:

the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served [2].

As implied by their definition, noncognitive traits figure prominently in Epstein and Hundert’s discussion of physicians’ professional competence. They cite attributes such as respect for patients, caring, emotional intelligence, teamwork, tolerance of ambiguity and anxiety, and basic communication skills as fundamental components of professional competence.

Although the importance of characteristics like those mentioned above is clear, it is quite difficult to assess the extent to which individuals possess them. Traits and skills related to providing humane medical care such as “caring” and “emotional intelligence” are much easier to recognize in practice than they are to explicitly define and measure. The crux of the problem is that traits and skills that develop over time through personal experience (i.e., learning-by-doing) in various social contexts can be difficult to express in words. They are often described as “tacit knowledge,” i.e., knowledge and skills that enable one to perform certain tasks without necessarily fully knowing, or being able to explain, how one does it. Lam notes that in contrast to the type of knowledge associated with cognitive skills (explicit knowledge), tacit knowledge is personal and contextual [3]. Consequently, it is difficult to articulate, formalize, and share with others.

Although physicians’ tacit knowledge enables them to recognize similar competence in others, there are two major drawbacks to overreliance upon tacit knowledge as the basis for admission and evaluation processes. First, physicians’ tacit knowledge reflects their personal, perhaps idiosyncratic, understandings of the essential noncognitive traits and skills. These views may vary considerably among faculty. To be used effectively in admission and evaluation processes, these views would need to be synthesized and articulated as a shared vision of the medical school community. Secondly, because tacit knowledge develops through social interaction over time, it most likely contains outmoded beliefs and biases that hamper objective evaluation of others. A recent AAMC literature review described evidence of erroneous tacit knowledge in the form of unconscious gender or race/ethnicity bias [4]. The review cited several studies showing that evaluators’ awareness of gender or race/ethnicity caused them to mistakenly favor one equally qualified candidate over another. Thus, unchecked reliance upon tacit knowledge can result in biased recruitment and evaluation decisions.

Therefore, the central challenge in evaluating noncognitive traits is to leverage the useful portions of physicians’ tacit knowledge into a common understanding of the most essential traits, while at the same time minimizing the influence of personal biases and irrelevant or mistaken information. Ultimately, if we are to select and develop physicians’ capacity for requisite noncognitive skills and traits, we need reliable, valid, and transparent methods for measuring them.

In the following section, we outline strategies for refining organizations’ processes to define and assess crucial noncognitive attributes. Effective use of research methods and data to move toward more explicit understanding of the desired characteristics and valid assessment of them is the guiding principle for this approach. The four steps are intended as elements of an iterative cycle to continuously improve processes for evaluating noncognitive attributes.

Improving Assessment of Noncognitive Attributes

1. Develop more explicit definitions of the desired skills and attributes. Oftentimes the daunting task of developing precise definitions of learning outcomes is addressed by committees or task forces. To support such work, preliminary qualitative research methods (c.f., Denzin and Lincoln [5], Giorgi [6]) can be used to describe and analyze the tacit knowledge available among medical school personnel regarding their interpretations and understandings of the noncognitive traits needed for professional competence and how these skills and attributes can be recognized in practice. 

   Foundational research, like the above, can guide committees’ deliberations as they seek to identify the most essential noncognitive traits and explore practical means for assessing them. Without locally developed research, members may struggle to articulate their tacit knowledge and get frustrated by the size and difficulty of their task. Under such conditions, committee members face the temptation of settling for the most easily defined and measurable traits rather than struggling to express and define the most essential ones. Research and conceptualizations can support efforts to make explicit their understanding (i.e., the “externalization” of tacit knowledge) of the desired attributes.

2. Structure data collection to observe instances of the desired traits. With a more explicit understanding of the desired noncognitive attributes, one can fine-tune the methods used to assess them. Assessment processes can be refined to elicit more revealing and relevant performances. Behavior-based interviewing [7] and the Multiple Mini-Interview [8] ( for example, are valuable approaches to consider for gathering more useful assessment information from applicants.
3. Train raters/evaluators to use the system. Better tools for assessing the desired traits will only improve outcomes if evaluators are trained and have the opportunity and resources to use those tools properly. Martell’s research provides evidence of the importance of sufficient time, information, structure, and training for reducing the use of irrelevant information, including stereotypes and bias in evaluations [9]. Like most skills, practice and experience also seem to improve the quality of evaluations [10].
4. Provide feedback to evaluators/raters. When aided by thoughtful reflection and feedback on the accuracy of their previous decisions, one would expect that evaluators would reap even greater benefits from their experience. Toward this end, the Implicit Association Test [11] is one example of a resource that can help individuals identify sources of unconscious bias affecting their evaluations.

Although there are numerous types of feedback that could be provided to evaluators, here we describe an analysis that provided evaluators feedback on bias in their evaluations.

In a recent study at the University of Iowa, we analyzed 5 years of clinical performance evaluation forms for evidence of unconscious gender bias in the ratings of our medical students [12]. Our method involved examining whether the meaning of adjectives was affected by the gender of the student being rated. Within a factor analysis framework, highly intercorrelated groups of adjectives are interpreted as having a similar meaning; the common meaning for a given adjective grouping is represented by an underlying factor. If raters use the same meaning of the adjective regardless of the student’s gender, then the expected pattern of intercorrelations and underlying factors among adjectives would be the same for men and women students. This hypothesis was tested statistically using Multigroup Confirmatory Factor Analysis (CFA). (See Brown [13] for an accessible description of this technique.)

From this analysis, we found that raters did, in fact, interpret the adjectives (i.e., “measurement models”) differently based on the gender of the student being rated. These different measurement models resulted in gender-biased evaluations. Women were given more credit than comparable men for being “compassionate,” “sensitive,” and “enthusiastic,” and men were given more credit than comparable women for being “quick learners.” Thus, this type of analysis enabled us to raise evaluators’ awareness of an unconscious bias evident in the pattern of their ratings.

In sum, physicians’ tacit knowledge of vital noncognitive attributes provides invaluable raw data for developing, implementing, and refining assessment processes. As outlined in the steps above, qualitative and quantitative research methods can facilitate efforts to externalize tacit knowledge, improve measurement processes, and correct implicit biases in judgments based upon tacit knowledge. Ultimately, however, it is physicians’ reflective and judicious use of such research that will enable them to create increasingly meaningful and accurate processes for assessing noncognitive attributes.