AMA Journal of Ethics^®

Imagine a defense attorney who, faced with an eyewitness who claims to have seen the defendant at the scene of the crime, wants to demonstrate that the eyewitness’s memory is false, the product of flawed recall. A company offers to put the witness into a brain scanner, ask her to recall the memory in question, and judge the likely fidelity of the memory based on the parts of the brain brought to the task. The eyewitness is scanned, and the company reports that the brain activity pattern of the witness looked very similar to the brain activity pattern associated with the false memory conditions in the neuroscience literature.

Should the scan be allowed as evidence in court, and, if not, what criteria must be met before it should be? To answer this question, we first shall explore functional magnetic resonance imaging (fMRI) as a specific form of general functional neuroimaging methods.

Functional neuroimaging describes a category of noninvasive methods by which brain activity may be recorded [1]. The term “imaging” is something of a misnomer, since the techniques by which these methods derive data entail mostly collecting and analyzing information on brain activity in the dimensions of space and time; the final step of translating this information into an image is not necessary, though it often helps in comprehension of the data. On the other hand, as we will see, presenting the data in the visual format is often misleading for those unversed in its genesis [2].

fMRI in the Experimental Setting

fMRI measures changes in the levels of oxygenated blood delivered to areas of the brain, with the assumption that more neural activity in an area of the brain creates the subsequent need for more oxygenated blood [3]. The change is referred to as the blood oxygen level dependent (or BOLD) signal in the brain. If an increase in the BOLD signal correlates with a certain cognitive process, researchers may hypothesize that the part of the brain where the BOLD signal increased is involved in that cognitive process.

Numerous mental states unrelated to the task of interest may occur in a subject during an experiment. Due to this, studies run the risk of measuring the neural correlate of a mental process unrelated to the one being investigated. Thus, before fMRI data are allowed in court, it must be established that the experiment was well controlled; the mental process the party is attempting to correlate with the recorded BOLD response must be the mental process the subjects were undergoing while being scanned.

Assuming the experiment is well controlled, the researcher must decide what statistical power he or she needs to call the change in BOLD response significant [4]. A change in BOLD response is quite a weak signal, and thus researchers run many different subjects many times in the same experiment, seeking to see the signal reliably reproduced. The researcher may choose which of several possible statistical thresholds, methods, and computer programs to use to establish significance [4]. Data that is not significant under one combination of thresholds, methods, and programs might become so under a different combination [4].

Once activation data is analyzed for each subject in an experiment, averages of subjects’ responses are composed, normally by warping each subject’s slightly different brain structure to the coordinates of a standard anatomical brain model [5]. As soon as each subject’s brain activation patterns have been fitted onto the standard model, the overlapping areas of activation are said to be the group-level activation patterns for that condition [5].

Variables unrelated to the experiment might affect the BOLD response. Every scanner is differentially sensitive to BOLD response changes in a way that is not clearly commensurable with other scanners [6]. Other factors, such as age, medical history, or possibly intrasubject variables such as stress level or time of day the scan is conducted, can affect the BOLD response [7]. Thus, average brain patterns gathered by one scanner, on one group of subjects, under one set of conditions are not necessarily comparable to those gathered by another running the same experiment.

In summary, then, most images produced through brain scanning are graphical representations of data that are the result of many choices made by the researcher during collection and analysis using possibly idiosyncratic equipment. This data is dependent on particular experimental choices and averaged over many subjects whose brain patterns could each be affected by variables unrelated to the experiment. Hence, there are many opportunities for an expert to manipulate the data to look the way an attorney wants, or, less cynically, to inadequately control for crucial variables.

Making a Reverse Inference from Group to Individual Activity Patterns

For the brain activity patterns (BOLD response) of the eyewitness in our scenario to be comparable to the group-level brain activity patterns found in the literature, there should be uniformity in the experimental design, characteristics of the subjects, statistical methods, and, optimally, the equipment used. Even if these criteria are met, the predictive value of any brain activation pattern for a particular mental event or capacity depends on the extent to which we can confidently infer the presence of a mental event from the activity patterns seen.

In our attorney’s case, the ability to do this is hampered by the “reverse inference” problem, which has two parts. First, we usually cannot conclude that a cognitive process is not present just because the brain state correlated with it in prior group studies is absent [8]. It is entirely possible that other brain states besides the one found to be correlated at the group level in past studies may also produce or imply the mental state or capacity. Second, even if the individual’s brain state does correlate with the brain state associated with a false memory in previous studies, we usually cannot be certain that this activation pattern is not also correlated with other mental processes or capacities [8]. Indeed, the same parts of the brain are used for many different mental processes.

Making inferences from group studies also faces intragroup problems. If there is variance in the group, an individual brain pattern that is quite different from the average may be subsumed in the averaging [9]. Indeed, no single subject may have an activation pattern that matches the averaged version, and thus group data does not tell how any individual’s brain pattern correlated with a cognitive process or capacity, but only how the average activation pattern correlated with an average process or capacity. For this reason, the data needed to establish the correlation is often hidden behind the averages at the group level.

At this point, it should be made clear that there need not be perfect correlation in order for brain data to be probative regarding the presence of a mental state. Rather, we need to have a reasonable amount of confidencein the correlation being claimed, even if the brain state predicts the mental state only, say, 60 percent of the time. Currently, most scientists do not believe correlations between known brain activity patterns and mental states are reliable enough to make a reasonable reverse inference [8].

Criteria for Admissible Evidence

Although the legal analysis is more complicated than the following might suggest, this is a starting point for thinking about the issues. Suppose the defense attorney now wants the company’s expert to testify for the jury about the findings, so that the jury might discount the eyewitness’s report. The prosecution may challenge the evidence on various grounds [10]. First, according to Federal Rules of Evidence (FRE) 901(a), the evidence must be “authenticated,” or, in other words, must be the product of methods that are reliable and scientifically valid, so that the court can be confident that the evidence demonstrates what the proponent claims that it does [11]. As we have seen, fMRI methods do not produce objective “pictures” of mental states, nor do the group level activation patterns normally allow a confident reverse inference at the individual level. As long as the court understands what the evidence actually illustrates, and the process used to create it, the prosecution’s argument that the evidence is inadmissible should be overcome.

Secondly, if the evidence is presented along with expert testimony, it may be challenged under the Daubert standard (which has been codified as FRE 702) [12]. This standard requires that the evidence be scientifically valid and reliable for the purpose for which it is being offered. Again, if the expert for the defense is trying to use the fMRI evidence to make statements about the witness’s mental states or capacity with undue certainty, it should not be allowed to be presented to the jury. Some states employ the Frye standard in place of the Daubert standard. The Fryestandard requires the science on which the testimony relies to be accepted by the relevant community, in our case the community of cognitive neuroscientists [12]. This standard is also likely to bar unjustified testimony of the kind we have been discussing.

Finally, the evidence may be challenged under FRE 401 and 403, which together require that the evidence’s probative value outweigh any undue prejudice created by presenting it to the jury [13]. We have seen that the probative value of neuroimaging evidence is usually low when inferences from group data in a lab experiment are applied to an individual in a court case. The FRE 403 requires that the probative value of evidence outweigh its misleading effect. Many commentators and studies suggest that the seemingly objective nature of the “image” of the brain scan causes jurors to overestimate its probative value [14]. Thus, if the expert can convey to the jury the limited nature of the evidence, reverse inference problem included, the fMRI report may still be admitted as having some probative value.

When analyzing whether the evidence passes the FRE 403 standard, a court can consider whether other pieces of evidence address the same fact with less undue prejudicial effect [15]. If so, even if the fMRI evidence is more probative than prejudicial, the court may decide that less prejudicial evidence should be used. As Emily Murphy and Teneille Brown point out, in the case of fMRI, psychological tests and inferences from the subjects’ actions are both alternatives that could speak just as well to mental state or capacity with less potential for prejudicial effect [15]. Whether Murphy and Brown are right is an empirical question that deserves further exploration.

There are ways to minimize the reverse inference problem. One method measures many more dimensions of brain activity than the typical study, leading to a very high threshold for specificity before brain patterns are said to be the same [16] and reducing the possibility that multiple mental states could be instantiated by that brain pattern. Researchers can also do scans under enough conditions with a representative population that the rate at which a brain state correlates with a mental state is better understood. However, given the large number of unique mental states that exist, and the variation in brain patterns in the population for the same general mental event, it seems a difficult task to establish correlations by this method [17].

An easier method for the individual experimenter is to scan a single subject enough times that a reverse inference can be made. This reduces the problem of high variation between subjects and the overall number of scans necessary to have the same level of confidence in the reverse inference. In our example, the company would be better off scanning the witness while she recalls a memory they know to be false (false memories can be planted in subjects with different methodologies), and those they know to be real. This would establish the brain patterns more robustly associated with false and true memories for this unique subject.

One day, perhaps, our fundamental understanding of the brain will advance to the point where we could observe an individual’s brain pattern and deduce the mental events instantiated. For now, however, our ability to read brain states is heavily dependent on an ability to make confident reverse inferences based on correlations. The use of neuroimaging research as evidence is currently a difficult enterprise that requires a court to carefully analyze the concerns outlined above. As cognitive neuroscience progresses as a field, efforts to satisfy the criteria necessary to responsibly use neuroimaging methods in court will surely expand and bear the fruit so many in the legal system desire.