What does the President of the United States have to do with forensic science? Answering that could take one, indirectly, back to Abraham Lincoln, who signed the legislation that led to the creation of the National Academy of Sciences. The NAS, in 2009, issued a report that called into question the validity and precision of many forensic disciplines. That report was STRENGTHENING FORENSIC SCIENCE: A PATH FORWARD.
But today the connection is more direct. An important part of the advisory apparatus available to President Obama is PCAST, the President’s Advisory Council on Science and Technology. And PCAST’s September, 2016 report FORENSIC SCIENCE IN CRIMINAL COURTS: ENSURING SCIENTIFIC VALIDITY OF FEATURE-COMPARISON METHODS – by some of this nation’s preeminent scientists – presented “findings concerning the ‘foundational validity’ of the indicated methods as well as their ‘validity as applied’ in practice in the courts…”
And why should judges and lawyers care? The Report shows that there are concerns even with the most highly-regarded of forensic disciplines – DNA analysis; and that those farther down the scientific ladder (and some clearly non-scientific) have even more foundational problems.
The word “foundational” is the focus of the Report. It is the metric of assessing:
that a method has been subjected to empirical testing by multiple groups, under conditions appropriate to its intended use. The studies must (a) demonstrate that the method is repeatable and reproducible and (b) provide valid estimates of the method’s accuracy (that is, how often the method reaches an incorrect conclusion) that indicate the method is appropriate to the intended application.
The Report highlighted a second and related concern. PCAST emphasizes that “[w]ithout appropriate estimates of accuracy, an examiner’s statement that two samples are similar—or even indistinguishable—is scientifically meaningless: it has no probative value, and considerable potential for prejudicial impact. “
Whether “scientifically meaningless” will translate into judicially “inadmissible” remains to be seen.
And how did the disciplines assessed by the Report fare? Discussed here are five prominent fields.
DNA:
There is foundational validity for single-source [DNA evidence coming from only one person] and simple mixtures where there is DNA from two individuals. However, where the mixture contains DNA from three or more persons, “studies have established the foundational validity of some objective methods under limited circumstances (specifically, a three-person mixture in which the minor contributor constitutes at least 20 percent of the intact DNA in the mixture) but that substantially more evidence is needed to establish foundational validity across broader settings.”
And the meaning of this? A judge must be hesitant in permitting conclusions about whether a DNA profile corresponding to that of a known individual is contained within a complex mixture of biological evidence.
LATENT PRINT [FINGERPRINT] EVIDENCE
Latent print examination received some support in the Report; it was deemed to have foundational validity but with “a false positive rate that is substantial and is likely to be higher than expected by many jurors based on longstanding claims about the infallibility of fingerprint analysis. The false-positive rate could be as high as 1 error in 306.” Unmentioned in the report is the separate issue of whether the ultimate conclusion – that a print did come from an individual and no one else – has a clear scientific basis and what number or constellation of features suffices to make that call. The Report also emphasized the subjective nature of the analysis, the risk of biasing information distorting the analysis, and the need for more “rigorous” proficiency testing to ensure that any individual analyst’s assessment deserves being credited.
Firearms:
Here, the Report found some but insufficient research to show foundational validity. The Report emphasizes that whatever current data about error rates are available need to be disclosed if courts permit firearms comparison testimony. Again, the Report notes concerns about the potential for cognitive bias and the need for proficiency testing.
Footwear Analysis:
The Report is clear and concise here, finding no validity to matching testimony that “associate[s] shoeprints with particular shoes based on specific identifying marks. Such associations are unsupported by any meaningful evidence or estimates of their accuracy and thus are not scientifically valid.”
Bitemark Analysis:
Perhaps the most damning words were saved for this field. “PCAST considers the prospects of developing bitemark analysis into a scientifically valid method to be low. We advise against devoting significant resources to such efforts.”
Responses to the Report have varied in tone. Judge Alex Kozinski, who served as an advisor to its preparation, has written in stark terms that “[o]nly the most basic form of DNA analysis is scientifically reliable, the study indicates. Some forensic methods have significant error rates and others are rank guesswork.” In slightly more muted terms, Judge Harry Edwards and Law School Dean Jennifer Mnookin, also advisors, wrote that “[t]he report finds that many forensic techniques do not yet pass scientific muster. This strongly implies these techniques are not ready for use in the courtroom either.”
The view from the prosecution side has been to the contrary, and terribly dismissive. The National District Attorneys Association released a press statement that “the opinions expressed by PCAST in their report clearly and obviously disregard large bodies of scientific evidence to the contrary and rely, at times, on unreliable and discredited research.” It decries the report as “scientifically irresponsible” and that adopting “any” of its recommendations would have a “devastating effect” on law enforcement.
For the judiciary, what does this mean? At a minimum, a judge must decide whether the Reports conclusions have merit and, if so, to be proactive and inquire into a discipline’s “foundational validity,” practitioner proficiency, risk of biasing information, and error rate, and then determine admissibility and scope of testimony; or weigh these factors if prompted by a motion to exclude or limit forensic discipline testimony.
If doing so, particular attention should be paid to terminology used by an expert. As the Report emphasizes,
courts should never permit scientifically indefensible claims such as: “zero,” “vanishingly small,” “essentially zero,” “negligible,” “minimal,” or “microscopic” error rates; “100 percent certainty” or proof “to a reasonable degree of scientific certainty;” identification “to the exclusion of all other sources;” or a chance of error so remote as to be a “practical impossibility.”
Judges will also have to assess whether the Report itself is admissible as a Government Report under Rule 803(8). If so, it becomes ‘evidence’ on its own for both an admissibility hearing and trial, as well as a document appropriate for examining witnesses.
And lawyers? The fundamental obligation is clear. It is to learn both the strengths of every forensic discipline and its limits, and to bring science to their understanding of evidence before it is presented in the courtroom.
[NOTE: A link to the Report may be found at https://www.whitehouse.gov/blog/2016/09/20/pcast-releases-report-forensic-science-criminal-courts . This author is also a member of the National Commission on Forensic Science. The views expressed here are solely his own and are not intended to represent the views of the Attorney General, the United States Department of Justice, or any government entity or agency.]