Can checklists reduce lawyer error? As they do for doctors or airline pilots or building engineers? Although the focus on this technique has largely been outside of the realm of the legal system, there is enough known to say that its application to lawyers is both necessary and likely to be beneficial.
That lawyers do make errors, and errors of grave consequence, cannot be doubted. While it has been written that “[n]o empirical research exists regarding error rates by lawyers, but it is reasonable to assume they make errors as often as doctors[,]’ McClurg, FIGHT CLUB: DOCTORS VS. LAWYERS – A PEACE PLAN GROUNDED IN SELF-INTEREST, 83 Temp. L. Rev. 309, 349 (2011), there are some estimates if not hard data.
The late Justice Scalia, when defending death penalty jurisprudence and practice, famously (or perhaps infamously) quoted a New York Times article that “the error rate [in criminal cases is] .027 percent–or, to put it another way, a success rate of 99.973 percent.” Kansas v. Marsh, 548 U.S. 163, 198 (U.S. 2006). Of course, this was an estimate of ultimate error – a truly wrongful conviction of an innocent person; and not the more pervasive issue of an error in attorney performance that deprived a litigant of a day in court, a proper objection, or a better [and fairer] outcome.
In the latter regard, performance error, the statistics are high and daunting. Professor James Liebman studied approximately 6,000 capital cases between 1973 and 1995 and found an overall error rate (cases reversed) of 68%, with 37% being for “egregiously incompetent defense lawyering.” James S. Liebman et al., A Broken System: Error Rates in Capital Cases, 1973-1995, at ii (2000).
In a study of Philadelphia homicide cases, who the lawyer was made a critical difference in outcome:
Compared to appointed counsel, public defenders in Philadelphia reduce the murder conviction rate by 19%. They reduce the probability that their clients receive a life sentence by 62%. Public defenders reduce overall expected time served in prison by 24%.
Anderson and Heaton, HOW MUCH DIFFERENCE DOES THE LAWYER MAKE? (Rand, 2011).
By citing to evidence of lawyer error in criminal cases, I do not mean to suggest that there is no or significantly less error in civil litigation. There, however, data are more difficult to unearth as there is no constitutional right to the effective assistance of civil counsel, and the methods of unearthing and proving error – malpractice litigation and disciplinary code enforcement – are much more cumbersome and/or are less likely to be utilized. Nonetheless, disciplinary records and some malpractice verdicts confirm the unsurprising conclusion that civil litigation is not immune from attorney error.
So, what can checklists offer? The most visible proponent is surgeon Dr. Atul Gawande, whose 2009 book The Checklist Manifesto details the use of this tool in aviation, engineering and, of course medicine. As he wrote in an earlier New Yorker article on the subject,
The checklists provided two main benefits…First, they helped with memory recall, especially with mundane matters that are easily overlooked in patients undergoing more drastic events. A second effect was to make explicit the minimum, expected steps in complex processes.
But is this applicable to lawyers? There are some initial efforts to use these for
- In-house corporate counsel (see THE CHANGING ROLE AND NATURE OF IN-HOUSE AND GENERAL COUNSEL: WHERE IS THE “QUALITY MOVEMENT” IN LAW PRACTICE?, 2012 Wis. L. Rev. 387, 391-393 (2012));
- Judges, for both substantive law and procedures (see AN AJA WHITE PAPER: A WHITE PAPER OF THE AMERICAN JUDGES ASSOCIATION MAKING BETTER JUDGES(R): MINDING THE COURT: ENHANCING THE DECISION-MAKING PROCESS, 49 Court Review 76, 95-96; COMMENT: AN OPEN COURTS CHECKLIST: CLARIFYING WASHINGTON’S PUBLIC TRIAL AND PUBLIC ACCESS JURISPRUDENCE, 87 Wash. L. Rev. 1203, 1235-1236 (December 2012);
- Prosecutors, in ensuring disclosure of exculpatory information (see NEW PERSPECTIVES ON BRADY AND OTHER DISCLOSURE OBLIGATIONS: WHAT REALLY WORKS?: PROFESSIONAL AND CONVICTION INTEGRITY PROGRAMS: WHY WE NEED THEM, WHY THEY WILL WORK, AND MODELS FOR CREATING THEM, 31 Cardozo L. Rev. 2215, 2240); Brown, : THE RIGHT TO COUNSEL AT TRIAL: DEFENSE COUNSEL, TRIAL JUDGES, AND EVIDENCE PRODUCTION PROTOCOLS, 45 Tex. Tech L. Rev. 133, 135-136 (2012); and
- Criminal defense counsel, particularly to screen client needs and be aware of potential collateral consequences of an arrest and/or conviction (see http://www.bronxdefenders.org/programs/checklist-project/ (last visited December 14, 2015).
The most far-reaching effort in this area is found in the Office of the Public Defender in San Francisco. Working in collaboration with the Center for Court Innovation, the Defender has developed checklists – checklists on tasks, investigative approaches and recurring legal issues in particular categories of cases, from drunk driving to homicide; and checklists on client interviewing and needs. See “Using Checklists To Improve Case Outcomes,” THE CHAMPION (January-February 2015).
For San Francisco Chief Defender Jeff Adachi, checklists have multiple utility – they reduce error, increase team communication and collaboration, and serve as a useful training tool, an accountability measure, and a standard to assess whether an institutional office has sufficient resources to operate effectively.
And Adachi’s next step? The checklists will be used in two offices, with some lawyers/teams having them and others not, and a controlled assessment will be conducted to measure error reduction.
How does one make checklists really work? Suggestions include a team effort in their creation, as collaboration increases a sense of personal investment in the success of the project while ensuring that that the checklists are covering the most important issues (and those where error is likely to occur). The rest may be a matter of transforming the office or practitioner “culture” to accept that checklists are necessary and complementary, rather than an impediment, to effective and creative lawyering. Imagine a checklist stapled to the inside of each file as a guidepost.
Will they really make a difference? Look no further than Delaware, where a death penalty conviction was reversed because counsel “forgot to examine an eyewitness to the shooting about statements the witness made…” Starling v. State, 2015 Del. LEXIS 665, *3 (Del. Dec. 14, 2015)(emphasis added).