A recent post on an attorney listserv included the following message \u2013 having never encountered an expert in a particular field before, the lawyer explained that they had \u201cused A.I. for a suggested cross-examination since I had never cross-examined [that type of expert] \u00a0before\u2026\u201d<\/p>\n
The danger to a client may be readily apparent, along with ethical concerns.\u00a0 And I wrote an article about them for the regional legal community newspaper.\u00a0 But when I sent that piece to Brain Lessons co-editor Grant Rost to ask if it belonged in out column, he wrote back the following.<\/p>\n
Could it be a brain lesson?\u00a0 Well, in its current state I think it\u2019s more \u201clesson\u201d than \u201cbrain\u201d if you catch my drift.\u00a0 Perhaps with some prefatory linking you can tie the two together more clearly.\u00a0 It currently looks more like a \u201cperils and\/or limitations of A.I.\u201d lesson than a study about the brain.\u00a0 The \u201clinkage\u201d that immediately popped into my head is from Kahneman himself.\u00a0 The brain prefers effortlessness\u2014which is clearly manifesting in brain of the lawyer in the post<\/em><\/p>\n Grant, unsurprisingly, nailed it.\u00a0 What I didn\u2019t know, and needed to research, was why a lawyer would trust AI.\u00a0 It turns out that there is ample research.<\/p>\n What I found first was the article \u201cThe Secret Reason People Trust Artificial Intelligence More Than They Fear It\u201d by the <\/strong>Editors of ScienceNewsToday<\/a> from August 28, 2025<\/a>.\u00a0 https:\/\/www.sciencenewstoday.org\/the-secret-reason-people-trust-artificial-intelligence-more-than-they-fear-it<\/a> (last visited April 30, 2026).\u00a0 That article described, and led me to, research by Phillip Brauner, a Ph.D. who studied computer science with specializations in psychology and human computer interaction.<\/p>\n Brauner and colleague surveyed 1,100 German adults about AI. \u00a0The primary purpose was to assess attitudes \u2013 for each of numerous AI applications, the subjects were asked about whether they had potential utility but also whether they were viewed as risky and detrimental. \u00a0As described in SECRET REASON, the findings included the following:<\/p>\n If participants saw an AI scenario as tangibly helpful\u2014such as supporting healthcare, easing everyday tasks, or providing companionship for the elderly\u2014they tended to rate it positively, even when some risks were acknowledged\u2026<\/p>\n Interestingly, perceived risks were consistently rated higher than perceived benefits across the board.,.But here\u2019s the twist: when it came to shaping overall evaluations, benefits carried far more weight than risks. A potentially useful application could win support despite acknowledged dangers\u2026<\/em><\/strong><\/p>\n That last point, that a useful application \u201ccould win support despite acknowledged dangers\u201d especially when it \u201ceas[ed[] everyday tasks\u201d drove me to contact Dr. Brauner.[i]<\/a>\u00a0 His response, prompt and exceptionally detailed, provided more articles and added insights.\u00a0 One involved \u201cautomation bias\u201d \u2013 the tendency to \u201cfavor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct.\u201d\u00a0 Hoffman, Automation Bias: What It Is And How To Overcome It, Forbes, March 10, 2024. \u00a0https:\/\/www.forbes.com\/sites\/brycehoffman\/2024\/03\/10\/automation-bias-what-it-is-and-how-to-overcome-it\/<\/strong><\/p>\n Dr. Brauner described another study of his.<\/p>\n We also did a study on that using a business simulation game and found that people tend to over-rely on automated systems (an effect or bias called automation complacency).\u00a0https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/0144929X.2019.1581258<\/a>\u00a0. The error of the Simulated Support System , clearly noticeable by the participant, \u00a0did not really change their behavior and they still relied on the system, despite its evident malfunctioning.<\/p>\n Automation bias is informed by \u201cmiscalibrated trust.\u201d\u00a0 Romeo, G., Conti, D. Exploring automation bias in human\u2013AI collaboration: a review and implications for explainable AI. AI & Soc<\/em> 41<\/strong>, 259\u2013278 (2026). https:\/\/doi.org\/10.1007\/s00146-025-02422-7<\/a>\u00a0 Romeo and Conti explain that \u201c[i]n situations with high workload, users may reallocate attention to manual tasks, neglecting the automation. Such behavior increases the risk of delayed detection of automation errors. A direct consequence of this behavioral pattern is the cognitive error of [automation bias].\u201d<\/p>\n So \u2013 we can now explain how the brain led this lawyer down the path to AI-reliance.\u00a0 Hopefully, that satisfied the Brain Lesson component of this article.\u00a0 What follows is the original law-focused article, the story of the lawyer\u2019s choice, my rejoinder, and what AI [Claude] had to say about it.<\/p>\n [THE ORIGINAL ARTICLE]<\/strong><\/p>\n The case in which the lawyer sought AI help was a child sexual assault prosecution; and the type of expert was one on the behaviors that might be exhibited by children who are victims of sexual abuse, including delayed reporting.\u00a0 The simple response to the use of A.I.\u00a0 is \u2013 bad idea.<\/p>\n Here are concerns that I raised in my listserv response:<\/p>\n I write to urge extreme caution in using A.I. in this fashion.\u00a0 There are a number of reasons:<\/em><\/p>\n But that\u2019s not all.\u00a0 I decided to ask an expert \u2013 CLAUDE -about the virtues and pitfalls of using A.I.<\/p>\n Here is my first prompt:<\/p>\n If I were to ask you to design a defense cross-examination of an expert witness in a child sexual assault prosecution where the expert would say that “the fact that the child had no particular behavioral problems does not mean there was no abuse,” could you do it and what risks of error would there be in the results? And would my input into Claude be public or private?<\/em><\/p>\n Claude responded quickly and thoroughly.\u00a0 It had some admittedly cogent ideas about such a cross, but volunteered these concerns:<\/p>\n Risks of Error in My Output<\/em><\/strong><\/p>\n This is important to be direct about:<\/em><\/p>\n I decided I needed to know more.\u00a0 Claude was surprisingly candid in replying to my next prompt, which was \u201cOne other question, please. How do I know the quality of the resources you would use for this type of inquiry?\u201d<\/em><\/p>\n Here is the A.I. confession:<\/p>\n This is one of the most important questions you could ask, and I want to give you an honest answer rather than a reassuring one.<\/em><\/p>\n The Short Answer<\/em><\/strong><\/p>\n You largely can’t verify my sources directly \u2014 and that’s a real limitation you should factor into how you use my output.<\/em><\/p>\n What I Actually Do (and Don’t Do)<\/em><\/strong><\/p>\n I don’t retrieve sources in real time for this kind of task.<\/em><\/strong> When I draft a cross-examination or analyze expert testimony methodology, I’m drawing on patterns learned during training \u2014 from legal textbooks, law review articles, case law, forensic psychology literature, trial advocacy materials, and similar sources. I don’t pull up a specific article and cite it to you unless I use a web search tool. This means:<\/em><\/p>\n I can hallucinate citations.<\/em><\/strong> This is a known and serious problem with AI systems. If you asked me to name the leading peer-reviewed studies on behavioral symptom presentation in child sexual abuse victims, I might give you author names, journal names, and publication years that feel authoritative but are partially or entirely fabricated. This has caused real harm when lawyers submitted AI-generated briefs with fake case citations.<\/em><\/p>\n To be clear, the lawyer who turned to A.I. didn\u2019t just regurgitate what was offered but adapted it. \u00a0And Dr. Brauner wrote me again to add these comments:<\/p>\n [M]ore complex systems have evolved tremendously over the past few years, especially with agentic AI. While the mentioned points, such as the cut off date of the training data or the lack of information on the jurisdiction is certainly true for the base LLM models, smarter tools are able to automatically send queries to the right databases, summarize or filter semantically (instead of an exact word match as a decade ago), and present the results in a meaningful manner. I do see many benefits of this across a broad range of disciplines and jobs (who even can read every case on a specific law anyway), but the main problem remains: what happens if errors occur and who takes the responsibility for it.<\/p>\n For this writer there remains the risk of \u2018garbage in, [adapted] garbage out.\u2019\u00a0 I am with Claude.\u00a0 If there is a listserv, ask for help from humans.\u00a0 Find out what the reputable sources are.\u00a0 And check your jurisdiction\u2019s law.\u00a0 Otherwise, what has superficial appeal may lead to disastrous consequences.<\/p>\n <\/p>\n ——–<\/p>\n <\/p>\n [i]<\/a> To learn more about Dr. Brauner, go to https:\/\/www.researchgate.net\/profile\/Philipp-Brauner<\/a>\u00a0\u00a0 To learn about how public perception of AI differs from that of AI experts, see his 2026 article Charting the AI perception gap: divergent views on risk, benefit, and value between experts and the public challenge the societal acceptance of AI.\u00a0 <\/strong>https:\/\/link.springer.com\/article\/10.1007\/s00146-026-03023-8<\/strong><\/a><\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" A recent post on an attorney listserv included the following message \u2013 having never encountered an expert in a particular field before, the lawyer explained that they had \u201cused A.I. for a suggested cross-examination since I had never cross-examined [that type of expert] \u00a0before\u2026\u201d The danger to a client may be readily apparent, along with<\/p>\n","protected":false},"author":31,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"generate_page_header":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[5],"tags":[],"coauthors":[330],"class_list":["post-4014","post","type-post","status-publish","format-standard","hentry","category-brain-lessons"],"yoast_head":"\n\n
\n
\n