The Perils of Artificial Intelligence in Law
The integration of artificial intelligence within the legal domain presents an enticing opportunity to enhance efficiency in litigation processes; however, inaccuracies pertaining to legal precedent and the implication of algorithmic risk assessments in trials threaten the integrity of court proceedings and judicial decisions. The outcomes of court proceedings, influenced by algorithmic risk assessments as exemplified in State v Loomis, alongside the reliance on false precedents observed in Mata v Avianca, reflect the repercussions of relying on artificial intelligence tools to perform legal tasks without imperative human oversight.
As artificial intelligence becomes increasingly prominent among various professional sectors, its ability to facilitate document review and legal research provides a compelling rationale for its incorporation into the legal field. The eDiscovery tool, TAR (Technology-Assisted Review), utilizes artificial intelligence to review documents, prioritizing relevant information and expediting document identification.1 Although not as widely accepted by courts, Generative AI serves as another form of document review emerging in the field as an established technology; nearly three-quarters of attorneys intend to use Generative AI in their work.2 Furthermore, the influence of artificial intelligence extends beyond the work of attorneys and is present in the process of adjudication. Risk assessment algorithms play a significant role in criminal law judicial proceedings: they are widely used to assist judges in decisions regarding bail, sentencing, and probation conditions, offering input without human bias. Thus, the opportunities created by the presence of artificial intelligence in law pave the path for an efficient and objective approach to legal work.
Although the advancements artificial intelligence presents are appealing, its ability to perform well on standardized law exams is ambiguous and further begs the question of its qualifications in the legal field. GPT-4’s calculated score in the 90th percentile on the Uniform Bar Exam is frequently used to support the argument that artificial intelligence is a proficient tool in law. However, estimates used by the Illinois Institute of Technology to produce this statistic largely compared GPT-4’s performance with repeat test-takers’ who had previously failed the exam and scored below average; consequently, GPT-4’s score was notably inflated. When using official data from the National Conference of Bar Examiners, GPT-4’s score was estimated to fall approximately in the 62nd percentile.3 The inauspicious results of GPT-4 on the Uniform Bar Exam indicate that it is neither more qualified nor competent than legal professionals and should be utilized only as a supplemental tool, not as a reputable source.
Regardless of such shortcomings, input from artificial intelligence programs is considered in numerous case proceedings. In 2017, Eric Loomis was found driving a car that had been used in a shooting and was convicted, but he only pleaded guilty to eluding an officer. When determining his sentence, the judge considered input from the algorithmic risk assessment program COMPAS. Algorithmic risk assessments predict the reoffending probability of defendants; these assessments are often based on factors such as age, gender, and race.4 Loomis was sentenced to 6 years in prison, and the Wisconsin Supreme Court ruled that the use of COMPAS did not violate his due process rights, as it was not the sole determining factor in his sentencing.5 As risk assessments are increasingly factored into critical decision making, it is essential they be reviewed. A ProPublica analysis of COMPAS concluded that the program inordinately classified Black defendants into the high-risk category.6 Additionally, a study by Dr. Melissa Hamilton analyzed the COMPAS risk assessment of violent reoffending and revealed that the rate for women rated high-risk was 25%, less than half that of men.7 These results indicate the tool overpredicts the likelihood women will reoffend. If high-risk classification yields consequences such as high bail, pretrial detention, or increases in the severity of jurisdiction, then such programs could cultivate gender-biased and racially motivated legal consequences, violating a defendant’s due process rights. While Loomis was a white man who did not face discriminatory classification by the program, the case’s holding sets the precedent that discriminatory patterns in algorithmic risk assessments will not be addressed because they are not considered to be the sole determining factor. Therefore, even with judicial skepticism and oversight, the incorporation of algorithmic risk assessments may invite discriminatory practices into the adjudication process.
Moreover, artificial intelligence is frequently utilized by attorneys in legal research processes and argument formulation; relying on such tools can have critical repercussions if the information provided by artificial intelligence is inaccurate. In 2023, Roberto Mata sued Avianca Airlines after being injured from a metal serving cart on the flight. Mata’s lawyer, Steven A. Schwartz, used Chat GPT to conduct his legal research and failed to fact-check its findings with alternative sources.8 Subsequently, it was discovered that the legal precedents Schwartz provided as evidence did not exist. The presentation of false evidence in court diminished Schwartz’s legal reputation and serves as a representation of the legal work that is produced by artificial intelligence.9 Although artificial intelligence holds potential as a supplemental tool for document review or management, it necessitates human supervision and should not be relied on as an unvetted source for legal research.10 As demonstrated in Mata v Avianca, artificial intelligence is an unreliable source of legal knowledge and its unattended use could have detrimental consequences on the future integrity of legal briefings and court proceedings.
The incorporation of artificial intelligence in the legal field offers a plethora of promising possibilities that could facilitate growth in the legal workplace. Nevertheless, its lack of credibility regarding legal data and precedent, and its discriminatory role in algorithmic risk assessments for criminal trials, imperils the veridicality of legal protocol. The holdings reached in criminal trials using algorithmic risk assessments, as demonstrated in State v Loomis, and the resulting arguments of legal research based on artificial intelligence, presented in Mata v Avianca, exhibit the consequences of utilizing artificial intelligence in legal work. The further integration of artificial intelligence into various facets of law, without comprehensive and critical oversight, threatens the integrity of the legal system and its ability to justly interpret and uphold the law.
Bibliography
Atherton, Kelly, et al. "Technology Assisted Review (TAR) Guidelines." BOLCH Judicial Institute, EDRM/Duke Law School, Jan. 2019, scholarship.law.duke.edu/cgi/viewcontent.cgi?article=1002&context=bolch. Accessed Dec. 2024.
Magesh, Varun, et al. "AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries." Stanford University Human-Centered Artificial Intelligence, Stanford University, 23 May 2024, hai.stanford.edu/news/ ai-trial-legal-models-hallucinate-1-out-6-or-more-benchmarking-queries. Accessed 7 Dec. 2024.
Martinez, Eric. "Re-evaluating GPT-4's bar exam performance." Institute for Law and AI, May 2023, law-ai.org/re-evaluating-gpt-4s-bar-exam-performance/. Accessed 7 Dec. 2024.
"The Implications of AI for Criminal Justice." Council on Criminal Justice, Oct. 2024, counciloncj.org/the-implications-of-ai-for-criminal-justice/. Accessed 7 Dec. 2024.
Israni, Ellora. "Algorithmic Due Process: Mistaken Accountability and Attribution in State v Loomis." Edited by Evelyn Chang. Harvard Journal of Law and Technology, 31 Aug. 2017, jolt.law.harvard.edu/digest/ algorithmic-due-process-mistaken-accountability-and-attribution-in-state-v-loomis-1. Accessed 7 Dec. 2024.
Larson, Jeff, et al. "How We Analyzed the COMPAS Recidivism Algorithm." ProPublica, 23 May 2016, www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm. Accessed 7 Dec. 2024.
Hamilton, Melissa. "Justice served? Discrimination in algorithmic risk assessment." Research OUTREACH, 19 Sept. 2019, researchoutreach.org/ articles/justice-served-discrimination-in-algorithmic-risk-assessment/. Accessed 7 Dec. 2024.
Maruf, Ramishah. "Lawyer apologizes for fake court citations from ChatGPT." CNN, 28 May 2023, www.cnn.com/2023/05/27/business/chat-gpt-avianca-mata-lawyers/index.html. Accessed 7 Dec. 2024.
Weiser, Benjamin. "Man Sued Avianca Airline. His Lawyer Uses ChatGPT." The New York Times, 27 May 2023, www.nytimes.com/2023/05/27/nyregion/ avianca-airline-lawsuit-chatgpt.html. Accessed 7 Dec. 2024.
Picard, Sarah, et al. "Beyond the Algorithm: Pretrial Reform, Risk Assessment, and Racial Fairness." Center for Court Innovation, www.innovatingjustice.org/sites/default/files/media/document/2019/ Beyond_The_Algorithm.pdf. Accessed 7 Dec. 2024.