Lawyering in the Age of Artificial Intelligence

Lawyering in the Age of Artificial Intelligence

Jonathan H. Choi, Amy Monahan, and Daniel Schwarcz, researchers at the University of Minnesota Law School, conducted the first randomized controlled trial of AI assistance’s effect on human legal analysis.

We fed their paper into Dodonai's Legal AI Summarizer - used for tasks such as automating deposition summaries - and posted the results below.

AI Significantly Improves the Speed of Legal Task Completion

The study conducted a randomized controlled trial to assess the impact of AI assistance on human legal analysis. Sixty students at the University of Minnesota Law School were randomly assigned to complete four legal tasks with or without the assistance of GPT-4, an AI tool. The results showed that access to GPT-4 slightly improved the quality of legal analysis but significantly increased the speed of completing tasks. The benefits of AI assistance were more pronounced for lower-skilled participants. Participants reported increased satisfaction from using AI and correctly predicted the tasks for which AI was most helpful. The study suggests that law schools should assess how and when to train students to use AI, while lawyers and judges should embrace AI with variations based on practice area and task. Purchasers of legal services should reconsider which matters should be handled in-house or externally. The study provides important insights into the potential of AI to improve the efficiency of legal services.

Perhaps adding to the evidence, Dodonai summarized this 51 page law review article in less than 2 minutes. You can user our AI legal tools to summarize deposition transcripts, medical records, briefing, and other legal documents.

Legal AI Improves Efficiency, But Lawyers Are Not Going Anywhere

The use of databases and e-discovery tools in legal research has increased efficiency, but the skills required for analyzing and applying cases and statutes remain the same. AI tools like predictive coding have started to replace some tasks traditionally done by lawyers, leading to concerns about job displacement. However, there is limited empirical evidence on the impact of AI on the legal profession. Studies have shown mixed results on the performance of AI on law exams and tasks, with some improvements in lower-performing students but limitations in identifying relevant issues and applying rules. Outside of exams, there is little evidence on how access to AI tools like GPT-4 affects legal tasks. In other fields, AI performance on exams has varied, with some inaccuracies and outdated information. AI assistance has shown improvements in certain writing tasks but may undermine human incentives to take care. AI has been found to be effective in increasing quality and productivity in certain types of tasks, particularly when combined with effective prompt engineering. Lower-skilled participants tend to benefit the most from AI assistance, but even those in the top half of skill rankings have shown improvements. However, blindly adopting AI outputs can lead to a decrease in performance. Overall, AI shows promise in assisting with legal and knowledge-based tasks, but there are limitations and shortcomings to consider.

How The Study Determined the Effectiveness of Legal AI Tools

The study examined the use of AI, specifically GPT-4, in assisting with legal tasks. Previous research has shown that AI can be helpful but also has limitations, such as inaccuracies and potential negative effects on highly skilled individuals. The study recruited law students to participate in the experiment and provided them with training on how to effectively use GPT-4. The participants then completed four different legal tasks, with half of them using GPT-4 for two tasks and the other half not using AI for those tasks. The participants' work was graded anonymously, and they also completed a survey about their experience. The study aims to contribute to the existing literature on the effectiveness and efficiency of AI in legal tasks

The task was graded by a single investigator using a grading rubric. Participants were asked to take an anonymous survey about their experience. The results showed that access to AI improved the speed of completion but had little effect on the quality of output in lawyering tasks. Participants who had the worst performance withoutAI received the most benefit from AI assistance. Participants reported positive impressions of AI and indicated that they would use AI tools in the future. The differences in grades received and time taken for each task were relatively small. Access to AI had the largest positive effect on contract drafting and had little effect on client memo and employee handbook tasks. The decrease in time taken on each task with AI assistance was statistically significant. The analysis showed that AI assistance benefited low performers the most and had little impact on top performers.

AI Will Become a Vital Tool for Lawyers

The article discusses the results of a study on the impact of AI assistance on legal tasks. The study found that access to AI tools like GPT-4 improved the efficiency of completing tasks without negatively affecting the quality of the work. Participants reported that AI had a larger effect on the quality and speed of their work, and it also increased their personal satisfaction. The study suggests that AI will become a vital tool for lawyers in the future, and its adoption may be driven by competitive dynamics. The passage also mentions that there are more specialized AI tools available for lawyers, and participants' limited experience with AI may have understated its potential. Additionally, the capabilities of AI are rapidly advancing.

The article mentions that as lawyers practice using AI, they will become more skilled at using it effectively and efficiently. The article also highlights the rapid acceleration of AI capabilities, with newer models like GPT-4 being better at legal analysis than previous models. The passage acknowledges that the study participants, who were law students, may have been particularly enthusiastic about AI and potentially had prior exposure to using it. It suggests that AI may benefit less-skilled lawyers more than highly skilled ones, potentially equalizing the profession. The passage also notes that the impact of AI on different tasks and practice areas may vary, with some tasks being more amenable to AI assistance than others. It mentions the potential forAI to improve the efficiency of tasks like summarizing complex documents. Overall, the passage suggests that the practice of law is on the verge of significant change and transformation due to AI, but the exact implications and consequences are uncertain.

AI's Potential Impact on the Legal Profession

The article discusses the potential impact of generative AI on the legal profession. It raises questions about the future demand for legal services, the range of services provided by law firms, lawyer pay, and the supply of lawyers and law students. The results of the study suggest that the assumption that the future will resemble the past is uncertain. The article also discusses the implications for law schools and law students, suggesting that generative AI should be limited in first-year law school classes to ensure the development of essential legal reasoning skills. However, upper-level classes should be developed to train students on how to effectively use generative AI tools. The article also highlights the need for lawyers and law firms to proactively explore the incorporation of generative AI into their practice, with many already doing so.

The use of generative AI in the legal profession is becoming more common, but there are risks involved. One lawyer relied on AI to write a brief and ended up including false information, resulting in public embarrassment. However, this does not mean that generative AI should be avoided altogether. Instead, lawyers and law firms should develop effective systems and procedures for using AI tools, such as verifying factual statements and avoiding entering confidential information. Clients should also be proactive in asking their attorneys about their use of AI and how it impacts the quality and cost of legal services. The use of generative AI can improve efficiency in legal work, but clients should monitor how it affects their legal bills. Judges should not ban the use of generative AI by lawyers, as it can benefit both clients and lawyers, and any misuse can be addressed through existing tools and sanctions.

Overall, AI has the potential to transform the legal profession by streamlining tasks, improving lawyer satisfaction, and reducing inequality.

‍

Learn more about how you can leverage Dodonai's legal AI tools to make your practice more efficient.