-
The Ethics of GenAI Lawyering
John Bliss 3/29/24. Many lawyers seem wary about the idea of incorporating generative AI in their practice, often citing uncertainties about how the rules of professional responsibility might apply in this novel context. A leading national figure in bar regulation around emerging tech, Andrew Perlman, has written an authoritative article on this topic (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4735389), which […]
-
B+ on Autopilot: LLM Achieves new Grade on Unassisted First Drafts
John Bliss 2/16/24. A new study from a group of law faculty at the University of Maryland provides an update on LLM performance on law school exams. The lead author conducted a similar study last spring (2023), finding that GPT-4 scored as high as a B. The new study finds that GPT-4-Turbo scored as high […]
-
Lawyers Replaced in Contract Review?
John Bliss 2/14/2024. A new study finds that large language models (“LLMs”) perform contract review at near human-level accuracy, while dramatically cutting the required time and cost. This could suggest—as the authors conclude—that junior lawyers and LPOs are on the verge of radical disruption and even some degree of replacement. However, the article is very […]
-
Anti-hype
John Bliss 1/28/24. A position paper argues that GPT-4’s performance on law exams does not provide evidence that AI is “set to redefine the legal profession.”[1] In a Substack post, the authors summarize their position, as follows, “Will AI transform law? The hype is not supported by current evidence.”[2] I think the authors are right […]
-
Hallucinating about Legal AI hallucinations
John Bliss 1/19/24. A new study suggests that generative AI’s legal hallucinations are “alarmingly prevalent,” providing “error-ridden legal answers” to 69-88% of legal questions.[1] But this finding is misleading and has been widely misinterpreted. The study itself is rigorous and has important implications. Yet, the response to the piece in mass and social media has […]
-
Has AI Passed or Aced Law Exams?
John Bliss 1/3/24. At various points in the past year, it has been widely reported that generative AI systems have aced the bar exam at the 90th percentile, floundered on law school exams, and failed to show utility in real-world legal tasks. Yet, the latest empirical findings push back against each of these claims, highlighting […]
-
Why SAIL? The Importance of Keeping up with the Research
John Bliss 12/30/23. Legal AI is a fast moving field. In just the past year, we saw OpenAI’s foundation models improve from failing the Unified Bar Exam (miserably, with GPT-3.5 scoring below the 1st percentile) to passing it (comfortably, with GPT-4 scoring at roughly the 62nd percentile among first-time takers).[1] We have seen similar jumps […]
-
Subscribe
Subscribed
Already have a WordPress.com account? Log in now.
Scholarship on AI Lawyering (SAIL) 