Following up on this morning's post, ChatGPT Gets B|B- Grade On Wharton MBA Exam:  Jonathan Choi (Minnesota; Google Scholar), Kristin Hickma

TaxProf Blog: ChatGPT Gets C+ Grade On Four Minnesota Law School Exams (C- In Tax)

submited by
Style Pass
2023-01-25 14:30:11

Following up on this morning's post, ChatGPT Gets B|B- Grade On Wharton MBA Exam:  Jonathan Choi (Minnesota; Google Scholar), Kristin Hickman (Minnesota; Google Scholar), Amy Monahan (Minnesota) & Daniel Schwarcz (Minnesota; Google Scholar), ChatGPT Goes to Law School:

How well can AI models write law school exams without human assistance? To find out, we used the widely publicized AI model ChatGPT to generate answers on four real exams at the University of Minnesota Law School. We then blindly graded these exams as part of our regular grading processes for each class. Over 95 multiple choice questions and 12 essay questions, ChatGPT performed on average at the level of a C+ student, achieving a low but passing grade in all four courses. After detailing these results, we discuss their implications for legal education and lawyering. We also provide example prompts and advice on how ChatGPT can assist with legal writing.

Overall, ChatGPT passed all four classes based on its final exam, averaging a C+ across all exams, an outcome which would earn credit toward the JD but place the student on academic probation. Notably, if such performance were consistent throughout law school, the grades earned by ChatGPT would be sufficient for a student to graduate. Despite performing sufficiently well to theoretically earn a JD degree, ChatGPT generally scored at or near the bottom of each class. ChatGPT received a B in Constitutional Law (36th out of 40 students), a B- in Employee Benefits (18th out of 19 students), a C- in Taxation (66th out of 67 students), and a C- in Torts (75th out of 75 students). ...

Leave a Comment