The California State Bar has stirred controversy by admitting that artificial intelligence (AI) was used to develop some questions for the February 2025 bar exam. This revelation comes after numerous complaints from test-takers regarding technical issues and the quality of the exam questions, raising serious concerns about the integrity of the legal licensing process.
Key Takeaways
- The California State Bar admitted to using AI in developing bar exam questions.
- Many test-takers reported significant technical issues during the exam.
- The State Bar plans to request score adjustments for affected candidates.
- Experts express concerns over the validity and reliability of AI-generated questions.
Background of the Controversy
In February 2025, the California bar exam faced a barrage of complaints from candidates who experienced technical difficulties, including crashes of the online testing platform and issues with saving their essays. The State Bar’s recent admission that some multiple-choice questions were created with AI has only intensified the scrutiny surrounding the exam.
According to the State Bar, 100 of the 171 scored multiple-choice questions were developed by Kaplan Exam Services, while 48 were recycled from a first-year law student exam. A smaller subset of 23 questions was created by ACS Ventures, the State Bar’s psychometrician, using AI technology.
Expert Reactions
Legal experts have expressed shock and concern over the use of AI in crafting exam questions. Mary Basick, assistant dean at UC Irvine School of Law, stated, "Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable." Katie Moran, an associate professor at the University of San Francisco School of Law, echoed these sentiments, highlighting the potential conflict of interest in having AI-generated questions validated by the same company that created them.
Andrew Perlman, dean of Suffolk University Law School, noted that while AI can be beneficial for developing assessment questions, it is crucial that such questions are thoroughly vetted by legal experts to ensure their validity.
The State Bar’s Defense
Despite the backlash, the State Bar maintains confidence in the validity of the exam questions. Leah Wilson, the executive director, stated that all questions, including those developed with AI, were reviewed by content validation panels and subject matter experts. The State Bar claims that the questions performed above the psychometric target for reliability.
However, critics argue that the use of AI and recycled questions from a first-year law exam raises significant concerns about the quality and appropriateness of the questions. Basick pointed out that the standards for a bar exam are fundamentally different from those of a first-year law school exam, emphasizing the need for a rigorous assessment of candidates’ legal competence.
Future Implications
The fallout from this incident has prompted calls for an audit of the State Bar and a return to traditional in-person exams. California Senate Judiciary Chair Thomas J. Umberg has voiced concerns over the integrity of the bar exam process, while many legal educators are advocating for transparency and a reevaluation of the exam’s structure.
As the State Bar prepares to meet on May 5 to discuss potential remedies for affected candidates, the future of the bar exam in California remains uncertain. The integration of AI in legal assessments may continue to evolve, but the need for rigorous oversight and expert validation will be paramount to maintain the integrity of the legal profession.
Sources
- AI helped write bar exam questions, California state bar admits | California, The Guardian.
- State Bar of California admits it used AI to develop exam questions, Los Angeles Times.