The volume of legal documents generated with the assistance of ChatGPT and other chatbots is rapidly increasing, leading judges to consider whether such systems should be held accountable for their legal guidance.
American judges are increasingly linking the rise in lawsuits filed by individuals representing themselves in court without legal counsel to the proliferation of artificial intelligence. A recent study analyzing 4.5 million federal civil cases from 2005 to 2026 indicates that the proportion of pro se filings—cases brought without attorneys—grew from 11% in 2022 to 16.8% in 2025. Concurrently, the volume of procedural documents in such cases more than doubled compared to the pre-2023 period.
Federal Magistrate Judge Maritza Braswell of Colorado believes AI is a contributing factor. She frequently observes characteristics of large language model usage in court filings, such as a distinctive writing style, fabricated judicial precedents, and non-existent citations. However, she also notes that many documents are now noticeably better written.
The study’s authors, Anand Shah from MIT and Joshua Levy from the University of Southern California, sought to test this hypothesis. They ran 1,600 randomly selected legal documents through the commercial AI text detector Pangram. While only 1% of documents showed signs of AI generation in 2023, this figure rose to 18% by 2026.
Judges report that while the influx of AI-generated documents increases their workload, it also streamlines their tasks. Documents that were once nearly illegible or presented convoluted arguments are now more frequently structured and coherent. Braswell points out that even with the need to verify texts for errors and AI “hallucinations,” it is often easier for her to understand the position of someone who utilized a chatbot than someone who drafted documents independently.
Despite the improved quality of the written texts, this has not led to increased success rates in these cases. The study found that individuals without legal representation continue to lose cases at a significantly higher rate than parties represented by professional attorneys. According to Levy, litigation involves more than just document preparation, and text generation addresses only a portion of the necessary tasks.
Online communities where users share instructions on using AI for filing lawsuits are also fueling these practices. A prominent example was a popular Reddit post in late 2024 discussing lawsuits against U.S. Citizenship and Immigration Services for delays in processing applications. The post suggested drafting an initial lawsuit using Microsoft Copilot, paying an attorney approximately $150 for a final review, and then filing it in Vermont federal court. Following this, the number of pro se cases filed in Vermont increased from roughly 45 per year to over 1,100 in 2024.
Simultaneously, courts have begun grappling with a new set of legal questions. One such issue concerns the confidentiality of user communications with chatbots. In February, a federal court in Michigan ruled that a person’s correspondence with ChatGPT during case preparation constituted protected legal work. However, on the same day, a federal court in New York reached a contrary conclusion regarding documents created with Claude, stating that a chatbot is not an attorney and users cannot expect the same level of communication confidentiality.
In March, Braswell issued a ruling asserting that the use of a chatbot in case preparation should not automatically forfeit an individual’s right to privacy. She noted that data collection by AI development companies does not, in itself, eliminate all reasonable expectations of confidentiality. Consequently, a unified stance on this matter among U.S. courts has yet to emerge.
Judges are also debating whether chatbots should bear responsibilities akin to those of attorneys. Federal Magistrate Judge Allison Goddard of California shared instances where ChatGPT provided individuals with inaccurate expectations regarding potential compensation amounts. In one case, a plaintiff sought $700,000 for an injury sustained in a store fall—a sum far exceeding the actual value of the claim. Further inquiry revealed that the calculation was based on ChatGPT’s recommendations.
Another contentious issue involves accountability for such errors. In March, Nippon Life Insurance Company filed a lawsuit against OpenAI, alleging that ChatGPT was engaged in unauthorized legal practice and assisted a user in reopening a settled dispute, leading to the generation of numerous unfounded documents. In May, OpenAI requested the dismissal of the suit, arguing that ChatGPT does not practice law as it is not human and lacks traditional professional legal expertise. The litigation is ongoing.
In parallel, lawmakers are attempting to establish new regulatory frameworks. A bill was introduced in New York in March that would prohibit chatbots from posing as attorneys, even with a disclaimer that the user is interacting with AI. In the U.S. Congress, several initiatives have been proposed to ban the impersonation of licensed professionals, including not only lawyers but also doctors. To date, none of these proposals have gained significant traction.
Judges observe that despite the risks, individuals continue to seek legal assistance from AI. According to Braswell, many litigants now respond to court inquiries with greater confidence because they have practiced potential conversation scenarios with chatbots beforehand. As a result, artificial intelligence has not necessarily simplified or made the judicial system fairer, but it has undeniably altered how ordinary citizens interact with it.
About the Author
