Federal judge: No AI in my courtroom unless a human verifies its accuracy

A federal judge in Texas has a new rule for lawyers in his courtroom: No submissions written by artificial intelligence unless the AI's output is checked by a human. US District Judge Brantley Starr also ordered lawyers to file certificates attesting that their filings are either written or reviewed by humans.

"All attorneys appearing before the Court must file on the docket a certificate attesting either that no portion of the filing was drafted by generative artificial intelligence (such as ChatGPT, Harvey.AI, or Google Bard) or that any language drafted by generative artificial intelligence was checked for accuracy, using print reporters or traditional legal databases, by a human being," according to a new "judge-specific requirement" in Starr's courtroom.

A certification must be submitted for each case and would cover all filings in the case. A sample certification says the requirements apply to any language in a filing, "including quotations, citations, paraphrased assertions, and legal analysis."

Starr, a Trump nominee in US District Court for the Northern District of Texas, said that AI "platforms are incredibly powerful and have many uses in the law: form divorces, discovery requests, suggested errors in documents, anticipated questions at oral argument. But legal briefing is not one of them." AI "platforms in their current states are prone to hallucinations and bias. On hallucinations, they make stuff up--even quotes and citations."

Starr's new standing order on AI was posted yesterday, according to an article by legal expert Eugene Volokh. "Note that federal judges routinely have their own standing orders for lawyers practicing in their courtrooms. These are in addition to the local district rules, and to the normal Federal Rules of Civil and Criminal Procedure," Volokh wrote.

Lawyer cited fake cases invented by ChatGPT

Starr's order came after New York-based lawyer Steven Schwartz admitted that he used ChatGPT to help write court filings that cited six nonexistent cases invented by the artificial intelligence tool. Schwartz and his associates are awaiting possible punishments from Judge Kevin Castel of US District Court for the Southern District of New York.

In what Castel called "an unprecedented circumstance," the judge said filings from the plaintiff's lawyers included six "bogus judicial decisions with bogus quotes and bogus internal citations." The filings included names of made-up cases and a series of "excerpts" from bogus rulings that cited additional fake precedents invented by AI.

An affidavit filed by Schwartz said he "greatly regrets having utilized generative artificial intelligence to supplement the legal research performed herein and will never do so in the future without absolute verification of its authenticity." He also said he had "never utilized ChatGPT as a source for conducting legal research prior to this occurrence and therefore was unaware of the possibility that its content could be false."

Judge: AI swears no oath, holds no allegiance

Starr's new standing order discussed potential biases in AI tools. "While attorneys swear an oath to set aside their personal prejudices, biases, and beliefs to faithfully uphold the law and represent their clients, generative artificial intelligence is the product of programming devised by humans who did not have to swear such an oath," he wrote.

AI "systems hold no allegiance to any client, the rule of law, or the laws and Constitution of the United States (or, as addressed above, the truth)," Starr continued. "Unbound by any sense of duty, honor, or justice, such programs act according to computer code rather than conviction, based on programming rather than principle. Any party believing a platform has the requisite accuracy and reliability for legal briefing may move for leave and explain why."

Going forward, Starr's order said the court "will strike any filing from an attorney who fails to file a certificate on the docket attesting that the attorney has read the Court's judge-specific requirements and understands that he or she will be held responsible under Rule 11 for the contents of any filing that he or she signs and submits to the Court, regardless of whether generative artificial intelligence drafted any portion of that filing."