Most states have told lawyers how they should use AI. California is close to being the first state that can actually enforce it.
Senate Bill 574 passed the California Senate 39 to 0 in late January 2026 and is now working its way through the Assembly, which must pass it by August 31, the final day of the legislative session, or it dies for this session. If it clears the Assembly, it goes to Governor Gavin Newsom, who can sign it, let it become law without a signature, or veto it. If enacted, it would typically take effect the following January.
What makes SB 574 different from what has come before isn’t really its content. Most of what it asks of lawyers already exists in some form as guidance from the State Bar of California. What’s different is the form. Guidance is advisory. A statute is not.
What the bill actually requires
Introduced by State Senator Tom Umberg, chair of the Senate Judiciary Committee, SB 574 sets duties for attorneys and arbitrators using generative AI. Stripped of the legislative language, four things stand out.
First, confidentiality at the point of input. Attorneys would be prohibited from entering confidential, personal identifying, or otherwise nonpublic information into a public generative AI system. The bill does not attempt to define every edge case of what counts as confidential, but it does specify personal identifying information clearly: birthdates, Social Security numbers, driver’s license numbers, financial account numbers, addresses, and phone numbers, along with anything already sealed or protected by court order or statute.
Second, personal verification. This is the provision most directly aimed at the hallucination problem. An attorney responsible for a filing would have to personally read and verify every citation in it, regardless of whether AI, a paralegal, or the attorney themselves originally produced it. Delegating the drafting is fine. Delegating the verification is not.
Third, protection against bias. The bill includes language intended to prevent generative AI use from producing discriminatory or unlawfully biased outcomes, an obligation that sits alongside existing anti-discrimination duties rather than replacing them.
Fourth, disclosure consideration. Attorneys would need to consider whether disclosing AI use is appropriate when generative AI is used to create public-facing content, though this provision is framed as a duty to consider rather than a blanket disclosure mandate.
The bill also reaches beyond lawyers to arbitrators, who would be barred from delegating actual decision-making to a generative AI tool. An arbitrator can use AI as an aid. The independent analysis of facts and law has to remain the arbitrator’s own.
Why this is happening now
SB 574 didn’t emerge from nowhere. It’s a direct legislative response to a problem that has been building in courtrooms across the country for three years: AI systems producing fabricated case citations, invented quotations, and misstated holdings, some of which have made it past the attorney who signed the filing and into the official court record.
The scale of the problem is no longer a handful of embarrassing anecdotes. Researchers tracking the issue have documented well over a thousand US court proceedings in which a party relied on AI-hallucinated material and a court responded, with sanctions escalating sharply since the first widely reported case in 2023. Several federal appellate courts have noted publicly that warnings and reprimands alone have not slowed the trend.
California’s own State Bar has been working a parallel, related track. Its ethics committee, COPRAC, separately proposed folding AI-specific obligations directly into the state’s formal Rules of Professional Conduct, rather than keeping them in a non-binding practical guidance document. That rule-amendment process and SB 574 are distinct efforts moving on separate timelines, one through the bar’s rulemaking process and one through the legislature, but they point in the same direction: California moving AI obligations from advisory to enforceable.
What it would mean in practice
For law firm partners and litigators, the personal verification requirement is the one to plan around. It doesn’t ban AI-assisted drafting or research. It does mean the attorney who signs a filing cannot treat AI-checked as good enough; they need their own read of every citation, which has real implications for how review workflows and billing are structured, particularly on large filings.
For General Counsel, the relevant question is less about internal use and more about what standard to expect from outside counsel. If California codifies personal verification as a matter of law, it becomes a reasonable baseline to ask about when engaging or auditing outside firms, in California and, over time, likely well beyond it.
For legal operations and legal technology teams, the confidentiality provision is the most immediately actionable. It draws a clear line around public generative AI tools specifically, which puts pressure on firms and departments to be explicit about which tools are “public” versus private or closed instances, and to have that distinction documented rather than assumed.
What happens next
As of this writing, SB 574 is pending in the Assembly, working through committee review ahead of the August 31 deadline. A unanimous Senate vote signals strong support but does not guarantee passage in the Assembly, and the bill has already been amended once since its introduction in response to feedback from practitioners and legal educators. We’ll be tracking where it lands.
California has a track record of setting standards that other states eventually adopt in some form, from privacy law to AI regulation more broadly. Whether or not SB 574 becomes law this session, it’s a reasonable preview of where the rest of the profession is headed on AI accountability. Worth knowing now, not after it’s already the rule in your state.
This article reflects the status of SB 574 as of the publication date noted below. Legislative status can change quickly; readers should confirm current status before relying on this piece for compliance purposes.
