A California bill to regulate artificial intelligence cleared a crucial legislative hurdle Thursday as it advanced out of a key committee hearing despite heavy opposition from technology companies.
The State Assembly’s Appropriations Committee approved an amended version of the bill, SB 1047 by Sen. Scott Wiener, a San Francisco Democrat, on an 11-3 vote Thursday, moving it out of the “suspense file” where dozens of bills with potential budgetary impacts often are culled after being taken up in a quick succession of votes without public testimony.
Wiener’s bill would regulate the “development and deployment of advanced AI models” from large companies developing AI models that cost over $100 million, requiring safety testing, safeguards to prevent misuse of AI and monitoring after technology is deployed.
The bill also creates whistleblower protections for AI employees, and allows the state Attorney General to take legal action against companies who cause “severe harm” or public safety threats and creates a public cloud computer cluster, CalCompute.
The bill has divided Silicon Valley. Major tech companies and two Silicon Valley Congress members have come out against the bill, saying it would harm innovation. Meanwhile, some leaders in the AI field, including the “Godfathers of AI,” Geoffrey Hinton and Yoshua Bengio, have lent their support.
Several amendments sought by the bill’s critics including AI startup Anthropic were added before the bill passed out of the committee. Wiener agreed to cut criminal penalties from the bill, and removed the creation of a new regulatory body, the the Frontier Model Division, among other changes.
Wiener said the bill still provides important guardrails on the emerging technology that has both great promise and great potential for misuse, which could include robotic weapons to automated hacking and manipulation of financial markets.
“We can advance both innovation and safety; the two are not mutually exclusive,” Wiener said in a statement. “I believe we’ve addressed the core concerns expressed by Anthropic and many others in the industry.”
But critics in the tech industry and legislature remain opposed.
In a letter to Gov. Newsom Thursday, Bay Area representatives Anna Eshoo, Zoe Lofgren and Ro Khanna asked Newsom to veto the bill, raising concerns about effects on the state’s “innovation economy.”
“SB 1047 creates unnecessary risks for California’s economy with very little public safety benefit,” the letter reads. “SB 1047 is skewed toward addressing extreme misuse scenarios and hypothetical existential risks while largely ignoring demonstrable AI risks like misinformation, disinformation, nonconsensual deepfakes, environmental impacts, and workplace displacement.”
In early August, a letter from Jaikumar Ramaswamy, the chief legal officer at Andreessen Horowitz, a major AI investor, argued the bill would cause a multitude of harms to developers. The regulations would “deter innovation,” stifle open-source AI models and could prompt tech companies to move their offices out of state, Ramaswamy wrote.
The bill now heads to the Assembly floor, where it must pass by August 31. The bill would then head back to the state Senate for approval, then to Gov. Newsom, who has not yet publicly voiced an opinion on the measure.
The bill, among dozens of proposed AI legislation nationally, would be the first of its kind, with California legislators looking to get out ahead of possible federal regulations.
“Congress has passed no major technology regulation since computers used floppy disks,” Wiener said in a statement. “California must act to get ahead of the foreseeable risks presented by rapidly advancing AI while also fostering innovation.”
