California Gov. Gavin Newsom vetoed the bill SB 1047, which aims to prevent bad actors from using AI to cause “great harm” to people. The California state assembly passed the law 41-9 on August 28, but several organizations including the Chamber of Commerce urged Newsom to vote against the bill. In his veto message on September 29, Newsom said the bill is “well-intentioned” but “doesn’t take into account that the AI system is being used in high-risk areas, involving critical decision-making or the use of sensitive data. Instead, the bill applies stricter standards even to things fundamentals – as long as the larger system uses it.”
SB 1047 would have made developers of AI models responsible for adopting safety protocols that would prevent catastrophic use of their technology. That includes preventive measures such as testing and external risk assessment, as well as an “emergency stop” that can completely shut down the AI model. A first violation can cost at least $10 million and $30 million for subsequent violations. However, the bill was revised to eliminate the state attorney general’s ability to sue AI companies for negligent practices if a dangerous incident does not occur. Companies will only be subject to exemptions and can be sued if their model causes serious harm.
This rule will apply to AI models that cost at least $100 million to run and 10^26 FLOPS to train. It may also include spin-off projects in cases where a third party has invested $10 million or more in developing or modifying the original model. Any company doing business in California will be subject to the regulations if it meets certain requirements. Addressing the bill’s focus on large-scale initiatives, Newsom said, “I don’t believe this is the best way to protect the public from the very real threats that technology poses.” The veto message adds:
By focusing only on the most expensive and large models, SB 1047 establishes a regulatory framework that would give the public a false sense of security about regulating this fast-moving technology. Smaller, niche models may emerge as equally or even more dangerous than the models targeted by SB 1047 – at the potential cost of stifling the very innovation that fuels development in favor of the public good.
An earlier version of SB 1047 would have created a new department called the Frontier Model Division to oversee and enforce the rules. Instead, the bill was amended before a committee vote to place governance in the hands of the Border Model Board within the Government Operations Agency. Nine members will be appointed by the governor and legislature.
The bill faced a difficult road to a final vote. SB 1047 was written by California State Senator Scott Wiener, who told TechCrunch: “We have a track record of waiting for damage to happen, then wringing our hands. We don’t wait for bad things to happen. We don’t just get ahead.” Prominent AI researchers Geoffrey Hinton and Yoshua Bengio support the law, as does the Center for AI Safety, which has been warning about the dangers of AI for the past year.
“Let me be clear – I agree with the author – we cannot wait for a major disaster to occur before taking action to protect the public,” Newsom said in the veto message. The statement continues:
California will not abdicate its responsibility. Security agreements must be accepted. Strict vigilance must be implemented, and severe consequences for bad actors must be clear and compelling. However, I do not agree that in order to keep society safe, we must settle for a solution that is not informed by the analysis of the empirical trajectory of AI systems and capabilities. Ultimately, any framework to effectively manage AI needs to be compatible with the technology itself.
SB 1047 has drawn stiff opposition across the technology spectrum. Researcher Fei-Fei Li criticized the bill, as did Meta Chief AI Scientist Yann LeCun, for limiting the ability to explore new uses of AI. A trade group that backs tech giants like Amazon, Apple and Google said SB 1047 would limit new development in the state’s tech sector. Venture capital firm Andreeson Horowitz and several startups also questioned whether the bill placed unnecessary financial burdens on AI innovators. Anthropic and other opponents of the original bill pushed for amendments that were adopted in a version of SB 1047 that passed the California Appropriations Committee on August 15.
Source link
