California Gov. Gavin Newsom Signs Bill Creating AI Safety Measures

"The move comes as Newsom touted California as a leader in AI regulation and criticized the inaction at the federal level in a recent conversation with former President Bill Clinton. The new law will establish some of the first-in-the-nation regulations on large-scale AI models without hurting the state's homegrown industry, Newsom said. Many of the world's top AI companies are located in California and will have to follow the requirements."

"The legislation requires AI companies to implement and disclose publicly safety protocols to prevent their most advanced models from being used to cause major harm. The rules are designed to cover AI systems if they meet a "frontier" threshold that signals they run on a huge amount of computing power. Such thresholds are based on how many calculations the computers are performing. Those who crafted the regulations have acknowledged the numerical thresholds are an imperfect starting point to distinguish today's highest-performing generative AI systems from the next generation that could be even more powerful."

California enacted a law to prevent use of powerful AI models for catastrophic activities such as creating bioweapons or disrupting financial and infrastructure systems. The law requires companies running large-scale models to implement and publicly disclose safety protocols and to meet a "frontier" threshold based on computational calculations. The thresholds are acknowledged as imperfect for distinguishing current high-performing generative systems from more powerful future models. The law defines catastrophic risk as at least $1 billion in damage or more than 50 injuries or deaths and aims to guard against mass disruptions like power-grid hacking. Many top AI companies in California will be subject to the requirements. Companies must report critical safety incidents to the state.

Read at SecurityWeek