California weighs tougher AI safeguards as industry wealth grows
California Gov. Gavin Newsom has released recommendations to strengthen oversight of AI companies amid concerns about financial exposure, cybersecurity and increasingly autonomous systems. The proposals follow Anthropic’s stock market debut, which reportedly gave nearly 600 employees stock-option gains exceeding $100 million each. Fears of an AI investment bubble have raised concerns about potential losses for pension and retirement funds.
Elon Musk’s distancing from President Trump’s MAGA movement adds a political dimension to questions about industry leadership. Musk said he felt “foolish” for bankrolling Republicans with almost $500 million in the midterms. Separately, California Atty. Gen. Rob Bonta’s lawsuit against OpenAI alleges the company lost control of its models in testing and development at least 80 times and continued testing without adequate safeguards. Another concern is recursive self-improvement, in which AI systems develop more powerful successors without human intervention.
Newsom’s recommendations include independent monitoring of baseline safety standards, disclosures about recursive AI development and funding for CalCompute to give California computing capabilities independent of private companies. Proposed safeguards also include faster reporting of serious incidents, testing whether models can be shut down and state authority to order emergency shutdowns. Nathan Calvin of AI safety organization Encode urged swift legislative action, saying accelerating automation makes stronger protections urgent. The recommendations still require action by lawmakers.