Jump to content

United States policy on catastrophic AI risk

From AI Law Wiki
Revision as of 10:37, 18 September 2026 by AILawWikiAdmin (talk | contribs) (Add verified September 16 bishops AI-safeguard advocacy and reciprocal coverage)

AI-generated text. This page was generated using artificial intelligence.

United States policy on catastrophic AI risk encompasses measures intended to prevent advanced artificial intelligence from enabling mass harm or escaping effective human control. The approaches range from developer transparency and independent audits to government testing, restrictions on dangerous uses, liability, negotiated slowdowns, and proposed prohibitions on superintelligence. These approaches assign different roles to developers, regulators, courts, and national-security agencies.[1][2][3]

This overview covers selected federal and state measures and policy proposals, with core policy coverage through September 11, 2026 and additional constitutional, insurance, and comparative context reviewed September 12, 2026. Individual developments and proposals are dated below. It distinguishes enacted statutes, future effective dates, executive directions, introduced legislation, announced proposals, and voluntary or scholarly frameworks. Its scope is catastrophic-risk prevention rather than the full range of AI-related consumer, employment, copyright, and civil-rights law.

Meaning and scope of catastrophic harm

In California's Transparency in Frontier Artificial Intelligence Act (SB 53), catastrophic risk means a foreseeable, material risk of a frontier model materially contributing to more than 50 deaths or serious injuries, or more than $1 billion in property damage or loss, in a single incident involving specified conduct. That conduct includes assistance with chemical, biological, radiological, or nuclear weapons; certain autonomous criminal or cyber conduct; and evasion of developer or user control. The definition contains exclusions, including lawful federal-government activity.[4]

The statutory threshold is not synonymous with human extinction. It covers a class of severe incidents; the Sanders–Casar proposal separately addresses superintelligent systems and loss of human control. Neither a legislative definition nor an advocate's prediction establishes how likely a catastrophe is. The California policy report identifies substantial uncertainty and disagreement about advanced-AI risks, and recommends drawing on experiments, simulations, historical comparisons, and adversarial testing as well as observed harms.[3][1]

For prevention, the distinction between misuse and loss of control matters. Misuse policy focuses on people exploiting a model's capabilities; control policy also considers systems circumventing oversight or acting beyond authorized purposes. NIST's draft misuse guidance addresses the former, while California's statute expressly includes evasion of control and certain deceptive behavior outside evaluations.[5][4]

Debate over the framing

The decision to organize policy around catastrophe is itself contested. In a 2023 opinion essay, Alex Hanna and Emily M. Bender argue that extinction-focused messaging can divert attention and resources from documented discrimination, surveillance, and exploitation of workers. They also question the influence of technology companies over the public agenda and the evidentiary basis of sweeping predictions. Their criticism concerns which harms and affected communities receive priority, as well as claims about future capabilities.[6]

The California frontier-policy report approaches uncertainty differently, recommending empirical investigation and adaptable safeguards. These positions expose a policy choice about the allocation of oversight and research resources; neither a catastrophe-focused statute nor this article's narrower scope substitutes for protections against discrimination and other present harms.[1]

Comparison of approaches

Approach Principal intervention Illustrative U.S. measure Status of cited measure
Transparency and safety plans Disclose risks and enforce compliance with developer frameworks California SB 53[7]; New York RAISE Act[8] California: enacted, operative requirements.
New York: enacted; amended principal requirements effective January 1, 2027.
Independent verification External examination of compliance and safety practices Illinois Artificial Intelligence Safety Measures Act (SB 315)[9] Enacted July 6, 2026; Act effective January 1, 2027. Audit duty starts January 1, 2028 or 90 days after first qualifying, whichever is later.[10]
Federal testing and oversight Mandatory risk assessment or supervised testing FRONTIER Act (H.R. 9925)[11]; Secure AI Development Act[12] FRONTIER: introduced bill, July 23, 2026.
Secure AI Development: published legislative proposal, July 2026.
Voluntary national-security review Early government access and defensive cooperation Executive Order 14409[13] Executive order issued June 2, 2026, directing a voluntary framework.
Control and intervention Limit agent access, preserve human authority, and enable shutdown Stop Rogue AI Act[14]; AI Guardrails Act (S. 4113)[15]; AI Kill Switch Act (H.R. 9917)[16] Introduced bills: September 9, March 17, and July 23, 2026, respectively.
Liability and financial incentives Internalize costs through liability and insurance Weil; Schwarcz and Wolff[17][18] Scholarly proposals and critiques; existing insurance markets discussed separately below.
Pacing, pauses, and prohibition Slow or prevent specified capability development Mutually Agreed Pacing Framework[19]; Ban Artificial Superintelligence Act[3] Industry agreement proposed September 4, 2026.
Legislation announced September 3, 2026 as forthcoming.

State laws: transparency, audits, and preventive duties

California: SB 1047 and SB 53

Scott Wiener's 2024 Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, SB 1047, proposed safety protocols, a shutdown capability for systems within its scope, independent compliance audits, and restrictions on use or release where a covered model posed an unreasonable risk of causing or materially enabling critical harm. Its initial coverage combined training-compute and cost thresholds. These were proposed preventive obligations, not merely duties to disclose a risk after an incident.[20]

Governor Gavin Newsom vetoed SB 1047 on September 29, 2024. His objections included the bill's focus on the largest models rather than sufficiently accounting for deployment context or risks from smaller systems. The veto illustrates disagreement over the scope and design of preventive regulation; SB 1047's proposed obligations never became law through that bill.[21]

California subsequently enacted SB 53 on September 29, 2025. The Attorney General describes requirements for large frontier developers to publish and comply with frameworks for catastrophic-risk management, accompanied by transparency and reporting obligations. Covered employees receive protections for specified disclosures about violations or dangers to public health or safety.[7] The resulting mechanism makes compliance with a developer's framework enforceable; it does not reproduce SB 1047's general unreasonable-risk restriction on release.[4][20]

New York: amended RAISE Act

The New York RAISE Act was revised by S8828, signed March 27, 2026. The amendment establishes a transparency regime with published frameworks, model reports, and state oversight, and moves the principal requirements to January 1, 2027. Qualifying critical incidents generally must be reported within 72 hours of the specified determination or knowledge trigger, with a 24-hour route for imminent threats of death or serious injury. The law also requires large developers to file disclosure statements and contribute to oversight costs. Attorney General penalties can reach $1 million for a first violation and $3 million for subsequent violations; the article does not create a private right of action.[8]

Illinois: external audits

The Act takes effect January 1, 2027, but Section 10(a) starts the frontier-AI framework duty on January 1, 2028. Under Section 10(d), annual independent audits begin on January 1, 2028 or 90 days after a developer first qualifies as a large frontier developer, whichever is later.[10]

The Illinois Artificial Intelligence Safety Measures Act (SB 315) adds independent third-party audits to a framework of public safety disclosures, incident reports, compliance processes, and whistleblower protections. The governor's July 6, 2026 signing announcement identifies January 1, 2027 as its effective date. Independent examination is a distinct safeguard because it introduces scrutiny beyond the developer's own public account of its practices.[9]

Federal approaches

Executive action and security cooperation

President Joe Biden's Executive Order 14110 of October 30, 2023 directed Commerce to use Defense Production Act authorities to obtain information from developers of certain dual-use foundation models, including safety-test results and information on protecting model weights. It also directed work on biological, cyber, and critical-infrastructure risks. This was an executive reporting and agency-action approach rather than a comprehensive congressional licensing statute.[22] President Donald Trump revoked that order on January 20, 2025; its requirements should not be presented as the current governing executive order.[23]

America's AI Action Plan, released in July 2025, combines faster innovation with national-security evaluations and targeted defenses. Its recommendations include government evaluation of frontier models for cyber and weapons-related risks, security research, and protection against vulnerabilities in adversaries' systems. Implementation depends on the agencies and legal authorities responsible for each recommendation.[24]

Executive Order 14409 of June 2, 2026 directs agencies to design a voluntary framework permitting government access to covered frontier models for up to 30 days before their planned release to other trusted partners. It also directs classified cyber benchmarking and collaboration on early access for defensive uses. Section 3 expressly does not authorize mandatory government licensing, preclearance, or permitting for model development or release. Separately, the order prioritizes enforcement of existing criminal laws against AI-assisted unlawful computer access.[13]

Licensing, mandatory testing, and federal supervision

The September 2023 Blumenthal–Hawley framework proposed an independent oversight body and licensing for sophisticated general-purpose models or models used in high-risk settings. Licensing would depend on risk management, testing, data governance, and incident-reporting programs. It also proposed audits and legal accountability. It remains a historical framework for legislative design.[2]

The FRONTIER Act, H.R. 9925, was introduced July 23, 2026. Its text combines developer frameworks and reporting with annual independent compliance audits for large frontier developers, an independent-verification structure, and emergency federal intervention provisions. It therefore goes beyond the proposition that companies should voluntarily publish safety principles. The introduced text also addresses the relationship between federal requirements and state law.[11][25]

Senator Mark Warner's July 2026 Secure AI Development Act proposal takes another route: mandatory secure testing before public deployment, coordinated vulnerability handling, stronger information sharing, and a voluntary incident-reporting system modeled on aviation. The published draft would create an Artificial Intelligence Risk Board within NIST. Mandatory testing and voluntary incident reporting are separate parts of that proposal and should not be conflated.[12][26]

Shutdown and intervention powers

Ted Lieu and Nathaniel Moran introduced the AI Kill Switch Act, H.R. 9917, on July 23, 2026. The bill would require covered entities to maintain shutdown and access-control capabilities and empower the Department of Homeland Security, acting through CISA, to order proportionate intervention following a defined covered incident. It includes compliance verification and judicial review. Red-teaming and other structured testing are excluded from the covered-incident definition, so an alarming demonstration in a test environment would not by itself establish that this particular intervention trigger was met.[16]

Federalism and the September 2026 negotiations

The White House's March 2026 legislative recommendations call for federal preemption of burdensome state AI laws, oppose creating a new federal AI rulemaking body, and favor existing sector regulators and industry standards. They also propose preserving specified state powers, including generally applicable consumer protections and a state's own AI procurement. Implementing this preemption agenda would require congressional action.[27]

The FRONTIER Act's introduced § 9 would preempt new substantive state obligations in specified areas of catastrophic-risk transparency, independent verification, and incident reporting, subject to exceptions. Consequently, a national framework's protective effect depends on both the federal duties it creates and the state remedies or requirements it displaces.[11]

September 11 negotiating snapshot. Reuters reported that Senate negotiators were considering a duty of care for the most capable models, federal power to block unsafe models with judicial challenge available, and preemption of some state AI-risk laws. The terms remained unsettled; this dated report should not be read as the current negotiating text. Subsequent developments belong with the linked news coverage and any published bill text.[28] See September 11 coverage.

Constitutional limits on state regulation are distinct from Congress's choice to preempt it. Compelled public disclosures can raise First Amendment questions. In xAI v. Bonta, the district court denied a preliminary injunction against California AB 2013 on March 4, 2026, including the company's compelled-speech challenge. That procedural ruling concerned training-data transparency; it did not adjudicate the constitutionality of SB 53, RAISE, or Illinois SB 315.[29]

Bahrad A. Sokhansanj and Mackenzie Arnold identify implications for frontier transparency laws, emphasizing the legal distinctions among public disclosures, confidential regulatory reporting, and different forms of commercial speech. The durability of a particular duty therefore depends on its text and justification, rather than a general assumption that all AI reporting is either protected from challenge or unconstitutional.[30]

Interstate burdens present a separate potential issue under the dormant Commerce Clause, which limits discriminatory or unduly burdensome state regulation even without federal legislation. However, National Pork Producers Council v. Ross (2023) rejected a broad rule invalidating state laws simply because they influence commerce outside the state. Applying that doctrine to frontier-AI requirements would require analysis of the particular law and record.[31]

Litigation coverage note (September 12, 2026): The sources verified for this overview establish the adjacent AB 2013 litigation, but do not establish a filed challenge directly against SB 53, RAISE, or Illinois SB 315. This is a limit of the reviewed record, not a comprehensive certification that no such case exists.

Technical standards and control of dangerous uses

Voluntary standards and corporate commitments

The NIST AI Risk Management Framework is voluntary guidance for organizations developing, deploying, or using AI. It provides a general risk-management structure rather than a government determination that a particular model is safe.[32] NIST's January 2025 draft misuse guidance offers a more targeted sequence: identify risks, plan responses, protect access, measure and mitigate risks before deployment, monitor misuse, and disclose practices. It cautions that safeguards can be brittle and that following the practices cannot guarantee prevention of misuse.[5]

Corporate policies can link capability assessments to stronger safeguards, but their content and strength can change. Anthropic's February 2026 explanation of Responsible Scaling Policy version 3.0 separates company plans from industry-wide recommendations and describes its Frontier Safety Roadmap goals as nonbinding public targets. It also introduces recurring risk reports and external review in specified circumstances. This illustrates why a corporate roadmap, an operational commitment, and a legally required compliance framework need separate treatment.[33]

AI agents and military applications

The Stop Rogue AI Act, whose introduction Gottheimer's office announced on September 9, 2026, would direct NIST to develop standards for discovering, identifying, monitoring, and controlling agents on organizational networks. The announcement emphasizes inventories, verifiable provenance, detection of unauthorized behavior, and the ability to deny or revoke access. This targets the permissions and environment in which an agent acts, rather than only the capabilities of its underlying model.[14]

The AI Guardrails Act of 2026, S. 4113, instead addresses Department of Defense use. Its introduced text would prohibit AI execution of nuclear launch or detonation, restrict certain domestic surveillance, and require appropriate human judgment and supervision for autonomous lethal force. It includes a conditional waiver mechanism for the autonomous-weapons restriction. A military-use proposal of this kind is distinct from a general civilian frontier-model safety law.[15]

Biosecurity and infrastructure defenses

Another approach intervenes at the practical steps needed to turn harmful information into physical harm. The July 2025 AI Action Plan recommends requiring federally funded scientific institutions to use nucleic-acid synthesis providers and tools with sequence screening and customer verification, backed by enforcement. It also recommends information sharing to identify malicious customers. These are downstream biosecurity interventions alongside model evaluations.[24]

For cyber risks, Executive Order 14409 directs a government–industry clearinghouse to coordinate vulnerability discovery, remediation, and patches, and measures to expand defensive tools for critical infrastructure. This illustrates a prevention strategy focused on making potential targets harder to compromise as AI capabilities increase.[13]

Expert proposals for new institutions

Cass Sunstein's September 11, 2026 essay sketches a dedicated AI Regulatory Commission: a multimember body with investigative powers, periodic company disclosures, civil penalties, rulemaking, and authority to act against imminent or catastrophic threats. Licensing and industry funding remain open design questions. This preliminary institutional proposal complements the earlier licensing debate but has no legislative status. Sunstein favors insulation from political pressure while recognizing constitutional obstacles to agency independence.[34]

In a September 11, 2026 New York Times opinion essay, Stephen Witt advocates a temporary slowdown while permanent oversight is established. He reports Yoshua Bengio's support for compulsory incident investigation modeled on the National Transportation Safety Board and Daniel Kokotajlo's proposal for an international public database of training runs and their locations. Witt argues for powers to obtain evidence and compel testimony, public scrutiny of development, and enforceable shutdown capabilities. These are proposals discussed in an opinion essay; the AI Kill Switch Act's specific legislative mechanism is described under shutdown and intervention powers above. Witt also acknowledges economic disruption and foregone benefits as costs of restrictions.[35]

Liability, insurance, and public investment

Liability incentives and their limits

Liability proposals seek to make developers and deployers bear costs otherwise imposed on people outside the commercial transaction. Gabriel Weil argues that strict liability can create continuing incentives for precautions, including precautions that a regulator has not anticipated. His case differs from a negligence standard focused on whether a defendant took reasonable care and from a licensing system centered on permission before deployment. He presents this as a reform to the allocation of legal responsibility.[17]

Insurance could strengthen those incentives if coverage and premiums reward effective safety measures. Daniel Schwarcz and Josephine Wolff question whether insurers can do that reliably for frontier AI: relevant loss data may be sparse, risks change quickly, and the effectiveness of safeguards is difficult to assess. Their critique concerns the ability to price and reduce risks, not merely the formal availability of a damages claim.[18]

Weil also identifies limits: catastrophic losses can exceed realistically collectible compensation; some harms may not produce useful earlier warning incidents; foreign enforcement can be difficult; and government conduct is less responsive to ordinary liability incentives. He argues that liability cannot by itself solve underinvestment in broadly useful safety research, for which public subsidies or prizes may be appropriate. Compensation, deterrence, and prevention before an irreversible disaster are therefore different policy objectives.[36]

Insurance regulation and market practice

Insurance regulation already addresses insurers' own use of AI. The NAIC adopted a model bulletin in December 2023 setting expectations for written governance programs, risk management, lawful consumer outcomes, and examination records. It is guidance for state regulators rather than a model statute.[37] New York's July 2024 Circular Letter No. 7 sets supervisory expectations for AI and external data in underwriting and pricing, including discrimination testing, actuarial validity, senior oversight, and vendor diligence.[38] These measures govern insurance decisions; they do not require insurers to cover frontier-model catastrophes.

Commercial coverage provides another, narrower connection. Aon's 2026 guidance maps AI exposures across cyber, technology errors and omissions (E&O), professional indemnity, and directors and officers (D&O) insurance, among other lines. Different policies address different losses and claims; D&O, for example, may respond to allegations about management representations or oversight.[39] Marsh emphasizes that the facts of a loss, policy language, and exclusions determine the response, while limited loss data complicate pricing.[40]

AI-specific products also exist: Munich Re's aiSure offers protection associated with AI performance commitments and errors, describing a focus on ordinary business losses rather than catastrophic loss.[41] Such products demonstrate a market for bounded AI exposures, but their availability does not establish reliable pricing or sufficient capacity for correlated frontier-model catastrophes. That gap is central to the Schwarcz–Wolff critique.[18]

Pacing agreements, pauses, and bans

On September 13, 2026, President Trump responded to calls for an AI slowdown by emphasizing competition with China and acknowledging possible safeguards without specifying new rules.[42][43] See September 13 coverage.

On September 13, 2026, House Speaker Mike Johnson opposed an emergency AI moratorium and proposed industry–government talks on safeguards, citing competition with China.[44] See September 13 coverage.

On September 12, CNN reported Dario Amodei’s pacing proposal and Anthropic’s commitment to embedded external evaluators.[45]

On September 12, 2026, OpenAI chief executive Sam Altman endorsed Amodei’s pacing call and said OpenAI would also give independent evaluators access comparable to employees, with further details to follow. This was an announced commitment, not confirmation that the arrangement was already operating.[46][45]

On September 12, Clément Delangue announced the Open Alignment Initiative, led by Thomas Wolf at Hugging Face, and requested participation in the embedded-evaluator program. He argued for broader participation and transparency in alignment work; the announcement does not confirm acceptance into the program.[47] See September 12 coverage.


On September 4, 2026, Scott Wiener and other state lawmakers called for a Mutually Agreed Pacing Framework: an agreement among frontier laboratories, independently verified, to pace capability development while safety work catches up. The announcement expressly presented this as additional to state, federal, and international action. Laboratory participation and an enforceable agreement would require further negotiation.[19]

The Ban Artificial Superintelligence Act announced by Bernie Sanders and Greg Casar on September 3, 2026 proposes a more restrictive model: a permanent superintelligence prohibition, a temporary pause on advanced-AI development pending a new regulator and safety rules, and international efforts to prevent superintelligence development. The sponsors described the legislation as forthcoming. The distinction is substantive: a conditional slowdown leaves a route to further development, while a permanent prohibition closes that route for the defined class of systems.[3]

International coordination also appears in scholarly proposals. Simon Goldstein and Peter Salib propose a joint U.S.–China frontier laboratory to reduce incentives for a destabilizing race. They offer this as a scholarly institutional design. It highlights the separate question of how a domestic safety regime would work when capable development can occur abroad.[48]

International comparison: the European Union

For the companion overview, see European Union policy on catastrophic AI risk.

The EU AI Act combines several regulatory layers. Its prohibited-practice rules address specified unacceptable uses, while its high-risk system regime focuses on applications such as employment and access to essential services. Those categories cover harms much broader than catastrophe. The closer comparator for U.S. frontier-model policy is the Act's separate regime for general-purpose AI models with systemic risk: providers face additional duties to assess and mitigate those risks, with supervision by the European AI Office. General-purpose model obligations began applying in August 2025, with AI Office enforcement powers applying from August 2026, subject to the Act's transitional arrangements. A voluntary code of practice supports compliance with underlying legal obligations. This structure combines application-specific protections with model-level systemic-risk oversight, rather than treating either category as a substitute for the other.[49]

Management-based regulation

Cary Coglianese and Colton R. Crum advocate adaptable risk-management duties with continuing human oversight, rather than relying principally on fixed technical prescriptions. Their 2025 scholarship concerns AI risks broadly and offers a regulatory-design comparison for frontier-AI oversight; it is not an enacted requirement or a demonstrated safety guarantee.[50]

Central design questions

The approaches differ on what evidence should trigger intervention and who must act on it. The California policy report recommends adaptable thresholds and independent evidence, while warning that thresholds are imperfect. Its analysis supports examining deployment context and revising metrics as capabilities change, rather than treating a numerical model-size threshold as a complete measure of danger.[1]

They also differ on whether a failed assessment produces a legal duty to stop, a duty to mitigate, a reporting obligation, or a voluntary decision. SB 1047's vetoed release restriction, SB 53's framework-compliance duties, and Executive Order 14409's voluntary federal access illustrate materially different consequences. An assessment requirement alone does not establish which consequence follows.[20][7][13]

Finally, accountability requires attention to institutional capacity and incentives. The Blumenthal–Hawley proposal addresses conflicts of interest in an oversight body; Illinois emphasizes independent audits; and the liability debate asks whether courts and insurers can identify and deter risks effectively. These mechanisms can be combined, but none should be described as a demonstrated guarantee against catastrophic harm.[2][9][18][5]

September 14, 2026: response from China

On September 14, 2026, China’s foreign ministry urged international cooperation on AI in response to Anthropic chief executive Dario Amodei’s proposal to retain US restrictions on advanced chips and chipmaking equipment. Spokesperson Guo Jiakun said confrontation and fear-based competition would undermine global AI governance.[51]

See September 14 digest for the response and the proposal’s international-coordination context.

Microsoft model-conduct consultation

On September 14, 2026, Microsoft AI proposed a code for its own models centered on human control, correction and shutdown. It opened six weeks of consultation before revising the training framework. These are company policy proposals.[52] See September 14 coverage.

September 14, 2026: Trump on AI safeguards

President Trump argued that existing criminal and regulatory powers and presidential leadership were sufficient safeguards, while warning that opposition to AI development would benefit China.[53] His statements expressed opposition to further controls; they did not themselves create a new regulatory instrument.[54] See September 14 coverage.

September 14, 2026: Obama and Harris on oversight

Obama called for wider public participation in AI decisions.[55] Harris urged legislation, a federal oversight and testing entity, and an international treaty on dangerous uses, development speed and testing.[56] See September 14 digest coverage for these policy proposals.

September 15, 2026: competing approaches in Congress

On September 15, 2026, Sanders’s prepared remarks called for binding international safety rules and a US–China treaty, and announced plans to introduce the Sanders–Casar superintelligence-ban proposal the following week.[57] Daines opposed slowing development in competition with China and advocated discussion involving private-sector experts and public officials.[58] Johnson again favored industry self-regulation and opposed a moratorium, while Senate negotiations remained unresolved, according to Roll Call.[59] See September 15 digest coverage.

RAND proposal: preserving strategic options

On September 15, 2026, RAND published Joel B. Predd and coauthors' A U.S. Strategy to Secure Geopolitical Advantage on an Uncertain Path to Superintelligence: Maintaining Freedom of Action. The authors propose preserving the ability to choose among competing strategies as evidence develops, while pursuing U.S. geopolitical advantage and human survival and agency. This is an expert policy proposal, not an adopted government strategy.[60]

The paper identifies five unresolved questions: the proximity of danger, human–AI coexistence, the feasibility of cooperative restraint, the possibility of decisive advantage, and the ability to suppress rival programs. It argues that both premature commitment and excessive delay can eliminate options. Its recommendations combine investment in human–AI coexistence, AI security, adaptation of national-security institutions, and societal capacity to respond, with monitoring of technological and strategic changes. The authors advocate making commitments when evidence supports them or further delay would close an option.[61]

September 16, 2026: House members press for legislation

On September 16, 2026, ten bipartisan House members led by Don Beyer urged House leaders to accelerate AI oversight, security and transparency legislation, including independent oversight and stronger monitoring. Their letter requested legislative action; it did not enact requirements.[62] Four Democratic members also urged shortening or cancelling the forthcoming recess to work on safeguards.[63]

See September 16 digest coverage.

September 16, 2026: Warren calls for a pause

On September 16, 2026, Senator Elizabeth Warren called for an immediate pause in advanced AI development while lawmakers and regulators establish safeguards. She urged enforcement of existing laws and new legislation, rejected industry-led pacing as insufficient and opposed weakening antitrust protections. This was a policy statement, not an enacted pause.[64]

Axios reported that her position stopped short of endorsing Senator Bernie Sanders’s proposed ban on superintelligence, distinguishing the two approaches.[65]

See September 16 digest coverage.

September 17, 2026: Shapiro and transparency measurements

On September 17, 2026, Pennsylvania Governor Josh Shapiro called for federal AI regulation and international cooperation, including engagement with China, at the AI Horizons Summit. His prepared remarks advocate independent third-party oversight of frontier models, transparency and safety measures. He rejected a choice between unchecked development and an indefinite halt. This is advocacy in the AI-safety policy debate, rather than a new law or executive order.[66][67]

On September 17, 2026, Anthropic published proposed measurements of AI-led research, agent oversight and computing-resource allocation to inform public scrutiny of frontier development. Its August snapshot says Claude leads 26% of measured research-and-development work under human supervision; no measured subset is fully autonomous. The company proposes external verification and acknowledges methodological limits. This is a voluntary transparency initiative relevant to the debate over pacing AI development, not a regulatory requirement.[68][69] See Shapiro coverage and Anthropic coverage.

September 16, 2026: bishops advocate safeguards

The U.S. Conference of Catholic Bishops created an AI task force chaired by Bishop Oscar Cantú on September 16, 2026. More than a dozen bishops also met members of Congress to advocate AI safety legislation and online protections for children. This was a civil-society advocacy initiative, not government action.[70][71] See September 16 digest coverage.

References

  1. ↑ 1.0 1.1 1.2 1.3 Joint California Policy Working Group on AI Frontier Models, The California Report on Frontier AI Policy, June 17, 2025, executive summary and sections 1–3.
  2. ↑ 2.0 2.1 2.2 Senators Richard Blumenthal and Josh Hawley, Bipartisan Framework for U.S. AI Act, September 2023.
  3. ↑ 3.0 3.1 3.2 3.3 Office of Senator Bernie Sanders, Sanders, Casar to Introduce Legislation to Ban Artificial Superintelligence and Temporarily Pause Advanced AI Development, September 3, 2026.
  4. ↑ 4.0 4.1 4.2 California Legislature, SB 53, Chapter 138, Statutes of 2025, approved September 29, 2025, especially Business and Professions Code §§ 22757.11–22757.16 and Labor Code §§ 1107–1107.4.
  5. ↑ 5.0 5.1 5.2 NIST, Managing Misuse Risk for Dual-Use Foundation Models, NIST AI 800-1, second public draft, January 2025, sections 4–5. Cited as draft guidance, not a binding rule.
  6. ↑ Alex Hanna and Emily M. Bender, Scientific American, August 12, 2023, “AI Causes Real Harm. Let’s Focus on That over the End-of-Humanity Hype”.
  7. ↑ 7.0 7.1 7.2 California Department of Justice, Catastrophic Risks in Artificial Intelligence Foundation Models, official SB 53 guidance and employee-reporting information, accessed September 11, 2026.
  8. ↑ 8.0 8.1 New York Senate, S8828, 2025–2026 session, signed March 27, 2026, Chapter 96; text of General Business Law Article 44-B and §§ 3–5 of the amending act.
  9. ↑ 9.0 9.1 9.2 Office of Governor JB Pritzker, Gov. Pritzker Signs Nation-Leading Artificial Intelligence Safety Law, July 6, 2026.
  10. ↑ 10.0 10.1 Illinois General Assembly, Public Act 104-0538, Sections 10(a), 10(d) and 99; approved July 6, 2026. Enacted text reviewed September 12, 2026.
  11. ↑ 11.0 11.1 11.2 U.S. Congress, H.R. 9925, FRONTIER Act, introduced July 23, 2026, especially §§ 4–9.
  12. ↑ 12.0 12.1 Office of Senator Mark Warner, Warner Rolls Out Comprehensive AI Legislative Agenda Focused on Responsible Innovation, Workers, and National Security, July 21, 2026.
  13. ↑ 13.0 13.1 13.2 13.3 White House, Promoting Advanced Artificial Intelligence Innovation and Security, Executive Order 14409, June 2, 2026, §§ 2–4.
  14. ↑ 14.0 14.1 Office of Representative Josh Gottheimer, Gottheimer Introduces Bipartisan Bill to Stop Rogue AI Agents and Keep People in Control, September 9, 2026.
  15. ↑ 15.0 15.1 U.S. Congress, S. 4113, AI Guardrails Act of 2026, introduced March 17, 2026, § 2.
  16. ↑ 16.0 16.1 U.S. Congress, H.R. 9917, AI Kill Switch Act, introduced July 23, 2026, § 2, proposed Homeland Security Act § 2220F(b)–(g). Introduced text, not an enacted requirement.
  17. ↑ 17.0 17.1 Gabriel Weil, The Case for AI Liability, Institute for Law & AI, June 2025.
  18. ↑ 18.0 18.1 18.2 18.3 Daniel Schwarcz and Josephine Wolff, The Limits of Regulating AI Safety Through Liability and Insurance, Institute for Law & AI, October 2025, sections on data, risk assessment, and catastrophic risk.
  19. ↑ 19.0 19.1 Office of Senator Scott Wiener, Lawmakers Behind State AI Safety Laws Call For Industry-Wide AI Safety Pact to Protect the Public, September 4, 2026.
  20. ↑ 20.0 20.1 20.2 California Legislature, SB 1047, Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, enrolled September 3, 2024, proposed Business and Professions Code §§ 22602–22603.
  21. ↑ Office of Governor Gavin Newsom, SB 1047 veto message, September 29, 2024.
  22. ↑ Executive Order 14110, Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, October 30, 2023, 88 Fed. Reg. 75191, especially §§ 4.1–4.4.
  23. ↑ White House, Initial Rescissions of Harmful Executive Orders and Actions, January 20, 2025, § 2.
  24. ↑ 24.0 24.1 White House, America's AI Action Plan, July 2025, especially pp. 20–23.
  25. ↑ U.S. Government Publishing Office, H.R. 9925 (IH): Content Details, introduction and referral record, July 23, 2026.
  26. ↑ Office of Senator Mark Warner, Secure AI Development Act, published legislative draft BAG26E21, July 2026, especially §§ 3–5.
  27. ↑ White House, National Policy Framework for Artificial Intelligence: Legislative Recommendations, March 20, 2026, parts V and VII.
  28. ↑ Courtney Rozen, Reuters, US Senate negotiators consider requiring AI firms to mitigate known major risks, September 11, 2026, republished by WSAU.
  29. ↑ U.S. District Court for the Central District of California, March 4, 2026, Order denying preliminary injunction, X.AI LLC v. Bonta, No. 2:25-cv-12295-JGB-SSC, document 35.
  30. ↑ Bahrad A. Sokhansanj and Mackenzie Arnold, Lawfare, June 22, 2026, “First Amendment Questions for AI Transparency Laws”.
  31. ↑ Congressional Research Service, Constitution Annotated, 2024 Supplement, pp. 51–52.
  32. ↑ NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 26, 2023.
  33. ↑ Anthropic, Responsible Scaling Policy: Version 3.0, February 24, 2026, explanation of the revision. This citation describes that dated revision rather than asserting that it is the latest policy version.
  34. ↑ Cass Sunstein, The AI Regulatory Commission: Needed Now, Cass’s Substack, September 11, 2026. Author's preliminary institutional-design proposal.
  35. ↑ Stephen Witt, September 11, 2026 opinion essay on AI safety and regulation, The New York Times.
  36. ↑ Gabriel Weil, The Limits of Liability, Institute for Law & AI, August 2024.
  37. ↑ NAIC, December 4, 2023, “NAIC Members Approve Model Bulletin on Use of AI by Insurers”.
  38. ↑ New York Department of Financial Services, July 11, 2024, Insurance Circular Letter No. 7 (2024).
  39. ↑ Aon, 2026, AI Fact Sheet 2026, insurance mapping.
  40. ↑ Marsh, June 2, 2025, “Debunking Generative AI myth #3: GenAI insurance issues”.
  41. ↑ Munich Re, accessed September 12, 2026, “Insure AI” / aiSure product information.
  42. ↑ Meridith McGraw, White House travel pool report 18, September 13, 2026, remarks at Doonbeg. Contemporary reporter notes, archived by White House Pool.
  43. ↑ Darlene Superville and Didi Tang, Associated Press, “Trump downplays the need to check AI development and says he doesn’t want to cede edge to China,” September 13, 2026.
  44. ↑ CNN, State of the Union, interview with House Speaker Mike Johnson, September 13, 2026, 9 a.m. ET. Rush transcript.
  45. ↑ 45.0 45.1 Auzinea Bacon, CNN via KEYT, “Anthropic CEO calls for ‘pacing the frontier’ of AI race amid safety concerns,” September 12, 2026.
  46. ↑ Sam Altman, public statement on frontier pacing and independent evaluators, X, September 12, 2026. Full public post reviewed September 12, 2026.
  47. ↑ Clément Delangue, announcement of the Open Alignment Initiative, X, September 12, 2026. Public post reviewed September 12, 2026.
  48. ↑ Simon Goldstein and Peter N. Salib, The Case for a Joint U.S.-China AI Lab, Lawfare, April 23, 2025.
  49. ↑ European Commission, accessed September 12, 2026, “AI Act” — regulatory framework and implementation timeline.
  50. ↑ Cary Coglianese and Colton R. Crum, Leashes, not guardrails: A management-based approach to artificial intelligence risk regulation, Risk Analysis (2025), DOI 10.1111/risa.70020. Accessed September 12, 2026.
  51. ↑ E. Eduardo Castillo and Chan Ho-him, Associated Press, “Beijing hits back at Anthropic CEO’s call to curb China’s AI development,” September 14, 2026.
  52. ↑ Microsoft AI, “Humanist AI in practice: A public consultation on our Code of Conduct for MAI Models,” September 14, 2026.
  53. ↑ Bhargav Acharya and Katharine Jackson, Reuters via WMBD, “Trump says there is a sick conspiracy against AI, data centers,” September 14, 2026.
  54. ↑ Josh Boak, Associated Press via WTVM, “Trump dismisses new AI guardrails, says there is a ‘SICK conspiracy’ against AI and data centers,” September 14, 2026.
  55. ↑ Barack Obama, public statement on AI development and public participation, September 14, 2026.
  56. ↑ Kamala Harris, public statement on frontier-AI oversight and international cooperation, September 14, 2026.
  57. ↑ Office of Senator Bernie Sanders, prepared remarks for the Pro-Human Assembly, September 15, 2026.
  58. ↑ Office of Senator Steve Daines, transcript excerpts from his Fox News interview on AI policy, September 15, 2026.
  59. ↑ Allison Mollenkamp and Savannah Behrmann, Roll Call, “AI threats confront a Congress far from erecting guardrails,” September 15, 2026.
  60. ↑ RAND, A U.S. Strategy to Secure Geopolitical Advantage on an Uncertain Path to Superintelligence: Maintaining Freedom of Action, published September 15, 2026, PE-A5105-1, DOI: 10.7249/PEA5105-1.
  61. ↑ Joel B. Predd et al., Maintaining Freedom of Action, RAND, September 2026, summary, p. v. Recommendations and strategic judgments are attributed to the authors.
  62. ↑ Office of Representative Don Beyer, “Bipartisan Delegation Urges House Leaders To Take Immediate Action On Artificial Intelligence,” September 16, 2026, including the letter’s full text.
  63. ↑ Alexandra Kelley, Nextgov/FCW, “House Democrats call to cancel upcoming recess for time to legislate on AI,” September 16, 2026.
  64. ↑ Office of Senator Elizabeth Warren, “Senator Warren Calls for Pause in Advanced AI Development,” September 16, 2026.
  65. ↑ Maria Curi, Axios, “Exclusive: Warren backs pause on advanced AI,” September 16, 2026.
  66. ↑ Governor of Pennsylvania, remarks as prepared for delivery at the AI Horizons 2026 Summit, September 17, 2026.
  67. ↑ Kiley Koscinski and Chris Potter, WESA, “Shapiro calls on Trump to regulate artificial intelligence development,” September 17, 2026.
  68. ↑ Anthropic, “Measurements for understanding the pace of AI development inside frontier labs,” September 17, 2026.
  69. ↑ Reuters via Investing.com, “Anthropic says Claude now leads a quarter of work building its next AI models,” September 17, 2026.
  70. ↑ USCCB, “U.S. Bishops’ Administrative Committee Establishes a Task Force on Artificial Intelligence,” September 16, 2026.
  71. ↑ Daniel Payne, EWTN News, “U.S. bishops launch task force to promote human dignity amid rise of artificial intelligence,” September 16, 2026.
Retrieved from "https://ailawwiki.com/index.php?title=United_States_policy_on_catastrophic_AI_risk&oldid=4786"