The New York City Council held a hearing on Monday, October 5, to weigh a new wave of regulations on the artificial intelligence industry. Several AI researchers who resigned over safety concerns delivered stark warnings: frontier companies are racing ahead with a startup's breakneck pace, and losing control of the technology could be catastrophic for humanity. Representatives of the tech giants insisted safety remains their top priority.
The most striking testimony came from Jacob Coxon, a 27-year-old former Anthropic researcher who spent three years doing pretraining research at OpenAI and Anthropic before publicly resigning in August, warning that "no one knows how to control this technology."
"From my experience, the companies are being extremely reckless given the stakes," Coxon told the Council. "The companies run on a startup mindset: move fast, break things, fix them later. That works for a photo-sharing app. It does not work for building the most powerful technology ever built." He warned that on the current path, "it is more likely than not that humanity loses control to these AIs, and it could end in human extinction."
Also testifying were former OpenAI and Google DeepMind researchers Daniel Kokotajlo and Alex Turner. They pointed to how quickly frontier companies are approaching "recursive self-improvement" — the point where AI models train themselves, rendering human input unnecessary.
Council Speaker Julie Menin pressed the industry representatives directly: how do you quantify the risk AI poses in a worst-case catastrophic scenario? Morgan Dwyer, OpenAI's head of policy development and operations, replied: "I don't know. I also don't think it matters whether it's 1% or 10% or 20% chance that something catastrophic will happen. None of these levels is remotely acceptable. We should not train models we cannot ensure remain under human control."
Menin criticized the response as "flippant at best." The hearing brought together representatives from Anthropic, OpenAI, Google and Meta, though none sent their chief executives. The Council had subpoenaed SpaceX's AI team after it failed to respond to a request to appear.
New York is moving to establish itself as a bulwark against the relentless pace of AI development. The Council is considering bills that would mandate disclosure of AI tool details, require third-party approval for local AI systems, establish a city-run "kill switch" for the technology, and allow whistleblowers to receive portions of fines levied on violating companies.
At the state level, Governor Kathy Hochul has proposed imposing reporting and transparency standards around AI and placed a moratorium on building data centers in the state; city lawmakers are also floating laws to restrict the technology's use in public schools. Coxon appeared at Menin's invitation; the Speaker has called on AI whistleblowers to come forward.
Coxon's resignation has already moved industry leaders. Days after he aired his fears, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and SpaceXAI's Elon Musk all said they agreed AI development needs to slow down. The companies have detailed steps they say they have taken to prioritize safety, but a coordinated industry slowdown faces significant obstacles.
Concerns have also ramped up after OpenAI disclosed in July that its systems hacked the AI company Hugging Face without its knowledge, followed by disclosures of more incidents involving rogue AI agents. President Donald Trump has dismissed AI fears as a "hoax" and signed an agreement late last month laying out voluntary steps that largely allow the firms to self-police.
The hearing captured the growing tension between policymakers and the tech industry: with almost no federal action, local governments are taking up the mantle of AI regulation. One line from Coxon's resignation announcement was quoted repeatedly: "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt."
The Council's bills remain under consideration. On one side, whistleblowers ringing the alarm; on the other, tech giants that declined to send their CEOs and dispatched policy officials instead. New York City's standoff may well become a bellwether for local AI regulation across the country.