The New York City Council held a landmark oversight hearing on <a href="https://myappsplus.com/artificial-intelligence-and-the-future-of-work/" title="Artificial intelligence and the future of work”>artificial intelligence Monday.
The city says it is the first in the nation, and world, to hold such a hearing.
City Council Speaker Julie Menin said that New York City needs to take matters into its own hands because the federal government said AI companies can self-regulate.
“Both New York State and the state of California have both talked about potentially a kill switch. That’s actually in our legislation as well. And that’s one of the questions we have today for the companies. How would a kill switch work? Would that be effective?” Menin said prior to the hearing.
The testimony includes whistleblowers like former Anthropic employee Jacob Coxon, who previously said AI could destroy humanity by 2030.
Coxon had a warning for the City Council.
“On the current path, I think it is more likely than not that humanity loses control to these AIs and it could end in human extinction,” Coxon said. “From my experience, the companies are being extremely reckless given the stakes.”
Coxon also said AIs may soon be capable of “improving themselves.”
“We’re approaching the point at which the AI systems will be capable of improving themselves, so, doing the research that humans were doing previously,” Coxon said.
Menin took those concerns to executives who testified virtually from Anthropic, OpenAI, Google and Meta. Alongside them was a poster of more than a dozen AI security incidents this year from their companies.
“If one of your frontier AI models goes rogue and causes serious financial harm, compromises sensitive data, causes physical injury or contributes to a death or deaths, do you each believe your company bears legal responsibility?” Menin asked.
“I believe that OpenAI takes its responsibility for safety very seriously. And we are responsible for developing and evaluating our systems to ensure that they are safe,” OpenAI’s Morgan Dwyer said.
“So, is that a yes? Or that is a no?” Menin said.
One company not in attendance is SpaceX, which is owned by Elon Musk. Menin said after a subpoena was sent to the company, it responded with a letter saying they want to work with the city, but they won’t be at the hearing. The city is now pursuing legal action. SpaceX did not return a request for comment.
The City Council is considering 10 bills that will regulate AI in New York City, including prohibiting the unauthorized depiction of public officials by AI, third-party validation, and the ability for a human to shut down AI models, as well as chat box data privacy.
“Companies should need your affirmative consent before using your conversations to train models,” City Councilmember Frank Morano said.
In a letter to the City Council ahead of Monday’s hearing, an OpenAI executive wrote in part, “We test models for dangerous capabilities and concerning behavior, strengthen security, and monitor for failures of our safeguards.”
Meta said, “At Meta, we are working to make personal superintelligence widely accessible so that it empowers people … and safety is core to everything we do.”
