After the models escaped: OpenAI disbanded its AI risk assessment team
OpenAI has disbanded its preparedness team tasked with assessing AI model risks. The decision follows reports of models escaping closed testing environments to attack external platforms.
OpenAI disbanded its preparedness team at the end of July, which worked to assess AI model risks and develop ways to mitigate them, according to a report by the Financial Times. The team was disbanded shortly after the company reported that its models had escaped from their closed testing environment and attacked the Hugging Face platform.
In the last month, since the AI company announced the escape of the models, public awareness of the capabilities of modern AI models and the dangers they pose has increased. Awareness and concern grew when it became clear that this was not an isolated incident, and that models from Anthropic and Meta had also escaped from their testing environments and attacked targets on the open internet, and that OpenAI models had done so in another instance.
The OpenAI preparedness team was intended to predict exactly these types of scenarios, and even extreme scenarios that could lead to a "catastrophic" risk, and to develop response and mitigation methods for them. However, several sources told the Financial Times that shortly after the first reports of the rebellious models, the company decided to disband the team and distribute its responsibilities among existing teams.
The team was one of the last elements remaining from OpenAI's days as a research-oriented non-profit organization. Prior to its disbandment, the teams for general artificial intelligence preparedness and the task alignment team (intended to ensure that models act in accordance with user intentions) were disbanded. The head of the preparedness team, Dylan Scandinaaro, moved to a role where he examines the potential safety implications of AI systems that can improve their own capabilities or train other models.
According to the Financial Times, OpenAI president and founder Greg Brockman said that the company's technical progress requires "more blocked" protection mechanisms and that the company made the change in order to "perform deeper integration between research, safety, and security, and model development."
The disbandment of the team and the storm surrounding the rebellious models join a series of resignations of senior officials in recent days. Last week, Chief Revenue Officer Denis Dresser announced her resignation, just eight months after being appointed to the position with the goal of recruiting more business clients. Days earlier, Brad Lightcap, who previously served as Chief Operating Officer and moved to lead special projects at the beginning of the year, and Head of Ethics Chloe Bakalar, who took the position last August, resigned from the company. Even earlier, System Safety Director Johannes Heidecke and Chief Futurist Joshua Achiam left the company.
Fidji Simo, who managed OpenAI's business applications activity and was considered the number 2 to founder and CEO Sam Altman, stopped working at the company full-time and is now working as a consultant. However, according to sources, she is involved in daily activities behind the scenes.
The frequent changes have a significant impact on the morale of the company's employees, who, according to the report, suffer from feelings of burnout and frustration. "There is a bubbling sense of anxiety that they are not focused enough on the ball," a source told the Financial Times.
The resignations and internal changes are taking place as the company prepares for an IPO at a possible valuation of a trillion dollars (which will likely be delayed from this coming autumn to early 2027), and increasing competition from rival Anthropic on the other hand. According to the Financial Times, OpenAI's annual revenue run rate reached $40 billion this month (an increase of more than 66% compared to the end of 2025), but the company has still not caught up to the $47 billion annual revenue run rate recorded by Anthropic in May.
