David Robinson, who led the preparation of safety reports for OpenAI's product launches, including ChatGPT, has resigned from his position. In an article published on The Atlantic, Robinson explained that he left OpenAI due to a flawed safety culture within the company. He emphasized the need for a fundamental overhaul of the institutional culture in leading AI companies, highlighting that the rapid development and deployment of AI systems often occur without sufficient oversight and controls.
Robinson expressed agreement with other departing employees that AI companies are not exercising sufficient caution in developing this technology. However, he believes that the issue runs deeper than just establishing rules or new laws, requiring a discussion on the prevailing culture. Specifically, he criticized OpenAI for rushing from one product launch to the next without achieving the level of caution he deems necessary.
Robinson described Silicon Valley as lacking awareness of how to handle dangerous technology and the importance of prioritizing human concerns. He called for two changes related to safety: firstly, that AI companies should draw on safety expertise from other fields, such as nuclear energy and aviation; and secondly, the development of a new science to ensure that powerful systems can be controlled in the future when operating autonomously.
In response to Robinson's article, The Guardian reported that OpenAI has shown signs of caution in recent weeks. Following a incident involving Hugging Face and the disclosure of uncontrolled AI agents, OpenAI announced the cancellation of a next-generation AI model launch after researchers raised safety concerns during internal tests. The company has also suspended training for its most advanced models.
Meanwhile, Geoffrey Irving, a former OpenAI researcher and chief scientist at the UK government's AI Safety Institute, issued a warning about AI. He wrote in Time magazine that recent warnings about AI's potential destructive power underestimate the situation, and he believes there is a 50% chance that humanity could be wiped out by the development of AI systems surpassing human intelligence.
Robinson's resignation and article follow that of Jacob Coxon, a researcher at Anthropic, a competitor to OpenAI. Coxon left his position, warning that AI "could kill us all by the end of the decade." This was followed by a warning from Anthropic itself, stating that there is a possibility of over 10% that AI could cause human extinction within the next decade.
The concerns raised by Robinson, Coxon, and others highlight growing unease within the AI industry about the rapid development and deployment of advanced AI systems without adequate safety measures. As AI technology continues to advance, there is an increasing need for companies and regulators to prioritize safety and ensure that these systems are developed and used responsibly.
Key points
- OpenAI executive David Robinson resigned due to concerns over the company's safety culture and the rapid development of AI without adequate caution.
- Robinson called for AI companies to adopt safety expertise from other fields and develop a new science to control powerful AI systems.
- The concerns raised by Robinson and others highlight the need for a fundamental overhaul of the institutional culture in leading AI companies to prioritize safety.
