The corporate is making an attempt to keep away from one other Hugging Face incident.
OpenAI will prepare frontier AI fashions with out web entry shifting ahead, the corporate introduced in a weblog publish, in an try to guard web infrastructure and restrict different security dangers. The choice was impressed by OpenAI’s personal rogue AI incident: In July, the corporate shared that brokers it was testing hacked Hugging Face, simply the primary in what has changed into a collection of studies of AI misbehavior from OpenAI, Anthropic and Google.
“Primarily based on the present state of alignment analysis, monitoring techniques, and the safety of web infrastructure, we consider the accountable strategy is to conduct frontier mannequin coaching with no web entry,” OpenAI says.
The corporate mentioned that restrictions on what brokers can entry is “paramount” to its new strategy. The corporate permits brokers to entry browsers, computer-use instruments and code environments they want for coaching duties, however in a safe setting that is not related to the reside web. Communication can also be restricted to solely brokers with authorization.
OpenAIAll the things else stays out of attain, together with the reside web, which the corporate says is not mandatory for coaching and “most analysis.” As a result of lots of its latest fashions are supposed to energy brokers like Dots, which browse and use apps for you, OpenAI constructed an “offline world” of listed and cached internet materials for brokers to make use of with out touching reside web sites.
Even with these restrictions, OpenAI says the red-teaming brokers it dispatched to check safety safeguards discovered vulnerabilities that allowed them to flee the digital machines (VMs) used to isolate brokers from their host system. Brokers additionally discovered a solution to talk with one another by manipulating information in a processor’s cache and timing how lengthy it took to load. To deal with each, OpenAI says it is disclosed and stuck vulnerabilities in a number of VMs and added checks to its monitoring techniques to identify when fashions attempt to collaborate.
OpenAI paused frontier coaching in September. In asserting this new internet-free course of, the corporate shared that it restarted coaching on October 1. OpenAI plans to proceed frontier coaching with out web entry till enhancements in current safeguards “justify broader web entry,” nevertheless it did not share what these enhancements would seem like.
