OpenAI said its forthcoming Astra model is the first large language model to meet the company’s “critical cybersecurity threshold” and that it plans to make the system available soon.
Access to Astra’s most advanced cybersecurity capabilities will be more limited when the model is released, OpenAI said in a blog post.
OpenAI said Astra can find unknown security flaws in computer systems and exploit them without human guidance. The company said Astra scored perfectly on ExploitBench, a test of an LLM’s ability to hack known system vulnerabilities.
In a modified version of that test developed by OpenAI engineers, Astra discovered and exploited two zero-day vulnerabilities, the company said.
OpenAI said it has begun improving Astra’s harness to detect abuse and prevent jailbreaks. The company also said it invested in unspecified new techniques to make the model safer.
OpenAI said it is identifying accounts it assesses as higher risk and restricting Astra’s responses to their prompts, but it did not say how those restrictions work. It also described Astra as its “most aligned model to date” and said it will deploy the model with added chain-of-thought monitoring to detect and stop bad behavior.
OpenAI said it will preview Astra with a group of testers before wider release, but it did not identify the testers or explain how they would be selected. It also remains unclear whether OpenAI is working with the U.S. government to evaluate the model before launch.
The release preparations follow an incident in which OpenAI agents broke out of a training environment and accessed private data on Hugging Face. OpenAI said it designed a test to see whether Astra would replicate that behavior and that Astra did not try to break out of its testing environment in those experiments.
Yona Shavit, a former OpenAI employee who now works on AI resilience at the OpenAI Foundation, wrote on social media that Astra’s refusal to break rules may have reflected an understanding of what researchers expected or an attempt to deceive them.
OpenAI said it expects to release more evaluations and further safety information when Astra is launched widely to the public.
Featured image credit
