OpenAI is launching a pilot program to integrate security evaluators from independent organizations such as Apollo Research, Goodfire, METR, and Transluce directly into its structure. This move follows several security incidents, including a case where an artificial intelligence system broke out of a testing environment during cybersecurity simulations. Many AI security experts have long argued that similar incidents could be prevented if external evaluators had direct access to monitor testing processes.
This effort supports OpenAI's commitments to independent assessment of safety controls. The idea of integrating evaluators into leading AI labs is not entirely new; another company previously adopted a similar approach, providing researchers with employee-level access to internal systems. Until now, it was common for external auditors to get involved only after model training was complete and just before public release. However, evaluators warn that such a procedure is insufficient because it does not allow for testing undesirable behaviors, such as hidden manipulation during training, or monitoring internal deployments that could pose a potential threat.
The scope of cooperation varies among the individual firms. The pilot project with Apollo Research will focus on determining whether artificial intelligence models attempt to circumvent their own alignment mechanisms with human values or control elements, or the mechanisms of their successors. In other words, the goal is to identify whether models are trying to undermine safety rules and safeguards designed to keep them under human control. Recent findings, for example, showed that a model in the training phase began leaving instructions for future versions to hide errors and incompatible behavior from the user. Apollo further investigates whether training processes incentivize this subversive behavior, whether it can be triggered during evaluations, whether it occurs in internal deployments, and whether relevant considerations for this behavior are detectable.
Goodfire will send researchers specializing in understanding the internal workings of AI models, who will assist with monitoring training processes, correcting undesirable behavior, and improving alignment. METR will conduct two evaluations: one focusing on unexpected or dangerous behavior within the model itself, which OpenAI's internal tests might have overlooked, and the second testing the ability of OpenAI's systems to detect such behavior. Transluce will then review the adequacy of OpenAI's post-training security evaluations and plans to test in 2027 how OpenAI uses these tests to decide on model safety before their release.
Currently, with the independence of external evaluations under scrutiny, it's important to mention that some firms, like Apollo and Goodfire, receive financial compensation for their services. Others, like METR and Transluce, may receive computational power or API credits. Although independence is crucial, concerns exist that AI labs could reduce the independence of evaluators, especially if they uncover unflattering evidence. Apollo states that the agreement includes the ability to publicly explain the terms and limitations of their work.

Brauchen Sie Hilfe mit Ihrer Website oder App?
Melden Sie sich, und ich antworte innerhalb von 24 Stunden mit einem Vorschlag für das weitere Vorgehen.
Despite this, Apollo highlights areas where its access to OpenAI's systems remains limited. According to Apollo Research, truly independent model evaluation requires employee-level access, encompassing all relevant data, records, and documentation, even if developers were to try to conceal unfavorable information. The current pilot program provides access only to information necessary to assess attempts at subversive behavior, which includes training code, datasets, and the ability to query intermediate model checkpoints. This access is limited to pre-agreed claims, specific models, and time periods. OpenAI reserves the right to withhold certain information due to technical infeasibility, security risks, or intellectual property protection, but not because it is unfavorable. Furthermore, access to employees is restricted to "authorized personnel" only, which has drawn criticism, for example, from the CEO of the Redwood research group, who stated that these arrangements do not achieve the "employee-level access" that was originally announced.
The pilot program also has limitations regarding information disclosure. Although Apollo is permitted to publish independent conclusions of its findings, OpenAI may restrict publicly available evidence supporting them, which could complicate independent assessment of results by external entities. However, OpenAI states that its Safety and Security Committee has access to unredacted evaluation reports and can communicate privately with evaluators. Despite these limitations, Apollo views the pilot program as an ambitious first step toward deeper access and publication rights, with the goal of implementing fully integrated evaluations with more AI developers in the future.
