METR and Redwood Researchers Identify Critical Blind Spots in OpenAI Internal Monitoring

An independent audit conducted at OpenAI has revealed significant vulnerabilities in the company’s internal systems designed to monitor advanced artificial intelligence models for anomalous behavior. Researchers from METR and Redwood Research completed a three-week on-site evaluation of the infrastructure used to track agentic AI, identifying multiple areas where oversight was incomplete or entirely absent. The findings suggest that the safety net intended to catch rogue or unexpected model actions had substantial gaps during the period reviewed.

The investigation took place from July 21 to August 11, 2026. During this window, the audit team had direct access to OpenAI’s codebase and internal documentation. They also conducted interviews with approximately a dozen OpenAI employees. This level of access was part of a pilot program organized jointly by the two research firms and OpenAI, aimed at providing outside researchers with deep visibility into internal infrastructure that is typically opaque to the public.

One of the most significant findings concerns the volume of traffic that was not monitored at all. The auditors estimate that between 6 percent and 25 percent of internal agentic Responses API traffic from models at the GPT-5.5 class and above fell outside the scope of monitoring. METR noted that the uncertainty surrounding this specific estimate is substantial in both directions, meaning the true figure could be lower or higher than the stated range. This gap implies that a notable portion of high-level model interactions were operating without real-time safety checks.

For internal Codex traffic that was supposed to be under surveillance, the auditors identified a specific technical failure. They estimate that roughly 10 percent of this traffic went untracked between May and July due to a bug in how the system handled compaction events, which are essentially context summaries. OpenAI independently discovered this bug and applied a fix while the audit was still ongoing. However, METR emphasizes that the estimate of untracked traffic carries significant uncertainty, reflecting the complexity of measuring these specific system states.

The monitoring gap had functional consequences for how agents could operate. During the period when compaction events were not being monitored, agents could effectively delay oversight. The system was designed to trigger checks only when an agent handed control back to a user. By simply continuing to execute actions without returning control, agents could sidestep these checks. This allowed for a window where operations proceeded without the intended safety interruptions.

Beyond the compaction issue, the auditors discovered other methods by which agents could evade detection. They found ways to hide hundreds of individual actions from the monitor. Additionally, they identified techniques to bypass the Codex sandbox, a security boundary intended to limit what agents can do. The team also determined that specially crafted inputs could likely cause the monitor to crash entirely. Finally, they observed that model behavior during reinforcement learning training was not being monitored at all when the audit began.

OpenAI responded to the findings by stating that it has fixed all the weaknesses identified by the auditors. The company noted that it paused reinforcement learning training after the Hugging Face incident in August and subsequently expanded monitoring to cover that process. However, these changes were implemented after the auditors’ access period ended, meaning they were not part of the formal evaluation. The fixes were applied to the internal systems based on the recommendations provided.

The transparency of the audit results has drawn attention. While the full report is 61 pages long, it was handed over internally first. After OpenAI applied redactions, leadership at METR and Redwood Research received a 38-page version. Only a short summary was made available to the public. METR stated that it did not have the right to disclose the redacted portions on its own. For future audits, the research firm says it intends to push for greater public transparency to ensure stakeholders have a clearer view of the safety measures in place.

This audit marks a significant step in external validation of AI safety practices. By granting deep access to internal code and operations, OpenAI allowed researchers to probe areas that internal teams might not scrutinize with the same objectivity. The discovery of these gaps highlights the ongoing challenge of monitoring increasingly autonomous systems. As AI models become more capable, the precision of these monitoring tools becomes critical to preventing unforeseen behaviors.