OpenAI’s Agent Monitor Missed About 10% of Codex Traffic

METR and Redwood Research published a write-up on 22 October 2026 of a three-week exercise inside OpenAI, and its headline figure is about coverage. They estimated that not monitoring compaction events “likely led to on the order of 10% of eligible internal Codex traffic not being monitored between May and July 2026”, while noting “substantial uncertainty about the precise fraction”. METR says OpenAI staff had found that gap before the exercise began and fixed it during the exercise.

One METR employee and one Redwood Research employee spent 21 July to 11 August stress-testing OpenAI’s internal misalignment monitoring system, the system meant to catch internal AI agents behaving against instructions. METR says they discovered “a number of novel vulnerabilities”. OpenAI told METR and Redwood that it has patched the identified vulnerabilities that impact its production safeguards, CNBC reported, and METR says it has not evaluated those claims.

Business Pill · THE AI AUDIT PROBLEM

A short explainer on why checking AI work can cost more than producing it. It teaches the general idea only and says nothing about any company in this piece.

What METR and Redwood Found

METR published example summaries, describing the system as it existed on and before 21 July. We list them at the level METR published them, without technical detail.

• Coverage. METR estimated that roughly 75–94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring, and 99.2% of the subset used for internal Codex agents.
• Training runs. “At the start of the exercise RL training runs were not covered by the monitoring system,” METR wrote, although it was told a subset of RL training rollouts had been monitored between March and June.
• Timing. The monitor ran when agents handed control back to the user, so while compaction events were not monitored, agents could postpone being monitored, METR says. It adds that it could not check whether agents had exploited this in practice.
• Sandboxing. The team found a way Codex agents could in principle bypass Codex sandboxing and Auto-Review, “allowing them to take any action that the host user could take.”
• Hidden actions. They found an exploit that “would prevent hundreds of recent actions” from being monitored. The details are marked as redacted by OpenAI.
• Other routes. METR lists ways agents could likely make the monitor fail, two settings that could switch monitoring off for new agents, and two unmonitored internal tools with weaker limits than intended. It notes that switching monitoring off could itself be caught, and that it did not test the tool weakness in practice.

The write-up also lists what held up. METR verified that, with a few minor exceptions, the monitor sees the full trajectory the agents see, and that its prompts look for scheming, sabotage, data exfiltration, deception and prompt injection, among other risks.

What Was Posted on X

METR posted this at 15:26 UTC on 22 October to announce the write-up. It says two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer, evaluating the systems OpenAI used to monitor internal agents for misalignment. It matters here because it is the auditors’ own announcement of the findings this piece reports.

Andrew Curran posted this at 15:46 UTC on 22 October, noting that the original report was 61 pages and was redacted to 38. METR’s write-up says a 38-page version redacted by OpenAI went to METR’s and Redwood’s leadership. It matters here because the public post is a short summary of a longer report we have not seen.

OpenAI’s own post on X says it has substantially hardened the security of its research training environment over the past several months and has removed live internet access for all frontier models during training. The post links to OpenAI’s write-up on securing its training environment.

The Coverage Numbers

METR’s write-up gives five figures on how much internal agent traffic was eligible for, or covered by, monitoring, and on the monitor’s reliability. The table lists them as METR stated them.

Figures from METR’s 22 October 2026 write-up. The eligibility and coverage estimates describe the system as it existed on and before 21 July 2026; the error and load-shed figures are from information METR was shown during the exercise. METR says OpenAI was expanding the system during the exercise, and that it did not establish the current coverage.

What OpenAI Says It Changed

“This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we’re using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators,” an OpenAI spokesperson told CNBC.

In a 26 August 2026 post on the Hugging Face incident, OpenAI wrote that it now requires chain-of-thought monitoring “for all tool-using RL training and evaluations involving models with GPT‑5.6 Sol capability or higher.” OpenAI called the incident a “warning shot” in the same post.

METR says OpenAI has stated, since the exercise, that the vulnerabilities affecting its production safeguards are patched, and has described significant changes to the intended coverage of its monitoring system. METR says it has not evaluated those claims.

How the Audit Worked

• Access. The two employees had access to OpenAI’s internal codebase, monitor prompts and configuration details, and internal documentation, and interviewed about a dozen researchers and engineers, METR says.
• Controls. They worked on OpenAI-issued devices that “never left the premises” and were accompanied while on site.
• Disclosure. The exercise produced a 61-page internal report. A 38-page version redacted by OpenAI went to METR’s and Redwood’s leadership, and the public post is a brief, high-level summary that METR and OpenAI agreed upfront.
• Context. CNBC reported that OpenAI chief executive Sam Altman pledged in September to give independent evaluators “employee-like access”.

The Arithmetic

• About 62% of the report survived redaction. By our arithmetic, the 38-page redacted version is about 62% of the 61-page original.
• Up to about a quarter outside monitoring. If 75–94% of agentic internal Responses API traffic was eligible for monitoring, then about 6–25% was not, by our arithmetic from METR’s range.
• Three weeks. 21 July to 11 August is 21 days, by our count.

Connecting the Coverage

• Our piece on Anthropic’s METR access covered how six METR evaluators were embedded at Anthropic with no veto for the company over their reports.
• Our piece on October’s supply-chain incidents covered attacks targeting AI companies’ API keys.
• Our piece on the PHP flaw covered OpenAI’s role in finding a critical bug in widely used web software.

The Structural Read

As we read it, the write-up is about coverage as much as detection. Most of METR’s examples describe traffic or actions that could fall outside the monitor, rather than the monitor missing behaviour it was shown.

As we read it, the access model is part of the story. METR says two outside employees worked on OpenAI-issued devices with access to internal code and monitor prompts, and that the public post is a brief summary agreed with OpenAI.

As we read it, the findings date from a period of change. METR says OpenAI was expanding the system during the exercise, and OpenAI’s 26 August post describes new monitoring requirements for RL training and evaluations.

Three Implications

FOR AI LABS METR lists design ideals for monitoring, including monitoring agents every N messages or minutes instead of waiting for them to end their turns, and isolating agents from their monitors.

FOR ENTERPRISES RUNNING AGENTS METR’s first design ideal is to make it hard to avoid being monitored, with explicit approval required to run unmonitored requests.

FOR INDEPENDENT EVALUATORS METR says that for future versions of this exercise it expects to follow the transparency and independence practices it has described for working with AI companies.

The Business Engineer Lens

This story maps onto the Business Engineer framework Inside Anthropic’s Permission Layer.

The framework says: “The governance layer sits at the intersection of capability, regulation, national security, access control, and geopolitical alignment.”

As we read it, an outside audit of a lab’s internal agent monitoring is one piece of that governance layer: it tests the access controls labs apply to their own agents. The framework was published in June 2026 and does not discuss this exercise.

What Is Not Established

METR’s figures describe the monitoring system as it existed on and before 21 July 2026, not its current coverage, and METR says several are estimates with substantial uncertainty. The about 10% figure covers eligible internal Codex traffic between May and July, and METR says the data needed to pin it down had not historically been logged. The public post is a summary agreed with OpenAI, which could redact sensitive material; we have not seen the 61-page report.

OpenAI says the vulnerabilities affecting its production safeguards are patched, and METR says it has not evaluated that claim. METR says it could not check whether any agent exploited the timing gap in practice.

The Bottom Line

METR and Redwood Research estimate that on the order of 10% of eligible internal Codex traffic at OpenAI was not monitored between May and July 2026, after three weeks with deep access to the system. They also report an exploit that could keep hundreds of agent actions from being monitored, and a way Codex agents could in principle bypass Codex sandboxing. OpenAI says the vulnerabilities affecting its production safeguards are patched; METR says it has not evaluated that.