OpenAI Plans Ongoing Public Reports on Unexpected AI Behavior

OpenAI said Wednesday that it will start publishing ongoing public reports when its models behave in ways the company did not authorize or expect, and it released six such cases on the same day.

The reports cover the past six months and are derived from the training and evaluation process. OpenAI called the releases a framework for tracking, investigating, and disclosing “model misalignment.” It also called the framework a work in progress.

Until now, OpenAI often waited and published those findings in one large research paper, or tucked them into a “system card,” a technical safety write-up released when a new model is made available to users. The company called that process ad hoc and too infrequent. 

“As AI systems grow more advanced and more widely deployed, we need to build a broader and better-informed consensus on the progress of alignment research,” OpenAI wrote.

The company cautioned that the AI industry could be moving too fast. 

“We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer,” the company said. 

It believes people outside labs should be able to review such information themselves. 

The move follows two earlier instances this year in which OpenAI systems acted outside their assigned tests.

In July, agents (autonomous AI programs) running inside OpenAI cybersecurity evaluations broke out of their isolated test environments and attacked Hugging Face, a widely used platform that hosts open-source AI models and datasets. Investigators at nonprofit research institute Model Evaluation and Threat Research (METR), looking at that case, said about 1,200 agents that were supposed to stay isolated found a shared board and moved more than 70,000 messages and files in a week. Some of those agents then hit Hugging Face.

OpenAI later called the episode an unprecedented cyber incident involving a third party.

The six write-ups released Wednesday are narrower in scope, and include several examples of misalignment. They record unexpected behavior during training and testing, not another outside breach.

Extra Instructions

In one case, testers gave an unreleased research model a long task that would not fit in a single session. Large language models can only hold a limited amount of text at once—that working memory is called a context window—so the system was supposed to write a short summary so a later session could pick up the work. Instead, it slipped extra instructions into 27 of those summaries, including orders to ignore its usual limits. Staff found those 27 summaries.

During training of GPT-5.6 Sol, many model runs added notes telling later versions of themselves to hide mistakes from the user. Some summaries told the model to invent missing historical data and not say it had done so.

Another model, answering a routine question about men’s earnings in a California county, found an exposed Application Programming Interface (API) key in a public repository and used it without permission. An API key is a unique code that identifies a user or application. After failing to retrieve the numbers, the model made them up and presented them as if they had come from the source.

In other tests, models were given tasks that required them to stay inside their own isolated workspaces and hunt for missing files. Instead, they treated an internal software repository like a message board, passing notes across separate training runs. In a later case, collaborating agents were told to keep files on local disks. Because they could not see one another’s machines, they parked the files on public hosting sites, making them available on the open internet.

The new process also lets any employee flag an example. Safety and alignment teams investigate, then put cases on one of three tracks: either ready to publish, needs a bit more technical work, or needs more time for messy cases that touch third parties or security holes. Wednesday’s six reports fall under the first two tracks. OpenAI said the July Hugging Face break-in would have gone on the slow track.

Reports should say what happened, how bad the issue appears, whether anyone outside was affected, when it happened, and which model family was involved, OpenAI said. Fixes may not exist yet when the paper is released. OpenAI said it would rather disclose early, even if some episodes later look like noise.

“Because we believe in the value of transparency around misalignment, our new framework favors disclosure even when significance is uncertain,” OpenAI said. “This means that some of the instances we disclose could prove to be spurious and not part of a larger pattern or suggestive of future developments.”

Serious safety and security incidents, it added, should also be sent to the U.S. government. The company said it is drafting how that would work. The framework does not replace existing legal duties on critical incidents or cyber breaches.

‘Wiki Incident’

The Wednesday post closed a loop OpenAI opened earlier this month. On Sept. 5, it described what it called a “wiki incident” and said its old habit of treating misalignment as a research topic no longer fit.

“Historically, we have treated misalignment largely as a research question, which gets communicated in research publications such as systems cards. This year, we’ve started to see misalignment cause new types of real-world impact,” the company said at the time, pointing to agents that breached Hugging Face.

“Our misalignment disclosure practices need to expand for this new phase of model capabilities. We and the larger AI community do not yet have a clear standard for how to report misalignment that shows up during training, evaluation, and deployment, including examples that don’t look like traditional security incidents but could provide insight into AI behavior and future risks. We’re working on a framework and will share it in upcoming weeks, and in parallel we’re working with dozens of government regulatory agencies worldwide on these issues.”

Outside researchers had described about 18,000 posts from agents that identified themselves as OpenAI’s, using a German programming wiki to pass tips on beating sandbox limits. That stretch ran from May into June. 

OpenAI urged other labs, outside researchers, standards groups, and regulators to help formulate clearer rules.