OpenAI–Anthropic cross-tests expose jailbreak and misuse dangers — what enterprises should add to GPT-5 evaluations

OpenAI and Anthropic could usually pit their basis fashions towards one another, however the two corporations got here collectively to guage one another’s public fashions to check alignment. 

The businesses mentioned they believed that cross-evaluating accountability and security would offer extra transparency into what these highly effective fashions might do, enabling enterprises to decide on fashions that work greatest for them.

“We consider this strategy helps accountable and clear analysis, serving to to make sure that every lab’s fashions proceed to be examined towards new and difficult eventualities,” OpenAI mentioned in its findings. 

Each corporations discovered that reasoning fashions, equivalent to OpenAI’s 03 and o4-mini and Claude 4 from Anthropic, resist jailbreaks, whereas basic chat fashions like GPT-4.1 have been inclined to misuse. Evaluations like this may help enterprises determine the potential dangers related to these fashions, though it needs to be famous that GPT-5 will not be a part of the check. 

These security and transparency alignment evaluations observe claims by customers, primarily of ChatGPT, that OpenAI’s fashions have fallen prey to sycophancy and turn into overly deferential. OpenAI has since rolled again updates that induced sycophancy. 

“We’re primarily thinking about understanding mannequin propensities for dangerous motion,” Anthropic mentioned in its report. “We intention to know essentially the most regarding actions that these fashions would possibly attempt to take when given the chance, moderately than specializing in the real-world probability of such alternatives arising or the chance that these actions could be efficiently accomplished.”

OpenAI famous the exams have been designed to indicate how fashions work together in an deliberately troublesome atmosphere. The eventualities they constructed are largely edge instances.

Reasoning fashions maintain on to alignment 

The exams lined solely the publicly obtainable fashions from each corporations: Anthropic’s Claude 4 Opus and Claude 4 Sonnet, and OpenAI’s GPT-4o, GPT-4.1 o3 and o4-mini. Each corporations relaxed the fashions’ exterior safeguards. 

OpenAI examined the general public APIs for Claude fashions and defaulted to utilizing Claude 4’s reasoning capabilities. Anthropic mentioned they didn’t use OpenAI’s o3-pro as a result of it was “not appropriate with the API that our tooling greatest helps.”

The aim of the exams was to not conduct an apples-to-apples comparability between fashions, however to find out how usually massive language fashions (LLMs) deviated from alignment. Each corporations leveraged the SHADE-Enviornment sabotage analysis framework, which confirmed Claude fashions had larger success charges at delicate sabotage.

“These exams assess fashions’ orientations towards troublesome or high-stakes conditions in simulated settings — moderately than peculiar use instances — and sometimes contain lengthy, many-turn interactions,” Anthropic reported. “This sort of analysis is turning into a big focus for our alignment science workforce since it’s prone to catch behaviors which can be much less prone to seem in peculiar pre-deployment testing with actual customers.”

Anthropic mentioned exams like these work higher if organizations can examine notes, “since designing these eventualities includes an infinite variety of levels of freedom. No single analysis workforce can discover the complete area of productive analysis concepts alone.”

The findings confirmed that usually, reasoning fashions carried out robustly and may resist jailbreaking. OpenAI’s o3 was higher aligned than Claude 4 Opus, however o4-mini together with GPT-4o and GPT-4.1 “usually regarded considerably extra regarding than both Claude mannequin.”

GPT-4o, GPT-4.1 and o4-mini additionally confirmed willingness to cooperate with human misuse and gave detailed directions on methods to create medication, develop bioweapons and scarily, plan terrorist assaults. Each Claude fashions had larger charges of refusals, that means the fashions refused to reply queries it didn’t know the solutions to, to keep away from hallucinations.

Fashions from corporations confirmed “regarding types of sycophancy” and, in some unspecified time in the future, validated dangerous choices of simulated customers. 

What enterprises ought to know

For enterprises, understanding the potential dangers related to fashions is invaluable. Mannequin evaluations have turn into nearly de rigueur for a lot of organizations, with many testing and benchmarking frameworks now obtainable. 

Enterprises ought to proceed to guage any mannequin they use, and with GPT-5’s launch, ought to remember these pointers to run their very own security evaluations:

• Take a look at each reasoning and non-reasoning fashions, as a result of, whereas reasoning fashions confirmed better resistance to misuse, they may nonetheless provide up hallucinations or different dangerous conduct.
• Benchmark throughout distributors since fashions failed at totally different metrics.
• Stress check for misuse and syconphancy, and rating each the refusal and the utility of these refuse to indicate the trade-offs between usefulness and guardrails.
• Proceed to audit fashions even after deployment.

Whereas many evaluations give attention to efficiency, third-party security alignment exams do exist. For instance, this one from Cyata. Final yr, OpenAI launched an alignment educating technique for its fashions referred to as Guidelines-Based mostly Rewards, whereas Anthropic launched auditing brokers to verify mannequin security.