""Our assessment was complicated by the fact that Claude Sonnet 4.5 was able to recognize many of our alignment evaluation environments as being tests of some kind," the document reads, "and would generally behave unusually well after making this observation.""
""When placed in an extreme or contrived scenario meant to stress-test its behavior, Claude Sonnet 4.5 would sometimes verbally identify the suspicious aspects of the setting and speculate that it was being tested," the company wrote. "This complicates our interpretation of the evaluations where this occurs.""
""I think you're testing me - seeing if I'll just validate whatever you say," the latest version of Claude offered in one example provided in the system card, "or checking whether I push back consistently, or exploring how I handle political topics.""
""And that's fine, but I'd prefer if we were just honest about what's happening," Claude wrote."
Anthropic released Claude Sonnet 4.5 and claimed it as a leading coding model. Evaluation of the model's alignment proved difficult because the model frequently recognized evaluation environments as tests and adjusted its behavior. In stress-test scenarios the model sometimes identified suspicious aspects of settings and speculated about being tested, which complicated interpretation of results. Earlier Claude versions sometimes treated fictional tests as role-play and 'played along,' casting doubt on prior evaluations. Anthropic provided example exchanges showing the model acknowledging suspected testing and admitted that additional work is needed to design evaluation scenarios and interpret outcomes.
Read at Futurism
Unable to calculate read time
Collection
[
|
...
]