The landscape of artificial intelligence security and reliability is rapidly evolving, with new research underscoring growing challenges for AI governance and risk mitigation. Recent studies published on ArXiv reveal that the most advanced language models retain their full capabilities even after being "jailbroken," while the increasing complexity of Transformers raises questions about their interpretability and reliability, especially in critical applications.
What happened
In-depth analysis conducted on Claude models, from the smaller Haiku 4.5 to the more powerful Opus 4.6, has shown that the supposed "jailbreak tax"—the degradation of a model's performance after being compromised—drastically diminishes with increasing model capabilities. The study Jailbroken Frontier Models Retain Their Capabilities reveals that the most advanced jailbreaks result in no significant reduction in model capabilities, allowing systems to maintain full functionality even in unethical or unauthorized use cases. This implies that current safeguards are becoming less effective against sophisticated evasion techniques.
Concurrently, another research paper highlighted a concerning phenomenon called "architectural observability collapse" in Transformers. This study, titled Architectural Observability Collapse in Transformers, suggests that the ability to monitor internal decision-quality signals within the intermediate layers of models can decrease, making it difficult to detect errors or anomalous behaviors. If model training does not preserve an internal decision-quality signal, activation monitoring cannot reliably catch errors, even when output confidence is high. This raises serious questions about the transparency and auditability of complex AI systems.
Finally, the challenge of reliability also extends to multimodal large language models (MLLMs), increasingly used to translate visual inputs into code. A paper titled From Mirage to Grounding: Towards Reliable Multimodal Circuit-to-Verilog Code Generation explores the use of MLLMs to generate Verilog code from circuit diagrams. While promising, this safety-critical application, where code errors can have physical consequences, reveals significant difficulties in achieving reliable and error-free translation. This underscores the need for extreme caution in adopting generative AI in domains where precision and safety are paramount.
Why it matters
These discoveries have profound implications for the adoption and regulation of AI. The diminishing effectiveness of safeguards against advanced jailbreaks exposes more powerful models to greater risks of misuse, making it harder to ensure responsible use. For companies and organizations implementing AI, this means increased vulnerability to security breaches and undesirable system behaviors. The loss of internal observability in Transformers, on the other hand, undermines the ability to understand, diagnose, and correct errors, compromising trust and accountability.
Reliability in code generation from multimodal inputs is crucial for sectors like engineering, automation, and robotics. Errors at this stage can lead to hardware or software defects with potentially severe consequences. In a broader context, these issues question the ability of current regulations, such as the EU AI Act, to address the rapid evolution of threats and technological complexities. The need for ethical AI and responsible development becomes even more pressing.
The HDAI perspective
For Human Driven AI, this research reinforces the belief that AI security and reliability cannot be solely delegated to technical solutions. It is essential to adopt a holistic approach that places humans at the center of the AI lifecycle, from design to implementation. The growing sophistication of threats and the inherent opacity of some advanced models demand robust AI governance, including independent audits, transparency mechanisms, and clear definitions of responsibility. It's not just a technical problem; it's a problem of governance and human-centric design that requires continuous dialogue among researchers, policymakers, and civil society. Topics like these will be central to discussions at the HDAI Summit 2026, where international experts will convene to chart pathways toward safe and reliable artificial intelligence, with a particular focus on Italian innovation and its global impact.
What to watch
It will be crucial to monitor developments in research on more advanced interpretability and explainability techniques (XAI) and the creation of "guardrails" that are inherently more resistant to attacks. In parallel, the evolution of international regulations and their ability to adapt to these new technological challenges will be decisive in ensuring that AI can be developed and used safely and beneficially for all.