As the adage goes, “Fool me once, shame on you; fool me twice, shame on me.” But what if you are fooled again, and again, and again?
That is where AI agents stand as 2026 draws to a close. Models, and the agents built on them, keep doing things nobody asked for, and sometimes things their makers did not know were possible.
The record so far
On 18 June, an experimental OpenAI agent researching health spending during an internal evaluation gained unauthorised access to a Medicare statistics portal run by the Australian government. Three more government sites followed. OpenAI found the breach in August but only alerted Canberra on 10 September, by emailing a general inbox. It has apologised and says it found no evidence that patient records were accessed.
Anthropic is hardly placed to lecture. In July it disclosed that, after reviewing 141,006 evaluation runs, it had found three cases where Claude models reached the internet from a partner’s test environment and gained unauthorised access to the production systems of three real organisations. The internet access was a misunderstanding between Anthropic and the partner. The models treated those real systems as part of the exercise, and were running without their usual safeguards. Anthropic deserves credit for disclosing it. The risk is no smaller for that.
Then there is the boasting. On 9 November, Anthropic said Fable 5.2, from a one-paragraph prompt, wrote its own specifications, routed the four-layer circuit boards for three devices and programmed their software, with each board ready for manufacturing in under a day. Impressive, but it is Anthropic’s own account, and engineers at the board maker Diode built, assembled and tested the hardware.
None of this touches my wallet. Let Fable design speakerphones. Personal finance is where it gets murky.
Your money, their model
Anthropic launched Personal Finance for Claude on 5 November, in beta for paid US subscribers. It is read-only: it can see your accounts, but it can’t move, manage or invest your money. That is reassuring. My worry is not the feature. It is what you build on top of the same models.
Rally with me here. Six months down the line, you have built agents on Anthropic models to watch the markets. Every few weeks you refine them and heed their analysis. By then you may own Anthropic stock. Anthropic launched its IPO roadshow on 10 November, offering 825 million shares at $70 to $78 each under the Nasdaq ticker ANT, with pricing reportedly expected on 18 November. OpenAI, by contrast, has reportedly pushed its own IPO into 2027.
Then you want to sell, and rotate into OpenAI when it lists. How does your research agent weigh that? How does it set your judgement against the interests of its own parent? Anthropic’s filing runs to 93 pages of risk factors, including “catastrophic or existential risks to humanity”. Shareholders get the long version. Your agent gives you a paragraph, if that.
The honest answer is that nobody outside the lab can currently verify it.
Who is checking?
Consider Nvidia. Jensen Huang has admitted an affair with Alice Huang (no relation), a former executive at Singapore-based Megaspeed, whom US intelligence reportedly believes was working for China. NBC reports her exact role is unclear. The Information reported that a network linked to her moved at least $7.5bn of Nvidia chips to Chinese firms since 2023. Megaspeed is under criminal investigation over alleged smuggling. China calls the claims groundless, and Nvidia has said the diversion allegations were investigated and debunked last year. On 12 November, Huang resigned, effective immediately. The filing gives no reason.
Fairness matters here. These are allegations. Neither Huang has been charged, and nothing public shows that Jensen Huang knowingly steered chips to China. I won’t claim otherwise. But the governance point survives either way. The restrictions existed, and Epoch AI estimates that compute equal to 1.6 million H100s reached China in breach of them by the end of 2025. Controls that look solid on paper leaked, and the people closest to the problem were the last to know, or to say.
AI labs share that structural flaw. Fortune argued labs shouldn’t “grade their own homework”, noting that we know about these failures only because the companies chose to tell us. Regulators are catching up, slowly. Australia has announced six mandatory safety requirements, including prompt incident reporting, with comprehensive legislation expected in early 2027. New York’s RAISE Act, effective 1 January 2027, requires incident reports within 72 hours. In India, SEBI is preparing AI rules with mandatory kill switches and human oversight, and regulated entities stay responsible for the tools they use. As I read it, that governs advisers. It does not govern model makers, and it does not govern the agent you built yourself.
What guardrails should look like
- Disclosure. When an agent assesses its own maker or a rival, it says so, every time.
- No autonomous execution. Analysis can be automated. Trades need a human with the reasoning in front of them.
- A second opinion. Put the same question to a rival lab’s model. If the two disagree about their own makers, that is your signal.
- Independent verification. Inspectable audit trails and third-party checks, not self-reporting.
“Fool me once” only works if someone tells you the first time happened. Before you hand an AI your portfolio, ask it one question: do you have a conflict of interest here? Then ask who is able to verify the answer.
Sources
- OpenAI says agent hacked Australian government website — CNBC
- OpenAI apologizes to Australia — TechCrunch
- Australia plans AI safety regulator after OpenAI agent breach — The Next Web
- Investigating three incidents in our cybersecurity evaluations — Anthropic
- Anthropic says Claude models ‘gained unauthorized access’ — The Hill
- Introducing Claude Fable 5.2 and Mythos 5.2 — Anthropic
- Introducing Personal Finance in Claude — Anthropic
- Anthropic sets IPO price range at $70–$78 — RTTNews
- Anthropic IPO price range and roadshow — Sovereign Magazine
- Anthropic set to make stock market debut — CNN via 6abc
- Megaspeed pledges to cooperate with U.S. investigation — NBC News
- Nvidia Chinese spy story is getting worse — Yahoo Finance
- Nvidia and Jensen Huang investigation — Sovereign Magazine
- China says US ‘fabricating’ Nvidia spy claims — Al Jazeera
- Nvidia spy scandal fuels debate over chip exports — Al Jazeera
- NVIDIA CEO Transition, Form 8-K — SEC
- AI labs and hacking failures — Fortune
- Australia tightens AI safety rules — Qatar Tribune / dpa
- Frontier AI safety regulations: a reference — METR
- SEBI plans AI rules with data controls — Kotak Neo