AI News · Edition 40 ·
Your AI vendor is reading the logs — and it still missed one
Anthropic's misuse report matters more for how you buy AI than for what you build with it. Plus: who is really running the model you pay for.
The one that matters
Anthropic says it caught state-linked misuse of Claude — and admits its own checks missed one
What happened
Anthropic published an account of people trying to misuse Claude and of what it did in response. More than twenty outlets picked it up. The findings: campaigns traced to Russian and Chinese actors; about five separate attempts that touched on biological weapons research, all of them blocked; an Iran-linked operation that used the model to help surveil people, including Israelis and Jewish communities abroad; and attempts connected to Houthi forces around ballistic missile work. Buried under the alarming headlines is the detail that matters most to you: a fourth incident that the company's own automated safety checks initially missed, and which only came to light afterwards.
What it means for your business
Three practical consequences, none of them dramatic.
- Your prompts are not a locked filing cabinet. Safety systems run over usage, and a provider can review an account when something trips a wire. Treat AI chat the way you treat email on a company server, not the way you treat a conversation in your own office. Patient names, student records, salary sheets and signed contracts should not go into a tool whose terms nobody in your organisation has read. Under Indian and Canadian privacy rules, responsibility for that data stays with you, not with the vendor.
- Accounts can be cut off, so do not build on just one. A 600-student school running admission enquiries and fee reminders through a single API key, on one account, with one card attached, has a single point of failure that has nothing to do with whether the technology works. Add a second admin user, a backup payment method, and a manual process the office staff can fall back to for a week.
- Automated detection is incomplete, including theirs. The company with full access to its own logs still missed a case and found it later. That is a fair standard to hold any vendor to when they sell you monitoring.
What it does not mean
It does not mean the assistant your accounts clerk uses for draft emails is dangerous. It does not mean anyone built a weapon: the reporting describes attempts that were detected and blocked, not harm that was produced. And it does not mean you need an AI risk assessment from a consultant this quarter. Expect the sales calls anyway — these headlines will be in security vendors' decks by next week, and at least one outlet has already stretched the fourth-incident finding into a story about an AI escaping containment, which is not what a blocked misuse attempt is. If a pitch leans on this news without naming a specific workflow in your business that changes, it is a pitch, not advice.
What a sensible owner does this month
An afternoon of work, at no cost. First, write down every AI tool actually in use, including the free personal accounts staff are quietly using on their phones — in a 40-person workshop that list is usually longer than the owner expects. Second, one page of rules: what data may go in, what may never, who to ask when unsure. Third, name one person accountable; in a 30-bed clinic that is normally the practice manager, not the doctor. Fourth, check the backup admin and payment method on any AI account a real process depends on. If someone quotes you more than a few hundred dollars to do this, they should be able to say which of your tasks changes as a result. Most cannot.
Also worth knowing
How this briefing is put together
Every morning we read the day’s AI announcements and reporting from the companies themselves and from the technology press, then pick the handful that actually change something for a working business. The analysis is ours and it is written for owners and managers, not engineers. Every story links to its original source — read them, and disagree with us where we’ve got it wrong.