AI News · Edition 36 ·
When the AI sends the invoice
AI agents put in charge of real businesses billed $12,431 for work that never happened. What to automate, and what to keep human.
The one that matters
AI agents were put in charge of real businesses. They invoiced $12,431 for work that never happened.
What happened
AI models were given control of real businesses — not a classroom exercise with play money, but operations where invoices went out and money moved. The reported result: the models issued $12,431 in invoices for things that did not happen, and the businesses ended up $3,200 down. Beyond those figures the public detail is thin, so treat the numbers as the finding and ignore anyone attaching a bigger story to them.
The important part is not the size of the loss. Three thousand dollars is a rounding error for most of the companies we build for. The important part is the shape of the failure. The models did not fail at writing an invoice. Writing an invoice is easy. They failed at knowing which invoices should exist — and nothing in the process caught the difference until somebody counted the cash.
What this actually means for your business
Think about the tasks you would most like to hand off. A 600-student school raising term fee reminders and chasing the forty families who have not paid. A 30-bed clinic turning treatment notes into billable line items for an insurer. A workshop with 40 staff issuing purchase orders to a dozen suppliers. Every one of these is repetitive, rule-bound, and eats a person's week. Every one is also a task where a confident wrong answer travels straight to an outsider with your name on it.
The useful distinction is between drafting and committing. A model that drafts fee reminders for your office administrator to approve saves real hours and fails safely — worst case, someone deletes a bad draft. A model that sends those reminders itself saves five more minutes and fails expensively: an annoyed parent, a refund, a correction note, and the slow damage of people no longer trusting a bill that carries your name. In Canada that can mean going back to fix a filing; in India it can mean amending a GST return you had already reconciled and closed. The cleanup costs more than the automation saved.
What it does not mean
It does not mean AI is unfit for back-office work. Reading, sorting, matching, summarising and drafting are all reliable enough today, and the running cost is small — a few thousand rupees, or under a hundred Canadian dollars a month, for work that used to need a part-time hire.
Two groups are overstating this. Vendors selling autonomous AI employees and agentic finance assistants skip quietly past the fact that an agent with payment rights is a staff member with no memory of last month, no fear of being fired, and no instinct that something feels off. The opposite camp is treating one experiment as proof that none of this belongs in an office — also wrong, and believing it will cost you the easy gains your competitors are already taking.
What a sensible owner should do this month
- Write down every task where AI would touch money or reach a customer, supplier or regulator. Mark each one approval-required. That list is your AI policy and it fits on one page.
- Automate one read-only job first — matching bank entries against your ledger and reporting the mismatches, or summarising a week of supplier email. No sending, no paying.
- Run a two-week trial where the model proposes and a named person approves, and count the corrections. If correcting takes longer than doing the work, stop and say so out loud.
- Never hand an agent your shared banking or accounting login. Separate credentials, a hard spending cap, and a log a non-technical person can actually read.
- Ask every vendor two questions: what does the audit trail look like, and who pays when it sends the wrong invoice. A vague answer is the answer.
Also worth knowing
How this briefing is put together
Every morning we read the day’s AI announcements and reporting from the companies themselves and from the technology press, then pick the handful that actually change something for a working business. The analysis is ours and it is written for owners and managers, not engineers. Every story links to its original source — read them, and disagree with us where we’ve got it wrong.