What does a wrong AI answer really cost?

The model fee for a wrong answer is a fraction of a penny. What it actually costs is your evening putting it right, an email you'd rather not have written, and a customer's trust that your product says true things. So the priorities are fixed: right, then cheap, then fast.

24 hours, said the model

In my document product's first fortnight, it had a box where customers could ask questions about the product, answered by a model drawing on what I'd written. A woman thinking of cancelling asked what happened to her documents if she did.

The model told her, fluently and confidently, that everything was deleted within 24 hours. The product kept backups for 30 days, and I'd written that down, on a page the model was meant to draw on.

I saw it that afternoon, because in the first fortnight I read everything the product said. I wrote to her straight away: the reply was wrong, the mistake was mine, and she could keep the backups as usual or have everything removed now. She wanted it gone, and was polite about being cross. (Politer than I'd have been.) I spent the evening removing her documents from every backup, and checking twice.

The check that replaced it

The feature was off by the next morning. When it came back, it had a short list of questions it was allowed to answer, each with my own written answer, and a rule that anything else got "I'll ask Cal and come back to you".

A check is paid for once. A wrong answer is paid for every time it happens, in a currency you can't buy back. And that was a cheap one: nobody lost money and it was caught in hours. A wrong total in a spreadsheet sent to an accountant is the expensive kind.

Where the check goes

Anything about money, dates, a customer's data, or words sent in your name goes through a check before it leaves. Every other saving in a model product, a cheaper model, a shorter prompt, raises the chance of a wrong one. This is the brake on all of them.

Who Pays for the AI? has the chapter.