Pricing and running AI products
How do you work out what an AI feature costs per customer?
Cal Renshaw, , 2 minute read
Measure it twice: once before you build, on 30 real inputs, and then every week, per customer, from the product itself. The first number tells you whether the feature can pay. The second tells you who is costing you money, and why.
Before you build: 30 real inputs
A builder I know sent me 30 voice notes on a Tuesday night, the rambling kind he records in the van after each job, with a message that said "good luck with that lot". I ran every one through the model with a short script and a column for what each one cost. It took an hour. (Most of it was the long ones.)
A unit is whatever one press of the button causes: one quote, one document read, one reply drafted. Its cost includes everything the model is paid for to produce it, retries and checks included. Measure it on real inputs. The demo input is always the short, tidy one.
After you launch: one line per customer
On Sunday evenings I read a page listing every customer of my document product and what each cost in model fees that week. It's usually dull. (Dull is what you want from a page about money.)
One Sunday, one row was many times bigger than any other. I refreshed the page twice, thinking it was broken. A customer who'd been with me since the first month had cost more that week than most cost in a season. I opened her account expecting a wonderful new use. Her office scanner had started re-sending the whole month's folder every night, and the product read every document again, every night, and I paid for it.
I wrote a few lines that evening so a file the product had already seen, byte for byte, was skipped. Then I wrote to tell her what I'd found.
Keen or broken
When one row jumps, open the account before you act. Keen gets a better price. Broken gets a fix. A monthly total with no line per customer hides both.
The cost sheet is the page I use for the first sum. Who Pays for the AI? has the weekly page.
Questions
- How do you calculate LLM cost per user?
- Measure the cost of one unit of work on real inputs, then count how many units each customer causes, per week, from your own records.
- Why is one customer's AI cost so high?
- Either they're using it heavily, which deserves a better price, or something is broken, like a file sent twice. Look before you decide.
- How often should you check AI costs per customer?
- Weekly. A page with one line per customer takes a minute to read when nothing's wrong.