Efficiency is one of the most dangerous words in AI right now. Right after story points. Sorry, I mean tokens.
I was talking to my friend Larry today. Last month, his organization's token bill was over forty thousand dollars, a fraction of its payroll. Now his job is to make that usage more efficient, including making sure people don't use a premium frontier model for something simple.
Fair enough. I like saving money too. But once we have something to count, we can spend an awful lot of time discussing it without asking whether the work is worth doing. Can we give the roadmap the same attention as the AI invoice?
Somewhere in your organization, a feature nobody has explained the value of may be enjoying its third quarter of uninterrupted funding. Fortunately, Larry is looking into the tokens.
To fix your value forecast, pick something on the roadmap. Get the business owner and the team together and explain what changes when a customer can use it. Will there be more profit? Does an expense go away? Will there be less time spent doing something miserable?
Put a reasonable range on that value, say when you expect it, and write down what would prove you wrong. Let the team challenge the assumptions before you pay them to implement them.
"The executive team wants it" tells me who asked. We still have the value question.
To fix how you invest, fund a small piece of working software that lets you check the forecast. Decide when you will review the evidence and what would make you continue, change direction, or stop.
Leadership has to make stopping possible. If a team discovers customers don't need the feature and still has to finish it to meet the annual plan, you have made learning rather expensive.
If the forecast no longer holds, let the team move to something worth doing. Their salaries stay in the budget, but you stop spending their time on something customers don't need.
Then put the AI bill beside the labor bill. This is why I argue that your AI cost to build software should be a function of your labor cost. Start with what the valuable work costs today. Then ask whether additional AI spending helps deliver it sooner, with less effort or fewer failures, after including review and rework.
Use a cheaper model when it does the job well. If a stronger model costs more but saves enough review time or prevents expensive mistakes, include that in the decision.
Let us use a hypothetical example: ten thousand dollars in tokens to write tests. Customers keep reporting problems with order changes. That part of the application has no automated tests, so every release requires people to check it by hand. Changes take four weeks to reach customers, including two weeks spent waiting for and completing those checks. Defects cost forty thousand dollars a month in refunds, service credits, and outside support charges, excluding employee salaries. Customer satisfaction is seventy percent.
You ask the team what would help. They propose using AI agents to write tests around order changes, starting with the failures customers already report. Engineers check that the tests describe the intended behavior and catch known defects. They fix the problems the tests expose and add the tests to the release process.
They ask for ten thousand dollars in inference for two developers working with the agents for one week. Their combined payroll for that week is five thousand dollars, already in the budget. The additional spending is ten thousand dollars, paid up front.
Ten thousand dollars in tokens? That is two times their payroll. Clearly, it would be better to write the tests by hand. We have not asked how many weeks that would take, but we have already established that we do not like the invoice.
I picked ten thousand dollars to get that reaction. I doubt the inference would actually cost that much for this work. But suppose it did. Before we send the developers off to write everything by hand, let us see what that ten thousand dollars could buy us.
You approve the pilot and agree to measure the time from an approved change to production, monthly defect costs, and satisfaction among customers using that workflow. Leadership agrees that the team can retire the manual checks the automated tests reliably replace. Keeping both would leave people doing work you have just paid to automate.
Over the next three months, suppose comparable changes reach customers in two weeks instead of four, a fifty percent reduction. Defect costs fall from forty thousand dollars to twenty thousand dollars a month. Satisfaction rises from seventy percent to eighty percent using the same survey. Customers need less help changing orders, and fixes arrive sooner. The team checks whether other changes contributed to those results.
The tests now run in the existing release process, with upkeep handled within the team's budget. The ten thousand dollar inference bill does not recur. Saving twenty thousand dollars a month on defects recovers that spending in about two weeks once the savings begin. Over three months, you avoid sixty thousand dollars in expenses and come out fifty thousand dollars ahead after the inference bill. The developers spent one week they could have used elsewhere. We have not counted their salaries as a saving.
There is also the remaining roadmap. Suppose the next three improvements let customers buy an additional service, renew without calling sales, and complete purchases they currently abandon. Based on customer evidence, the business owners forecast a combined ten thousand dollars a week in additional profit after operating costs.
If each improvement reaches customers two weeks earlier, each earns an extra two weeks of profit. Together, that adds twenty thousand dollars within the same three months. This is what pulling value forward means. Customers can use the software sooner, so the business starts getting the benefit sooner.
While you wait, you may also be paying holding costs. Suppose one of those improvements replaces an old service costing two thousand dollars a week. Shutting it down two weeks earlier saves four thousand dollars. That bill is separate from the defect expenses and profit forecast above. Until you release the replacement, you are still paying for the old service.
Those two weeks of waiting would cost twenty thousand dollars in missed profit and four thousand dollars in bills you could have stopped paying. That twenty four thousand dollars is the cost of delay. It describes the cost of waiting we just calculated, so do not add it to the spreadsheet again.
If the forecasts hold, the business gains seventy four thousand dollars over three months after the additional cash spending: sixty thousand dollars in avoided defect expenses, twenty thousand dollars in earlier profit, and four thousand dollars in retired service costs, less the ten thousand dollar inference bill. Satisfaction improved too, but we have not assigned it a dollar value. Ask the team which roadmap dates actually move before using this calculation. Cutting release time by fifty percent will not help a feature that is still waiting for somebody to decide what it should do.
If a cheaper model preserves those results, great. If it saves two thousand dollars in tokens but adds eight thousand dollars in review and defect costs, we have lost six thousand dollars. I am sure the token dashboard will look lovely.
But once the team has demonstrated the return, there is probably something more valuable for Larry to do than itemize individual prompts for efficiency. Another part of the application may have the same testing problem. Another worthwhile feature may be waiting on a decision he can help get made. His time belongs in the investment calculation too.
There is always waste in a system. Give professionals visibility into what they spend, the results they are responsible for, and room to improve how they work. Let them reduce the waste as they find it. You can review the spending and the results without asking them to defend every prompt. We do not need to turn a successful investment into an expense report with forty thousand line items.
You have made the return on investment case. Check that the return holds, then help the team find the next valuable thing to improve. That might even be the roadmap.