Time to step away from the AI hype and look at real costs for everyday AI use cases. This article focuses on internal tooling and operational spend, not so much more exciting user facing products; though the same cost dynamics apply.

My motivation here is I keep hearing AI will let teams “vibe code” their way out of SaaS, but AI can also turn the resulting products’ predictable Operating Expenses (OpEx) into volatile, usage-based costs. So I want to think through ongoing variable costs. Not ROI or quality risk, which matter too. Or the reality that AI talent may still be hard to find; just really more ongoing variables costs, which are different than our more predictable past tools.

Spoiler Alert: I’ll run through the thought process and offer up a spreadsheet. But the bottom line is this choice can be very business specific. Building your own with AI can easily spin up costs faster than you’d expect, as can burning tokens with SaaS solutions that add AI. But even speedier app development with AI assisted coding might cost you more than SaaS. Scroll down and just grab the spreadsheet or read on for all the details. It’s about more than just tokens vs. seats.

Quick Summary: SaaS often seems pricey per year, while “vibe coding” your own replacement can look cheaper early but gets expensive once you count labor, operations, and risk. The biggest drivers are seats, interaction complexity, and hidden costs like evals, testing, and ongoing auditability. DIY can win for small, short efforts, but at enterprise scale SaaS can still be cheaper on TCO even if the subscription line stings. In either case, for AI enabled products, there’s inference/token and other costs. Here’s the sheet if you’re skipping the rest of this article. (Note that older non-AI enabled build vs. buy examples are there just for historical reference and comparison.)

• On Scott’s Github
• In Scott’s Google Sheets

Also note that the cost assumptions I’ve put in here to start are radically higher than what simple token pricing might be compared to what you’ll find on a vendor’s chart. I’ve tried to add blended costs for things that reflect real production costs, like API costs, vector database queries for RAG, caching and so on. The point is for you to plug in your own numbers. If you want you can split out more granular costs to their own lines.

And, oh yes… don’t forget token costs are not the whole story. This piece is focused on costs, but when you shift to pricing and ROI, it’s worth reading John Rowell’s Context Is the Next Frontier in AI Economics.

[Read more…]

You’ve probably read about AI ethics. I want to go beyond the basics and sell you on the ROI of AI Ethics. We’re talking basic consumer and B2B issues here and more accuracy, not wider scale issues like battlefield autonomy or similar use case issues. I’ve worked on a few AI projects over the past year: one more retrieval-based (prompt/response) and another more generative (content creation with human oversight), plus some testing work. With today’s tools and vendors, you can often kick something out that seems solid pretty quickly, whether it’s a generative product or something using more traditional tools like predictive analytics. Doing it truly well, though, is sometimes orders of magnitude harder. It can be tough to justify the resources to do it right.

Here’s a cynical thought about AI ethics. Often, companies don’t really care. Actually, that’s too cynical. People may care, however, if you look at where pressure and resourcing usually go, it’s often more towards speed to market and growth, with maybe a thin layer of regulatory compliance. A few companies differentiate on quality. But most seem to be more in the feature race. “Non-functional” requirements? We’ll get to them. At some point.

This doesn’t mean everything needs to be perfect. If we waited for 100% certainty, nothing would launch. We make tradeoffs: a light/dark mode toggle is not an emergency cardiac-alert system. People get that. Especially AI, which is harder to understand and harder to control. How can we deal with these realities?

[Read more…]