Intro

I thought Part 4: Preparing for Production was going to wrap up the series. But wait, there’s more! Not every useful lesson about building with AI tools fits neatly into the larger themes of the past articles in this series. So this final section is a collection of extra bits and pieces that don’t quite belong anywhere else, but may still save you a lot of trouble. Yes, I could have just called it “Misc.” I suppose. But in any case, there’s just some random bits that didn’t fit cleanly into earlier docs and fell out of the edits.

Here they are in one random walk place.

Part 3 was about building durably, so what happens after that? A successful prototype creates a new problem if you actually put it into some form of production. Not too long ago, if we built what we called prototypes, maybe they went to user testing, or we ‘sort of’ put them out there. But now? A lot of so-called tests really aren’t. That is, they’re really more actual rollouts. Just maybe not necessarily with the same rigor we used to give product launches. The line between that which was intended as prototype testing and so-called Minimum Viable Product (MVP) seems like it might get blurry.

The next challenge with launching a kind of prototype, but that which is maybe real product is someone may decide to use it. So… Is it really a prototype? Meaning… is this just some brochureware for testing or is there actual sign-up functionality, any kind of real billing, private information collection, and so on.

This is when the low-stakes experiment begins accumulating customers, private information, billing relationships, operational dependencies, and consequences. You have to make your own call on this kind of thing. I have to tell you, personally I’ve never loved the “false front door” thing for testing. I get that it can be useful, but sometimes it’s on the edge of unethical in how it seems to trick people. If your prototype is really a test and that’s understood, great. If the truth is it’s not much of a test, but a rollout? Then there’s probably some more risk elements and due diligence that should be covered.

Prototypes have a way of becoming products. Temporary credentials become permanent. Test data becomes customer data. A one-time API connection becomes a critical service. The final part of an AI-assisted build is not clicking Deploy.

It’s deciding whether the thing can be operated safely. One thing that I’ve done along the way when building is created something I’ve called “Launch Blockers.” This is different than a UI backlog or minor changes you find along the way. It’s things like purging schemas, or making sure live production feature flags are set properly. It means a real security review. Things like that.

Note: This isn’t about general skill in building things with AI… it’s specifically about things called Skill files or their similar counterparts.

Are you a product person at any level who is either building yourself or managing others that are increasingly doing some direct building?

Like a lot of us, I’ve been making some of my own stuff with AI tools. Or in some cases experimenting to understand their use cases better. Among the hype cycle things of this year are Skill files. (Or more generally skill type instructions for AI tools.) There’s whole marketplaces for them. This post is just a warning I’m throwing out there as a caution along with some ideas for mitigating risk. It’s not meant to be overly alarmist, but the “You can use skills to do anything!” hype is so overwhelmingly thick sometimes, it just needs some balance. And actually, I think it’s not just clickbait, it sometimes feels irresponsible and dangerous.