For all the product tools we have, from customer profiles to journey maps, there’s been little use of prediction markets to inform decisions. Unlike AI-driven predictive analytics, prediction markets remain rare in product work. With generative AI pushing teams toward faster experimentation and shipping, they may be worth another look. I became interested in this while playing with a few markets myself and wondering how they might apply to product work.
[Read more…]In Part 2, I focused on the practical build layer: workflows, prompts, context, observability, and evals. This last section is about control and judgment, governance, kill switches, cost discipline, complacency, and the perhaps underrated skill of knowing when not to use AI in the first place.
[Read more…]In Part 1, I focused on why AI product work gets messy so quickly. Strategy theater, bad data, dependency problems, and workflow brittleness. This next section moves closer to the machinery itself: workflow tools, notebook sandboxes, exception handling, observability, skill files, context, and prompt engineering.
I use n8n.io for several tasks; a couple professional and several personal. If I was a real developer, maybe I’d be using something closer to pure code. But I’m not. Still, I think tools like these are more than just a crutch. And they’re certainly a fast way to wire things up for some basic testing. (Though some argue newer generative code tools are better and you can skip such things. I’ve been using both though, and find – as usual – what’s best depends on the use case.)
Regardless of platforms, the thing you really need to do with these tools is have an MLOps view from a Product perspective; either something you built yourself or in partnership with your AI/ML/Dev team members. Ideally you have a skilled architect on staff, however you should be collaborating on these things or at least in the loop. In a startup, there might be some more heavy lifting on the product side.
[Read more…]
This three part series is not a warm, fuzzy post about some AI strategic theater, though there are both strategic and tactical issues included. It’s a walk down some paths regarding practical issues. It’s about some of the challenging practical realities we face once product people try learning about or making tools work in actual workflows. You may feel some of my personal scar tissue in some of these passages. This is for semi-technical product people working with or learning several AI tools and dealing with some of the practical gotcha’s in making them go. It’s also raising a hand and calling BS on the spew of feed drivel about how “all you have to do is just set all this stuff up and crap magically happens.”
What’s Ahead? failures, data, workflow ops, context and prompting, evals, governance and kill switches, costs, complacency, and more, mostly related to smaller or mid-sized projects; though some ideas apply to all. This is a rather long series, even for me. But when I study and do things, I tend to go deep. As is often the case, these posts are really based on my notes to myself in my personal wiki over time, cleaned up somewhat and posted here to share.
These tools are great and I really enjoy them, even if there are some challenging spots, which we’re about to explore. Some of this stuff feels magical! And fun if you have that attitude. It really can be just fun and satisfying to see a workflow executing well if you’re working hands on. However, issues can quickly become a major hassle. It’s rarely as it is in so much of the feed fawning I see. On the surface, it looks like product managers, all of us really, are being sold a fantasy of frictionless AI tooling. It makes me wonder if some of these folks are actually using these tools for real. My own experience, and interacting with others or via Reddit and so forth, shows a usually more challenging and meandering path. That’s ok. I get it. The Happy Path is easier to write about and slap together a LinkedIn post or YouTube or whatever. Unfortunately, it also obscures some likely realities.
Here’s my message as we drive through the messy parts. You’re ok. It’s not you. These things can still be a little sloppy. Just keep pushing through.
[Read more…]