TetraMesa

Upgrading an AI with a RAG Vector DB

By

In a previous post about Building a PM Helper with AI, I showed how a fun personal AI project I’d built to be my personal Product Management tool searched across multiple sources before synthesizing answers. Unfortunately, I made both a strategic and a tactical error in that 1.0 version. The solution? Using Retrieval Augmented Generation with a Vector Database. What I’m going to do here is offer some super fast high level definitions as I go through the problem space, and maybe in future posts, go more deeply into RAG and Vector databases in terms of value.

tl;dr:

  • If your a product manager working with AI at any level, you will likely need to understand Retrieval Augmented Generation (RAG) to some degree. The following is a small, practical use case to help see the value in action.
  • For the most part, when using LLMs for your own custom work, you’re stuck with the foundation model.
  • Fine tuning can change the weights of the model to various levels, depending on how deep you want to go. These weights basically control how models transform input and can be in the billions. The deeper you want to have impact, the higher the cost. (You’re not likely fine-tuning for personal projects though. And if you are, it will all but certainly be with open source foundational models.)
  • Retrieval Augmented Generation (RAG) doesn’t change weights at all. RAG just passes more information into a Prompt, (which is a fancy name for an information query, unless you really add fuller instructions ), but is limited to something called a context window. Basically, how much info you can pass in. It’s like saying, “Here, read this before answering.” So theoretically RAG reduces the chances of hallucinations and offers more “truthy” answers. (Assuming good data in what you feed it.)
[Read more…]

Building a PM Helper with AI

By

What are We Doing?

We’re going to look at a plaything I built in just a handful of hours while digging into agentic AI a bit just for fun. (Well, for career-related things as well, but mostly for fun.) The toy is an AI enabled Digital Product Manager assistant app where you can ask questions about product management. Which, as it turns out, could actually be a real product. I built it as a toy project for fun, but might actually soft launch this thing. Because, why not? Can you just do this with any GPT? Sure. But this one is tuned specifically to product management in general and digital product management in particular. (There are others like it though, so maybe just leave it as a personal tool. We’ll see.)

Why do this? Why bother. And why do you care? Since the vibe coding thing is so very in right now, (maybe for another few months), I figured it was time to jump in a bit. While I’ve built some agent workflows in the past and built a variety of apps, it’s usually been team based. This is one of those, “Hey, if I can do this… anyone can” type posts. The question, of course, is do you have a reason / use case? But the whole argument that used to exist about some things being “too hard” or “too technical” or “too much time” are fading away with some of these tools. Not completely. And some things absolutely – my opinion – require “real” developers. But others? Lower risk things? Personal productivity things? Not as much anymore. So I’m going to go through my process just at a super high level. My goal is to convince other product manager types to dive into this area more deeply than just watching a webinar and learning some of the lingo. Even for senior roles and beyond individual contributor roles, I personally think it’s useful to get a visceral feel for how things work. Doing so helps offer better context for what teams might be going through as well as understanding in what might be possible. And also, getting a sense of what budget implications might be if you’ve got P&L responsibility for a product.

Next we’ll get into the details, but if you want to see the end result, it’s here: (But note, the functionality won’t work as I’ve got the public webhooks turned off so as not to burn up my paid quotas on the services in use. To see it working, check the Loom Demo.)

Direct Link to the App Test Website

App Demo / Loom Video

[Read more…]

AI Vibe Coding – The Good, The Bad, The Tragically Ugly

By

There’s good reasons why vibe coding is on a tear right now. In some ways, there’s potentially legitimate value that will come out of this. At the same time, I think it’s likely we’ll see some seriously tragic outcomes. Just in case you’re not up on the meme, vibe coding is simply a term for using AI to write computer code. (See the link for the origin of the meme.) Maybe you could argue that if we’re going to label things, you could call it vibe composing or vibe whatever-ing, given that there’s generative tools for a variety of object types from art to music and so on. But let’s focus on computer code for now.

So. Besides just having programmers possibly be more efficient, what are the benefits? Perhaps one of the loudest benefits right now is being shouted by a cohort of entrepreneurs and product managers. They’re thrilled! “I can just get software now!”

“I am free!”

What they mean is free from the stranglehold technology costs have had on the ability to get digital product done. The problem with the Temple of Tech has always been time, money, and perhaps worst, resource contention. And those pesky coders had something else the Kingpins of history have always had, “They who control the means of production control the world.” But now? Now it’s democratized! No more desperate postings by non-technical co-founders seeking technical help for zero dollars, but a little bit of stock. There’s perhaps some irony here if you look at historical Bourgeoisie Control the Means of Production. But we can skip the philosophy for now. If you’re old enough, you can remember that if you had a development request, you’d maybe literally head to some special room behind secure doors in a basement where florescent lights battled with the glow of the green or orange text emanating from terminals. There’d be a faint smell of ozone as even the loud ventilation system couldn’t quite keep up with all the buzzing electrons. Here, you would make your tech request. Ironically, it’s not that different now in many places. At least, from a relationship perspective, even if the scenery has changed.

So can entrepreneurs and product managers just ditch their tech colleagues now? Are the headlines true? Coding skills are no longer a hot ticket? Let’s take yet another deep breath. Come on with me now… breeethe… in through the nose, out through the mouth… one… two… three… Good. Yes. Things are changing again. And fast. But maybe don’t just lay off everyone just yet.

[Read more…]

Intrinsic / Extrinsic Product Values Framework 

By

In a prior post about Intrinsic / Extrinsic Product Value Dynamics, we looked at the basic differences between intrinsic and extrinsic values and consumer perceptions. Here, we’re going over some frameworks you can consider using to define and position your products along these dimensions. The idea is to make sure that everything from features to messaging align with intended values.

Please note that this framework is a suggested proposal; not something I’ve seen used or tested. It focuses on creating and expressing value deeper in product features and structures rather than surface level messaging. (Which is of course still critical.) There is no experimental data of which I’m aware that tests on varying such dimensions against each other. (Though in the Value Dynamics article, we did go over examples of successful and challenged products.) My goal is to offer a structure to explicitly consider these dimensions and use them as input to a larger scope product positioning strategy that would include the other usual elements of actual features, pricing, competition and so on.

[Read more…]

Intrinsic / Extrinsic Product Value Dynamics

By

As a Digital Product person, do you think about intrinsic and extrinsic values of your product or service? Traditional marketing has generally considered these issues from a high-level marketing and messaging perspective. But how often do you see them considered deep within the features and benefits of digital products? Not much, with the sometimes exception of gaming products and perhaps some health apps. And yet, the full User Experience (UX) that goes beyond the interface imbues our products with features that touch on one or the other or both. Game designers may deeply consider game mechanics, (rules and interactions), and dynamics, (emergent behaviors, experiences), but what about the rest of us?

The question is, when crafting products, are you considering intrinsic vs. extrinsic dimensions? At the highest level of Vision and Mission through Strategy right down to Story Creation and execution, we’re all imbuing our brands with… with… with what? The answer is likely with both intrinsic and extrinsic values; both in fact and perception. The question is: Are we doing so with intentionality.

Sometimes we can get stuck in feature factory mode and this presents a variety of risks. One is the time drain that prevents us from considering bigger picture items. Such as making sure features are truly aligned with our intended intrinsic and extrinsic values. Some of these product value dynamics might be an exercise for marketing more so than product. This can depend on company size, spans of responsibility, and the nature of cross-functional partnerships and team structures. Whomever is responsible for considering such things, it pays to execute such things properly.

[Read more…]

Demystifying Crypto – Tokenomics & Stablecoins

By

In past articles, I’ve tried to offer some of the basics of crypto as well as simplified understanding of the blockchains themselves. You can see these here, Demystifying Crypto Basics through Metaphor, and here, Demystifying Crypto – Blockchain Layers.

There are a couple of other fundamental topics worth discussing.

Tokenomics

Let’s talk Tokenomics. Tokenomics is a portmanteau word, derived from ‘token’ and ‘economics’.

Tokenomics is the economic design and structure of a cryptocurrency token, encompassing how it is created, distributed, used, and managed within its ecosystem. It includes factors like supply (fixed or inflationary), distribution and allocation, utility (how the token functions within a platform), governance (voting rights or protocol decisions), and incentives (rewards for participation, such as staking or mining). Well-designed tokenomics align incentives for users, developers, and investors, ensuring the token’s sustainability and growth within its network.

If you’re thinking of building a product or service that’s going to find using its own native tokens or coins useful, you’ll need to go a lot deeper than what’s going to be described here. Consider this as just a primer.

[Read more…]

Demystifying Crypto – Blockchain Layers

By


In Demystifying Crypto Basics through Metaphor, we started with some of the basic ideas about money and crypto tokens. But we didn’t talk about where these things live.

Blockchains

We kind of skipped this part. But I thought it was more important to go over some of the pieces first, then go big picture. So what’s a blockchain?

Here’s my simple definition of blockchains: It’s a place where you keep track of some stuff and let tons of people keep copies in a really secure way. That is really it.

Here’s the more typical matter-of-fact technical definition: A blockchain is a decentralized, cryptographically secured ledger that records transactions in a tamper-resistant chain of blocks across a distributed and decentralized network. There’s other details like how blocks are formed and linked, timestamps, and more. We’re not going to focus on those. Blockchains are mostly associated with crypto in popular perception, but guess what? They can be used for all kinds of other things; identity management, supply chain management, healthcare records, real estate and land titles, and a lot more. This is why this technology is simmering beneath the surface. It does have several core values and can be made actually useful. It’s not just about the so-called meme coins.

In the hype cycle of crypto, we’ve seen the whole market space rocket up and come back down, then slowly rise again. At least in terms of money and attention, it’s now just kind of there. After all, the Metaverse was popular for a couple of days, but then we got AI to gush about. Meanwhile, we still have the get rich YouTube crypto maxis, but the crypto industry itself has started – more quietly – finding some real world valuable use cases. And the underlying blockchain tech? Players in this area have been just trying to build more things of actual value. So it’s not going away. And if you do product management, it’s quite possible you’ll be looking at where or if some of these technical bits make sense in your offerings.

[Read more…]

Demystifying Crypto Basics through Metaphor

By

Fair Warning: If you’re already an experienced crypto user or builder, this article and any follow-ups likely aren’t for you. It’s also a long form article series. If you prefer snippets, this also might not be for you.

This is for friends and product management colleagues still looking to more deeply understand or round out understanding on some underlying concepts in the cryptocurrency and token space. If as a product person you’re still not up to speed on crypto, consider the following:

  • It is Happening. As with so many hype cycle things, we’ve seen the usual irrational exuberance periods and such. Sadly, crypto has been disparaged with the same – often clickbait foolish – brush of just being the latest ‘net craze of the year. At this point, it should be clear there’s plenty of real world use cases. Various types of crypto solutions are being adopted by major firms in finance and elsewhere, including early naysayers. And it will continue to impact product decisions across multiple industries.
  • Crypto Will Couple with AI: As intelligent agents begin to find real use cases, it is highly likely a wide variety of them will end up tightly coupled with crypto. The reason is simply that agents will need to transact at certain points in their journey. The most likely best means for them to do so will be via crypto tokens. How and where remains to be seen. But if you’re building with agentic AI, it’s possible you’re going to need to interface with some aspect(s) of crypto payment rails.

This is not a straight-line path “Intro to Crypto” article series. There is no clear beginning to end here. I’m not going to tell you how to get a crypto wallet. There’s plenty of that out there and all easily findable. And yet, it seems to me there’s some challenges understanding some basic concepts. So I wanted to try something maybe a bit different. My goal here isn’t to just offer a description or two. It’s to understand some basic concepts, and where and why they have value, and we’re going to take a long winding road to get there.

[Read more…]

GenAI UX Issues for Product Managers

By


In GenAI and Search: Differences from a Product & UX Perspective, we started looking at differences between search and GenAI from a user needs perspective. We looked at the tools in terms of use cases.

Now we’re going to turn more towards how we present things. This skips over everything in the Machine Learning Ops (MLOps) flow. And that’s okay. Because maybe we should be starting with the goal. After all, before you spend what could be millions, whether it’s for a consumer facing startup or an internal Enterprise tool, it’s probably wise to do some prototyping and testing anyway. (For MLOps, See: What is MLOps? (Amazon), What is MLOps? (Google), Why You Need MLOps.) By the way, I’m focusing here on user facing products, whether consumer or business. (As opposed to internal tools for analysis or production, marketing tools, etc. Though these certainly can have UX concerns as well.)

Figuring out how to design products to better serve users is of course not entirely your job. Whether you’re an entry level product manager or senior leadership, you’re ideally living more in the customer problem space. You’re looking across all things. Yes, you’re looking at features, functions, benefits. But also the dozens of other things to do. Which is why you work with your talented Design Leads. Whether direct reports or as a shared service, your design partners need to be getting up to speed on GenAI if they haven’t already. So your job is likely more along the lines of figuring out where or if AI is useful for your business; either for internal production or your actual products. And if it is, you might be the one – or at least be among the several – advocating for the resources to build out capabilities.

The classic question for Product Managers remains… How much do you really have to know in a specific domain; code, design, whatever? If you go too deep in any one area, chances are you won’t be very effective at your cross-functional tasks. This topic treatment is intended for skimming and basic understanding so you can work well with your talented design and tech colleagues. Our goal here isn’t to get anyone in Product to a practitioner level. It’s more to give you the tools to contribute effectively and have customer focused conversations with your specialist colleagues and drive requirements of value. Depth here once again depends on the type of product person you are. If you’re solidly on the business side, your whole product might be a P&L exercise for you and most functions and other team members are ‘just’ a line item on a spreadsheet as far as you’re concerned. A Technical PM? OK, you’re in the weeds with everything from APIs to whatever. Most of us are somewhere in between. One thing generally seems true though… if something comes up in a new domain where no one has clear responsibility yet, chances are you’re going to own it while an organization gets up to speed with where to put the new thing.

[Read more…]

Intro to AI Rubrics for Product Managers

By

What is a Rubric for AI Products?

A rubric is about evaluation and quality control, but also standardization, consistency and more. The origin of rubrics is from education and assessment so the term may be new to a digital product person. The general idea is to have a highly structured way to evaluate qualitative judgments. This seemed to be somewhat parallel to what was needed to evaluate AI output, so the model was adapted for that purpose. Rubrics for AI evaluation are used in academia, by tech companies, and regulatory and standards bodies. For traditional development, we have a variety of QA standards. A lot of them involve unit and integration testing and in modern workflows is often part of a continuous development and deployment plan. Rubrics can also be used along a development path, during early evaluation and fine tuning, pre-deployment, and for ongoing testing. However, at least a rough model must be fully available.

In the case of AI model quality assessment, a rubric is a structured framework for evaluation.

[Read more…]

Location

We're located in Stamford, CT, "The City that Works." Most of our in person engagement Clients are located in the metro NYC area in either New York City, Westchester or Fairfield Counties, as well as Los Angeles and San Francisco. We do off site work for a variety of Clients as well.

Have a Project?

If you have a project you would like to discuss, just get in touch via our Contact Form.

Connect

As a small consultancy, we spend more time with our Clients' social media than our own. If you would like to keep up with us the rare times we have something important enough to say via social media, feel free to follow our accounts.