What AI can’t—and shouldn’t—do for product managers

Protecting human connections from the AI-ification of everything.

AI is transforming every aspect of product management. However, these solutions threaten to make building great products harder.

In a Slack conversation about automating user research, we enthusiastically discussed how AI could handle scheduling, recording interviews, and taking notes. 

We excitedly explored taking it further and further into the user research lifecycle. Little did I know that this enthusiasm would lead to a gentle scolding from none other than Teresa Torres (I'll share more on that later).

AI is going to completely change how we do product management. At the same time, we have to draw the line somewhere. As product people, our primary job is to understand user and stakeholder needs. We can't outsource that to AI. 

In February, I hosted a webinar on AI in product ops. We showcased three demos, including my own product blueprint bot, and the potential is mind-blowing. It became crystal clear that we're just scratching the surface of how AI can supercharge product teams.

These demos highlighted ways for AI to enhance product management productivity:

The allure of automation and efficiency is undeniable. Who wouldn't want a job that's all creativity and no busywork? AI promises to take care of the mundane tasks, freeing us up to focus on the high-level, strategic thinking that drew us to product management in the first place.

We cannot forget: great products are built on human connection. We build products for real people, which requires deep empathy and understanding. 

Be mindful of where we deploy AI and ensure it's enhancing our ability to connect with users and stakeholders, not creating barriers.

Fully automating user research sounds alluring. AI tools promising to handle everything from creating studies, allowing you to interview fake users, and synthesizing insights are popping up everywhere. 

Even I got caught up in the excitement, until Teresa Torres brought me back down to earth. She argues that the work we do to synthesize our interviews is what really matters – that’s how we get an understanding of what happened in the interview. Reading someone else’s report is never the same as processing it yourself.

While these tools offer efficiency, they risk disconnecting us from the very essence of user research. We should be doing a majority of our user research manually because we need to understand our users, not just have reports about them.

Here's why we need to keep that knowledge in our brains:

Use AI to streamline tedious tasks in user research. Use it as a thought partner and to summarize what you’ve already reviewed. Use it to pull up relevant video clips and categorize feedback.

Resist the temptation to outsource the entire process. The magic of product development lies in those human-to-human connections and the insights that come from accumulated observations layered over time. 

Just as we need those human-to-human connections to understand our users, we need real connections with stakeholders to get things done across complex organizations.

AI-generated stakeholder reports should be standard in every company. Nobody likes writing status updates. But there’s a risk to them that we must acknowledge. 

A world where everyone reads AI-generated reports or constantly asks an LLM to understand what other teams are doing risks increased mis-alignment. 

Rather than do the hard work to get everyone on the same page, the AI will just communicate summaries back and forth that are in disagreement with each other. 

These are the reasons why AI reporting risks creating more misalignment between teams: 

Alignment is based on shared understanding and relationships, which AI can’t do for us. If AI-generated reports can get us out of the silos and talking to each other (and out of long status meetings), I welcome it. But creating meeting summaries isn’t the same as actually creating alignment. 

Used correctly, these tools should actually allow us to have stronger relationships – making sure you’re sending stakeholders just the right level of detail that they need, or that you can reach out to customers who are experiencing the exact problem you’re trying to fix. 

So what are the signs that you’re letting AI get between you and your relationships? 

When I run a product ops assessment at a new company, I do all the interviews myself.  I considered automating these interviews until I realized they aren’t just about information. They’re about trust.

That trust enables me to influence change later. The people I’m working with need to know I’m listening and that I care so they actually change later when we find the right solution. Deploying a bot to interview them sends the wrong message.

Look at the work you’re doing and check for these symptoms – if they’re present, you’re letting AI get in the way of your work, not help it. 

We might know the signs of when we’ve gone overboard with AI usage, but finding the right opportunities to use AI is just as important. 

These are the approaches I’m taking to AI tooling: 

Decide where the line is for you– what will you allow it to do for you, what do you want to do together, and what will you continue to do yourself. Teresa just published a first draft of her thoughts on where the line should be. 

When you feel that surge of excitement about a new AI use-case, ride that wave. Just take a pause before you go diving in head-first to make sure it’s actually going to help you build a better product. 

If we get to the point where product management is dead (presumably by AI in the cupboard with a GPU), much more than the title “product manager” will have died. The ability to build truly user-delighting products will be gone too.