post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Guest Post

How to build a companion product and win user trust

mission statement

Deep dive: Crafting your product vision and mission

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

Join us at #mtpcon London in 2025 – mark your calendars

Connecting fans to creators at Spotify (Dariusz Dziuk, Product Lead, Spotify)

Why teams often struggle with OKRs: A conversation with Jeff Gothelf

5 ways to invent and simplify as an AI/ML product manager

Darian has a diverse background ranging from air &amp; space defense, AWS, and OT cybersecurity. He has a passion for sharing my experiences in product management, development, and leadership. Currently, he is a senior product development manger at Rockwell Automation with a focus on B2B cybersecurity services and solutions

Darian Chavira

<div></div>
<ul>
<li>Are you stuck trying to figure out your product-market fit?</li>
<li>Do you feel that your team is making assumptions on tribal knowledge and not data?</li>
</ul>
If so, this article is for you and will explore the use of data science in product development.
<h2>Data, data, everywhere</h2>
Data science is an integral part of product management and culture. <a href="https://www.mindtheproduct.com/talking-about-data-science-by-jenessa-lancaster/" target="_blank" rel="noopener">Data science</a> allows product teams to move from focusing on time-to-market to time-to-outcome.
In a time-to-market model product teams rush to market to find out how their product sells. In time-to-outcome model product teams work towards an outcome based on customer data and input.
&#8220;Data is the new electricity of our economy, and just like electricity, it is touching every part of our society. AI is one of the most important technologies of our time, but it’s really customer data that&#8217;s the fuel that powers AI.&#8221; Satya Nadella, CEO of Microsoft
Machine learning (ML) is dependent on high-quality data for generating accurate predictions. The integrity and relevance of data are crucial in determining the effectiveness and dependability of ML models. Here&#8217;s how to ensure the collection of quality data:
<ul>
<li>Design targeted surveys tailored to specific analytical goals.</li>
<li>Establish clear data collection objectives in early-stage product development.</li>
<li>Iteratively refine data collection methods and analytic tools.</li>
</ul>
<h2>Selecting the right model</h2>
<h2><img decoding="async" class="alignleft" src="https://docs.google.com/drawings/d/s_E_1YGlMBtG5I-CMPRds3Q/image?parent=1IwHrzP2a-1g0PBWqcW2-Rj--Pm2YPhCI9FoE-ISZSf4&amp;rev=1&amp;drawingRevisionAccessToken=kixYT54ZTYylbw&amp;h=390&amp;w=533&amp;ac=1" alt="" /></h2>
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
Assuming we collected quality data, we can select our <a href="https://www.mindtheproduct.com/machine-learning-for-product-managers-a-quick-primer/" target="_blank" rel="noopener">ML model</a>. The model is dependent on the data and the business outcome.
To be most effective, we should have our outcome in mind when we develop our data collection plan. Working backwards flow:
Thought Process
Business Outcomes → ML Model → Data Prep → Data Collection → Data Planning
Execution
Data planning → Data Collection → Data Prep → ML Model → Business Outcomes
<h2>ML model examples</h2>
There are a ton of AI/ML/Deep Learning models that could drive outcomes. Some typical business use cases: Customer Churn Analysis, Market Basket Analysis, Marketing Analysis, Operations Analysis, Prediction, Automation, Sentiment Analysis, Recommendations, so forth and so on.
I outline three models: Market Basket Analysis, Customer Churn, and Marketing Analysis.
All pictures are from RapidMiner on synthetic data created on a local repository.
<h2>Market basket analysis</h2>
My customer bought widget X from me, and I wish I could know what other widgets they would buy? Or do our current customers have this install base what should we offer next?
<img loading="lazy" decoding="async" class="alignnone wp-image-41041" src="https://www.mindtheproduct.com/wp-content/uploads/2024/03/5478a189-7d5f-4dcf-827b-6bb9ae61ff6a_Untitled-300x190.png" alt="" width="714" height="452" srcset="https://www.mindtheproduct.com/wp-content/uploads/2024/03/5478a189-7d5f-4dcf-827b-6bb9ae61ff6a_Untitled-300x190.png 300w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/5478a189-7d5f-4dcf-827b-6bb9ae61ff6a_Untitled-1024x647.png 1024w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/5478a189-7d5f-4dcf-827b-6bb9ae61ff6a_Untitled-768x486.png 768w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/5478a189-7d5f-4dcf-827b-6bb9ae61ff6a_Untitled.png 1142w" sizes="(max-width: 714px) 100vw, 714px" />
Okay, let’s break this down.
1. Data:
a. Involves loading transaction data that contains a transaction id, a product id, and a quantifier. The quantifier indicates the number of times a product has been purchased in a transaction.
2. Prepare data through exploration and preparation.
b. Data needs to be cleaned and balanced.
3. FP-Growth:
a. This is a step where the FP-Growth algorithm is used to determine frequent item sets. FP-Growth is an efficient and scalable method for mining the complete set of frequent patterns by pattern fragment growth, using an extended prefix-tree structure for storing compressed, crucial information about frequent patterns called FP-tree.
b. Frequent item sets: These are sets of items that appear together in transactions more often than a defined threshold (support). Decision Tree and/or Random Forest outcomes
4. Create Association:
a. This step involves using the frequent item sets to generate association rules. These rules can suggest that if a customer buys a certain set of items, they are likely to buy another item (or set of items). The strength of these rules is determined by metrics like confidence and lift.
5. Outcomes:
a. Association Rules: Rules that indicate a strong relationship between the purchase of one set of items and another.
b. Frequent Item Set: The commonly purchased sets of items.
Image 1 shows all associations for potential products my customers would buy based on their current purchasing habits.
Focusing on the highest confidence level I narrow my product associations to four products with one association and one product with two associations. Product 20 would be the most likely candidate based on our customers previous purchasing history.
This product selection can be validated through surveys and interviews either externally with customers or internally with sales/commercial leaders.
<img loading="lazy" decoding="async" class="alignnone wp-image-41040 size-full" src="https://www.mindtheproduct.com/wp-content/uploads/2024/03/a77e6346-f30d-475a-8095-49dd76276511_Untitled.png" alt="" width="1483" height="586" srcset="https://www.mindtheproduct.com/wp-content/uploads/2024/03/a77e6346-f30d-475a-8095-49dd76276511_Untitled.png 1483w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/a77e6346-f30d-475a-8095-49dd76276511_Untitled-300x119.png 300w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/a77e6346-f30d-475a-8095-49dd76276511_Untitled-1024x405.png 1024w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/a77e6346-f30d-475a-8095-49dd76276511_Untitled-768x303.png 768w" sizes="(max-width: 1483px) 100vw, 1483px" />
<img loading="lazy" decoding="async" class="alignnone wp-image-41039 size-full" src="https://www.mindtheproduct.com/wp-content/uploads/2024/03/f33ccd0b-dcbc-4f8f-974d-2927d4c28631_Untitled.png" alt="" width="1477" height="565" srcset="https://www.mindtheproduct.com/wp-content/uploads/2024/03/f33ccd0b-dcbc-4f8f-974d-2927d4c28631_Untitled.png 1477w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f33ccd0b-dcbc-4f8f-974d-2927d4c28631_Untitled-300x115.png 300w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f33ccd0b-dcbc-4f8f-974d-2927d4c28631_Untitled-1024x392.png 1024w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f33ccd0b-dcbc-4f8f-974d-2927d4c28631_Untitled-768x294.png 768w" sizes="(max-width: 1477px) 100vw, 1477px" />
<h2>Customer Churn</h2>
Predicting when your customers will churn and why through decision trees and random forests.
<img loading="lazy" decoding="async" class="alignnone wp-image-41038 size-full" src="https://www.mindtheproduct.com/wp-content/uploads/2024/03/c81a16c7-218c-43c8-aea0-5ff736c85aac_Untitled.png" alt="" width="1037" height="800" srcset="https://www.mindtheproduct.com/wp-content/uploads/2024/03/c81a16c7-218c-43c8-aea0-5ff736c85aac_Untitled.png 1037w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/c81a16c7-218c-43c8-aea0-5ff736c85aac_Untitled-300x231.png 300w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/c81a16c7-218c-43c8-aea0-5ff736c85aac_Untitled-1024x790.png 1024w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/c81a16c7-218c-43c8-aea0-5ff736c85aac_Untitled-768x592.png 768w" sizes="(max-width: 1037px) 100vw, 1037px" />
Okay, let’s break this down.
1. Start with historical data of customer retention and churn.
a. Hopefully, you have a repository of customer interactions.
b. No customers, stop. Churn modeling is a later problem, not a now problem.
2. Prepare data through exploration and preparation.
a. Data needs to be cleaned and balanced.
3. Select model: Decision Tree and/or Random Forest
a. Key callouts: configuring model for accuracy, recall, and prediction is important to get desired outcomes.
4. Decision Tree and/or Random Forest outcomes
a. Predict the likelihood of future customer churn.
<img loading="lazy" decoding="async" class="alignnone wp-image-41037 size-full" src="https://www.mindtheproduct.com/wp-content/uploads/2024/03/f4075af8-2ced-4015-a143-d8addd5bc8f8_Untitled.png" alt="" width="1200" height="431" srcset="https://www.mindtheproduct.com/wp-content/uploads/2024/03/f4075af8-2ced-4015-a143-d8addd5bc8f8_Untitled.png 1200w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f4075af8-2ced-4015-a143-d8addd5bc8f8_Untitled-300x108.png 300w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f4075af8-2ced-4015-a143-d8addd5bc8f8_Untitled-1024x368.png 1024w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f4075af8-2ced-4015-a143-d8addd5bc8f8_Untitled-768x276.png 768w" sizes="(max-width: 1200px) 100vw, 1200px" /> 
Based on the decision tree, the factors contributing to customer churn can be prioritized as follows:
<ol>
<li>Number of support calls last year: This appears to be the most significant predictor of churn. Customers making more than 9.5 support calls are most likely to churn, indicating that a high number of support calls is a strong indicator of dissatisfaction.</li>
<li>Average bill amount: For customers with fewer than 9.5 support calls, the average bill is the next most significant factor. An average bill amounts greater than 14.5 leads to churn, suggesting that higher bills may be a churn factor among customers who don’t have extreme numbers of support calls.</li>
<li>Customer since (Date): For those with fewer support calls and a lower average bill, the duration of their relationship with the company plays a role. Newer customers (since after April 18, 2021) are more likely to churn than those who have been customers for a longer time.</li>
<li>Support calls last year (a lower threshold): At the lower end of support calls and average bill, if the customer made more than 4 support calls, they are predicted to churn. This indicates that even at lower levels, the number of support calls is still a predictor of churn, albeit less significant than the higher threshold of 9.5 calls.</li>
</ol>
<h2>Product and direct marketing</h2>
<img loading="lazy" decoding="async" class="alignnone wp-image-41036 size-full" src="https://www.mindtheproduct.com/wp-content/uploads/2024/03/f137b376-ffe2-4378-a2b5-463f4e95ffd1_Untitled.png" alt="" width="1035" height="791" srcset="https://www.mindtheproduct.com/wp-content/uploads/2024/03/f137b376-ffe2-4378-a2b5-463f4e95ffd1_Untitled.png 1035w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f137b376-ffe2-4378-a2b5-463f4e95ffd1_Untitled-300x229.png 300w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f137b376-ffe2-4378-a2b5-463f4e95ffd1_Untitled-1024x783.png 1024w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/f137b376-ffe2-4378-a2b5-463f4e95ffd1_Untitled-768x587.png 768w" sizes="(max-width: 1035px) 100vw, 1035px" />
Create a customer response model based on past responses to targeted marketing campaigns to predict those customers that are likely to respond to new campaigns and increase the conversion rate.
Okay, let’s break this down.
1. Start with historical marketing data.
2. Based on the historical calculate relevant weights for attributes.
a. Heavier weight means the attribute has more significance.
3. Apply a model to create ideal new customers to market to based historical results.
4. Select visualizations to represent models.
This model created a breakdown of the probability of sales representatives getting responses from their target list based on customer communication preferences, purchasing preferences, and recent communications in the data set.
Humphrey (salesperson) has a 70% probability of getting call backs from his list. From this analysis, we can further analyze the customers on the list and target a <a href="https://www.mindtheproduct.com/understanding-ai-based-data-and-insights-led-digital-product-management/" target="_blank" rel="noopener">customer segmentation</a> based off this list.
<img loading="lazy" decoding="async" class="alignnone wp-image-41035 size-full" src="https://www.mindtheproduct.com/wp-content/uploads/2024/03/5dade57c-a88b-42e0-9d42-b639f47adb85_Untitled.png" alt="" width="1600" height="855" srcset="https://www.mindtheproduct.com/wp-content/uploads/2024/03/5dade57c-a88b-42e0-9d42-b639f47adb85_Untitled.png 1600w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/5dade57c-a88b-42e0-9d42-b639f47adb85_Untitled-300x160.png 300w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/5dade57c-a88b-42e0-9d42-b639f47adb85_Untitled-1024x547.png 1024w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/5dade57c-a88b-42e0-9d42-b639f47adb85_Untitled-768x410.png 768w, https://www.mindtheproduct.com/wp-content/uploads/2024/03/5dade57c-a88b-42e0-9d42-b639f47adb85_Untitled-1536x821.png 1536w" sizes="(max-width: 1600px) 100vw, 1600px" />
<h2>Summary</h2>
We all have limited time in our day.
Product managers are always fighting uphill against customer calls, support tickets, and never-ending emails.
Using AI for automation and ML for prediction is one more tool in the product managers tool bit.
Explore more great product management content by exploring our <a href="https://www.mindtheproduct.com/content-a-z/" target="_blank" rel="noopener">Content A-Z</a>

core/freeform

How Google makes AI work at Enterprise scale – Miku Jha (Director, AI/ML and Generative AI, Google Cloud)

LLM workflows for product managers: 3 key takeaways (Niloufar Salehi, Assistant Professor at UC Berkeley) – ProductTank SF

The future of product management: Insights from ProductTank San Francisco

Product lessons learned making early moves with AI in media: Lindsey Jayne (CPO, Financial Times)

A year with ChatGPT and product innovation: Navigating the AI landscape

How to keep your head about generative AI (when everyone is losing theirs) by Claire Woodcock

How to obtain Product- Market fit with AI

Darian Chavira

Read more

Comments