post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Artificial Intelligence (AI) & Machine Learning (ML)

Microsoft lays off 3% of workforce while renogiating partnership with OpenAI

Podcast

How to build the right product culture during transformation – Joca Torres (Product Consultant)

B2B product management

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

Context, intent, change, purpose, constraints – a model for communicating requirements

5 difficult conversations and how to have them

DoorDash eats Deliveroo while OpenAI makes its biggest buy: This week’s news roundup

Why great product managers think like therapists

Tools and techniques to scale product teams – Charlotte King (Lead Product Manager, eBay)

George Fletcher is a Senior Product Manager that has been working in product for over a decade.  A passionate product manager promoting lean product management, rapid prototyping and the adoption of design sprints throughout the businesses he's worked for.  Having built successful products in a variety of different industries George has found a home in the acceleration and incubation hubs in larger corporate environments helping transform mindsets and processes towards lean, digital thinking.

George Fletcher

<p>In this ProductTank London talk, Ben Fields, formerly Lead Data Scientist at FutureLearn, looks at getting the most from the relationship between product managers and data people, working out people&#8217;s background understanding, the benefits of leading with a story, and how best to spread knowledge.</p>
<p>There are three key aspects of how data science can effectively sit within a product and software organisation:</p>
<ol>
<li>Gear and orient to your audience</li>
<li>Optimise for diversity</li>
<li>Embed with product</li>
</ol>
<h2>1. Gear and Orient to Your Audience</h2>
<p>How can you show your audience content that they would not otherwise have found? Whether a site is selling ecommerce products or promoting editorial content, delivering an accurate relationship between what the user is presently viewing and then where you direct them next is more likely to lead to conversion.</p>
<h4>A Story About Popularity Bias</h4>
<p>Ben says that a recommender system and widget built at FutureLearn attempted to orientate the user to the course they were viewing and then gear and direct them to other possible, related options &#8211; options that they may not have encountered without this widget.</p>
<p>A recommender system should make effective use of data that an application has available. However, this can push a model to decisions that might not be the best for the viewing user. Popularity bias can creep into data models.</p>
<p>The FutureLearn recommender system would promote courses based upon similar enrolment patterns and behaviour from the wider user base.  o, at its simplest if Person A enrolled in Course 1 and Course 2, then when Person B views Course 1 they may also be recommended Course 2.</p>
<p>However, FutureLearn found that its recommender system kept promoting a similar set of courses that didn&#8217;t necessarily have a fair relation to the course being viewed. This was because the system took popularity into account and one provider, the British Council, provided a number of courses that were popular among FutureLearn users. As a result, a lot of British Council courses appeared in the widget. This meant that the widget was not meeting the aim of showing courses that users would not have found otherwise.</p>
<blockquote><p>If we&#8217;re just repeating what we already saw then what work are we doing?</p></blockquote>
<p>Ben has another story about popularity bias. A popular book retailer had with a “people who bought this also bought…” widget. When browsing the product page for a Java RMI (remote method invocation) book the widget recommended two other software engineering related books and then finally &#8230; Harry Potter.</p>
<p>It this wasn&#8217;t incorrect, it just isn’t what was wanted and likely had little value. The system picked out Harry Potter because “everyone” has bought Harry Potter.</p>
<p>It is very difficult to move from popular courses, books, content, artists, products, deep into the long-tail, less popular and less visible items. However, that longer tail can hold a lot of dormant value which can be unlocked if it is seen by users at relevant opportunities.</p>
<h2>2. Optimise for Diversity</h2>
<p>One way to combat popularity bias is to optimise for diversity. When designing a complicated system such as a behavioural recommender, it is key to spend time working with the people who will find the edges of the requirements.</p>
<h4>Telling and Supporting Your Stories With Data</h4>
<p>FutureLearn wanted to build a prediction engine that would predict course enrolment and revenue. The system would take the known information about the course due to run and model it against behaviour seen on the site to get enrolment and revenue figures. Start date, course length, how many times has it been run, how much learning there is, and course density, are some of those elements that can be coupled with other factors such as seasonality, peaks, and troughs.</p>
<p>Using a machine learning tool called TPOT, the team at FutureLearn was able to cut short system setup and prove it worked.  It took a week to show that the system was able to predict revenue when given a set of data attributes.</p>
<p>This lean approach gave FutureLearn an immediate output and evidence base, supported by data. While not pristine, the system gave better predictions about which courses might be better than others than the previous rule-based, business process methods.</p>
<h2>3. Embed With Product</h2>
<p>Finally, the FutureLearn team found success by keeping their data scientists close to product. The product team outnumbered the data scientists in the business but this was offset by using an embedded rotation system. Data scientists moved around and embedded into the various product teams based upon priority and focus.</p>
<h4>Priority</h4>
<p>Here the data team would consider need and ask when/where are the data scientists going to be most effective?  Which product team needs them, when and for what scale of project?</p>
<h4>Focus</h4>
<p>The battle here is to ensure that the decision aligns with the strategic aims of the company. By serving and orienting to the business focus the team could better put the resources where they could be most effective.</p>
<p>Ben’s final parting thought about scarce resources was that effective internal communication and an understanding of the value that data science will allow companies to be far more effective.</p>


core/freeform

AI ethics advice from former White House technologist – Kasia Chmielinski (Co-founder, The Data Nutrition Project)

How Google makes AI work at Enterprise scale – Miku Jha (Director, AI/ML and Generative AI, Google Cloud)

LLM workflows for product managers: 3 key takeaways (Niloufar Salehi, Assistant Professor at UC Berkeley) – ProductTank SF

The future of product management: Insights from ProductTank San Francisco

Product lessons learned making early moves with AI in media: Lindsey Jayne (CPO, Financial Times)

A year with ChatGPT and product innovation: Navigating the AI landscape

How to keep your head about generative AI (when everyone is losing theirs) by Claire Woodcock

What we get wrong about technology by Tim Harford

Product management in the age of ChatGPT by Yana Welinder

How Canva uses AI-powered features to drive PLG

They’re Just Fancy Averages by Ben Fields

ProductTank Pune

ProductTank Hyderabad

ProductTank Valencia

ProductTank Auckland

ProductTank Helsinki

ProductTank Barcelona

ProductTank Lomé

ProductTank Wellington

ProductTank San Diego

ProductTank Canberra

ProductTank Perth

ProductTank Hong Kong

ProductTank Bangkok

ProductTank Doha

ProductTank Tel Aviv-Yafo

ProductTank Baku

ProductTank Reston

ProductTank Raleigh

ProductTank Charlotte

ProductTank Washington

ProductTank Richmond

ProductTank Cluj-Napoca

ProductTank Zagreb

ProductTank Amsterdam

ProductTank Edinburgh

ProductTank Paris

ProductTank Hamburg

ProductTank München

ProductTank Manchester

ProductTank Bristol

ProductTank Oxford

ProductTank Madrid

ProductTank Des Moines

ProductTank Dallas

ProductTank Portland

ProductTank Curitiba

ProductTank Chicago

ProductTank Austin

ProductTank Philadelphia

ProductTank Montevideo

ProductTank São Paulo

ProductTank Los Angeles

ProductTank Gdansk

ProductTank Stuttgart

ProductTank Pittsburgh

ProductTank Bangalore

ProductTank Chennai

ProductTank Mumbai

ProductTank Fribourg

ProductTank Norwich

ProductTank Zürich

ProductTank Newcastle Upon Tyne

ProductTank Jacksonville

ProductTank Florianópolis

ProductTank Tallinn

ProductTank Denver

ProductTank San Francisco

ProductTank Sheffield

ProductTank Tokyo

ProductTank Valletta

ProductTank Bologna

ProductTank Vancouver

ProductTank Milano

ProductTank Columbus

ProductTank Belfast

ProductTank Montréal

ProductTank Seattle

ProductTank Lausanne

Product Management Foundations BST

Product Management Foundations ET

ProductTank Reading

ProductTank Bielefeld

Communication & Alignment BST

Communication & Alignment ET

ProductTank Frankfurt

ProductTank Bonn