post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Guest Post

How to build a companion product and win user trust

mission statement

Deep dive: Crafting your product vision and mission

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

Join us at #mtpcon London in 2025 – mark your calendars

Connecting fans to creators at Spotify (Dariusz Dziuk, Product Lead, Spotify)

Why teams often struggle with OKRs: A conversation with Jeff Gothelf

5 ways to invent and simplify as an AI/ML product manager

Phil Hopkins is Director of Product Management at Amazon Music. He is a  data-driven product and program manager, Python and Java developer, and team leader delivering strong ROI for consumer-facing digital apps and data pipelines.

Phil Hopkins

<p>Current efforts to introduce ethical or moral guidance to intelligent machines are useful, and increasingly urgent. The UK-based Institute for <a href="https://www.mindtheproduct.com/solving-ethical-ai-problems-when-algorithms-go-wrong-and-the-skills-needs-to-work-on-ai-solutions-insights-from-kriti-sharma/" target="_blank" rel="noopener">Ethical AI</a> and Machine Learning attempts to grapple with these issues, and has put forth several principles for those who make these products. But the discipline called Value Sensitive Design (VSD) has been investigating this matter in AI and non-AI contexts for years, and offers us some useful preparation for this moment.</p>
<p>As product managers, we should all be familiar with the concept of VSD. But as we see by even casual use of most apps and websites, not to mention buildings and vehicles, those who manage and design products are still under-informed about how, why, and when to put human values at the center of design decisions. As a disabled person from a family of disabled people, when I ask a search engine for information about how to enter a building, I am acutely aware of the extent to which human values are extended to include me. The range and meaning of human values is not definitive. Yet, they must be addressed and defined to some extent to permit technology designs to incorporate them on a conscious level. All implementations of technology and products reflect human values, even by default, reflecting values such as apathy or insensitivity in the cases of products that were not designed with human values at the forefront.</p>
<h2>Current efforts toward ethical AI</h2>
<p>The Institute for Ethical A.I. and <a href="https://www.mindtheproduct.com/product-management-in-the-era-of-ai-and-machine-learning/" target="_blank" rel="noopener">Machine Learning</a> has published its eight principles to guide implementations and design for intelligent machines from a human values standpoint, and those principles are a good starting place. Many of the Institute’s injunctions, such as those urging Reproducibility and Accuracy, are important for the management of applied A.I.. Still, they do not cut to the heart of the human element, the element that ethical A.I. and product frameworks must make central.</p>
<p>The Institute’s principles are:</p>
<ol>
<li>Human Augmentation</li>
<li>Bias Evaluation</li>
<li>Explainability by Justification</li>
<li>Reproducible Operations</li>
<li>Displacement Strategy</li>
<li>Practical Accuracy</li>
<li>Trust by Privacy</li>
<li>Data Risk Awareness</li>
</ol>
<p>You can visit the Institute’s <span style="text-decoration: underline;"><a href="https://ethical.institute/principles.html">site</a></span> for an explanation of each of these principles, but they seem to me not to go far enough. As a former engineer, I see these as heavily oriented toward engineering best practices and they are useful as such, but they are not sufficiently descriptive of human values.</p>
<h2>Instructive examples</h2>
<p>I appreciate the injunction toward bias evaluation, which is intended to interrogate whether our machine intelligences have learned human biases that we might consider detrimental, discriminatory, or even illegal. I have worked at companies that have uncovered decision-making by their machines which adopted racial, gender, and other biases due to being trained on data sets from human beings who have those biases. For a bank, an algorithm that denies credit or loan requests based on race is breaking the law. But one that denies loan requests based on zip code is less likely to be questioned. The complexities are many. Especially for a field this new, detection, enforcement, and accountability paths are not clear.</p>
<p>Additional debates arise when we ask ourselves whether a content recommendation engine that adopts a sexist or racist person’s preferences in their video or audio content is reasonably seeking greater efficacy by doing so, or if the recommendation engine is actually responsible for reinforcing damaging human biases by failing to diversify the recommendations. I have led task forces at some of the world’s biggest companies that examined exactly these issues, finding no easy answers but agreeing that useful and healthy debates are progress toward better solutions.</p>
<p>For another perspective, we can look at the recent U.S. Air Force drone that was instructed to remove all obstacles to completing its mission of destroying the enemy target. When the drone decided during a simulation that the operator’s instructions to stand down from its mission were preventing it from destroying the target, the drone killed the drone operator. When that behavior was forbidden, in a subsequent mission the drone destroyed the communication towers that were issuing signals to stand down from the operator. While some officials have contradicted the initial report by Colonel Tucker Hamilton in his blog post, the scenario is plausible and illustrates that such a machine intelligence could abide by most of the principles laid out by the Institute for Ethical A.I. and Machine Learning and still kill its controller.</p>
<h2>VSD thinking from academia</h2>
<p>The 2019 book by Friedman and Hendry from M.I.T. press called <span style="text-decoration: underline; font-style: italic;"><a href="https://www.amazon.com/Value-Sensitive-Design-Technology-Imagination/dp/0262039532/ref=sr_1_1?keywords=value+sensitive+design&amp;qid=1686690128&amp;sr=8-1">Value Sensitive Design</a></span> is one of the central texts in the field, and its focus exceeds A.I. and machine learning, but includes them. The book’s subtitle, <span style="font-style: italic;">Shaping Technology with Moral Imagination</span>, reveals the thesis of its contents concisely. VSD is not solely about adding value for the stakeholders and users involved, or even about ensuring access for the disabled population, or protecting human life, for instance. According to a 2006 paper by Friedman and Hendry, Value Sensitive Design attempts to center “what is important to people in their lives, with a focus on ethics and morality.”</p>
<p>Quite clearly, what one person considers ethical and moral can differ drastically from what others believe. For Elon Musk, his definition of free speech overrides some of the values important to the critics of his management of Twitter. I propose that while universal agreement on ethics and morality is impossible, it is possible to consciously define the ethical and moral foundation of any technology or product to the satisfaction of its creators.</p>
<p>Any product’s creators can then state publicly the ethics and morality behind its value-sensitive design, and that product’s stakeholders and users can decide whether to use the technology or product. If Musk’s design is led by values that advertisers and users agree with, then Twitter will flourish with those people. The new concept I seek to introduce is that publicly stating the values behind the design will lead to a better informed group of stakeholders and users, and potentially a new level of transparency akin to the mandate on food items that they display their ingredients on the package.</p>
<p>Whether a product abides by its stated ethical values can and should be debated, but a movement toward published statements of that kind would push tech companies in particular toward greater public accountability. Good design is not just value-sensitive design, but value-led design. And such designs should make their ethical and moral frameworks public, or face scrutiny by stakeholders and users.</p>
<p>Friedman and Hendry’s principles for Value Sensitive Design are stated in their M.I.T. book, and their explanation for each can be found therein.</p>
<ol>
<li>Human Welfare</li>
<li>Ownership and Property</li>
<li>Privacy</li>
<li>Freedom from Bias</li>
<li>Universal Usability</li>
<li>Trust</li>
<li>Autonomy</li>
<li>Informed Consent</li>
<li>Accountability</li>
<li>Courtesy</li>
<li>Identity</li>
<li>Calmness</li>
<li>Environmental Sustainability</li>
</ol>
<h2>VSD applied to system design</h2>
<p>I admire the boldness of these principles in their effort to go beyond engineering best practices and to tackle questions of human ethics and morality. But any person seriously concerned with climate change might consider the principle calling for calmness anathema to the urgency and anxiety that many of these activists find to be mandatory regarding the state of the environment. Finding contradictions, assumptions, and lack of clarity in these principles is not my aim, but rather it is to further a vigorous debate with these as a starting place.</p>
<p>For a dating app, the notion of a profile recommendation engine that abides by the book’s Freedom from Bias injunction might lead to a stated VSD principle such as “We believe in the rights of individuals to choose to date anyone they want on whatever criteria they choose, but our machine learning models are instructed to diversify recommendations where possible.” This raises the question of whether race, gender, sexual preference and other criteria should be made available to machine intelligences that recommend content, dating matches, or anything else. We have seen that machines can learn human biases without awareness of those criteria, using proxies for decision-making based on race, for example. This occurs when models classify users into segments based on tendencies and preferences for which no explicit data category might exist, which can lead nonetheless to the machine intelligence recommending only racially homogenous profiles or TV shows to racially homogenous groups without awareness of the racial (or gender, or other) identifications involved.</p>
<p>All debates and public statements of this kind require careful consideration and agreement among the leaders and stakeholders of these product teams. But my call is for all product and engineering leaders to openly debate the principles that should apply in a value-sensitive design, and when groups of those leaders agree on them for a particular machine intelligence or consumer product, that they publish their principles for the information and welfare of their stakeholders and users. That alone would lead to a revolution in product design and AI safety.</p>
<h3>Discover more content on AI and value-centred design</h3>
<ul>
<li><a href="https://www.mindtheproduct.com/product-management-in-the-era-of-ai-and-machine-learning/" target="_blank" rel="noopener">Product management in the era of AI and Machine Learning</a></li>
<li><a href="https://www.mindtheproduct.com/build-products-that-are-better-for-society-7-principles-to-bring-in-responsible-tech-as-a-facilitator/" target="_blank" rel="noopener">Build products that are better for society: 7 principles to bring in responsible tech as a facilitator</a></li>
<li><a href="https://www.mindtheproduct.com/is-ai-another-tech-bubble/" target="_blank" rel="noopener">Is AI another tech bubble?</a></li>
</ul>

core/freeform

How Google makes AI work at Enterprise scale – Miku Jha (Director, AI/ML and Generative AI, Google Cloud)

LLM workflows for product managers: 3 key takeaways (Niloufar Salehi, Assistant Professor at UC Berkeley) – ProductTank SF

The future of product management: Insights from ProductTank San Francisco

Product lessons learned making early moves with AI in media: Lindsey Jayne (CPO, Financial Times)

A year with ChatGPT and product innovation: Navigating the AI landscape

How to keep your head about generative AI (when everyone is losing theirs) by Claire Woodcock

Value sensitive design and AI: A reconsideration

Phil Hopkins

Read more

Comments