Platforms Are Experimenting on Their Users… A Lot. Is That Okay?
Read time: 2 min
Core Idea
Untransparent testing on customers of platforms can erode customer trust and hinder income opportunities; establishing informed consent and results-sharing is vital to fostering a more ethical and beneficial experimental culture.
Top Thoughts
A/B testing is the practice of experimenting with different versions of a product or feature to evaluate customer preferences and outcomes.
Widespread testing of algorithms and features across platforms creates questions about the impact on customer livelihoods; e.g., LinkedIn's undisclosed experimentation with its networking feature affected 20 million customers and raised concerns about transparency and impact.
The consequences of untransparent experimentation include customer trust erosion, leading to doubts about platform promises and intentions, loss of income potential, loss of networking potential, and a negative impact on emotions and overall well-being.
Ways to ensure ethical experimentation include alerting customers to testing and receiving informed consent, sharing the results of testing and allowing external, third-party regulation to ensure ethical, more beneficial experimentation.
Delays resulting from experiment regulation may cause “beneficial friction” by slowing the process in the short term but bringing long-term benefits such as increased platform trust and engagement.
Implementing better practices can build long-lasting relationships with customers that encourage the customer loyalty needed to avoid competition and threats to business models like mandatory regulations.
Quick Quotes
"There’s nothing inherently wrong with experiments or algorithms but how they are used, how they are implemented, and the professional implications for people subject to them is where my work raises interest."
"The role for us… is to try to push towards more mutually beneficial outcomes, rather than eliminate the entire platform or eliminate experimentation, which is very useful."
"There isn’t systematic change that I’ve observed yet. But there is a recognition that you need to kind of create more long-lasting relationships with people on the platform... hopefully those relationships sustain themselves, when there are threats to their business model."
“The rules and criteria for success are now embedded in opaque algorithms… The rules for success, i.e., the ways to actually make money, or network, are a mystery to the users. It’s very frustrating.”
“Introducing this friction of informed consent and debriefing now, in the long term, beneficial outcomes are more trust of people in organizations.”
Action Steps
Prioritize customer trust and engagement - Customer trust is essential for long-term success. Prioritize transparency and ethics in experimentation to ensure customers feel valued, supported, and empowered on your platform.
Implement informed consent - Communicate clearly to users that your business conducts experiments, outlining potential changes and impacts. Make this information visible and easily accessible to foster transparency and user understanding.
Share experiment results transparently - After conducting experiments, share the results with customers in a clear and understandable manner. This helps then appreciate the rationale behind changes and builds trust in the platform's intentions.
Source:
Rahman, H., & Pavin, L. (2023, Aug 14). Platforms Are Experimenting on Their Users…a Lot. Is That Okay? [Web article, Podcast]. Kellogg School of Management. (Link)
