- Ambitious Mission Statements, Misaligned Execution
- How I Created the Impact Mindset
- Overuse of Usage Metrics as Success Criteria
- How Grand Ambitions Became Usage Obsessions
- The Pitfalls of Solely Prioritizing Usage
- Measurements of Success Beyond Usage and Usability
- Comprehensive Approach to Metrics for Your Team
- Chapter Recap
This chapter is from the book
This chapter is from the book
This chapter is from the book
The Pitfalls of Solely Prioritizing Usage
Measuring usage is not a harmful practice entirely; only when it is the sole metric that companies focus on does it lead them astray. Engagement is paramount to building successful products; many life-changing prototypes are collecting dust on academic and R&D (research and development) lab bookshelves. Usage metrics, such as daily active users, duration, and retention, are valuable leading indicators for building an easy and enjoyable interface. The missed opportunity to build long-term viable solutions emerges when the pursuit of expanding user numbers overshadows all other objectives, creating an environment in which problems arise without explanation.
The media sector confronted this reality in the early 2020s as audiences began disengaging.3 While part of this shift is attributed to the allure of newer, more captivating platforms like TikTok, the saturation of low-quality content and listicles (for example, “20 pictures you must see before you die”) had also peaked. As detailed in the book Traffic, by Ben Smith, for years, media giants, notably Buzzfeed, had shifted their content creation toward that which would become viral at as low a cost as possible, yielding engagement for cheap. Although each article might momentarily boost engagement, it rarely conferred lasting value to the reader.
These corporations lost sight of the fundamental reasons people sought content: information, understanding, and entertainment. Instead, these low-cost content pieces were devoid of depth or substance, offering only something to look at and maybe share with one’s social media community. As the cycle continued, user after user became uninterested in the piece, no longer choosing to go toward a solution that was not satisfying their desire. The gradual decline accelerated, and by 2023, once-dominant media entities like Vice and Vox faced bankruptcy, while Buzzfeed languished, a mere shadow of its former glory.
The media industry was among the first to see the pitfalls of an unwavering focus on user acquisition. When an organization’s focus is trained purely on adding more users, the outcomes that their users face become irrelevant. Growth comes at the cost of usability and use-case fulfillment. On a granular level, this might manifest as users abandoning a platform or hesitating to upgrade. On a broader scale, it distances companies from their visionary mission statements and, in more severe cases, contributes to societal challenges such as mental health issues, as touched on with social media, along with the reduction of privacy and increased social polarization.
These adverse outcomes caused by neglecting user-focused outcome metrics might take time. However, a pattern emerges over time: users are initially enticed, remain for a while, but eventually depart after realizing the platform doesn’t meet their needs. In product jargon, this is called a leaky bucket. Even products that inherently create switching costs for users, like social media with its vast network effect caused by a user’s connections being on the same platform, aren’t immune. As users discern the misalignment between their needs and the platform’s offerings, even these giants face attrition.
The stakes for products in competitive markets are even higher without such built-in retention mechanisms. Without solving problems for users, users can easily switch to another solution as soon as a competitor launches a better solution into the market. Only when products are doing something for the user can a switching cost begin to form where customers will stay because they don’t want to risk missing out on something valuable to them. And research shows that a 5 percent improvement in retention leads to an increase of at least 25 percent in profits.
Beyond retention issues, teams can commonly attribute operational challenges to an incomplete picture of product success caused by missing user outcome measurements. Groups operating in the dark about the real-world impact of their features are likely to grapple with product planning ambiguities. They must judge a feature’s success based on engagement alone and make improvement and depreciation decisions with benchmarks that share only what happened rather than why. Decisions become rooted in intuition rather than empirical evidence. Looking at their metrics, such as daily active users and NPS, might highlight a problem. However, without a thorough understanding, correlations remain elusive, and teams are once again left chasing a number that doesn’t matter to end users.
For product teams grappling with dwindling retention or significant challenges in planning, the initial inquiry should center on the product’s tangible value to users. This approach transcends mere user engagement or usability. It delves into whether a solution transformed a user’s behavior or perspectives through interaction. A product’s success is genuinely affirmed only when tangible, real-world outcomes shift. Otherwise, it’s hardly surprising when users swiftly transition to alternative solutions, lured by promises of faster, more affordable, or more innovative benefits.
Building new metrics requires time and effort, which most product people find scarce; it also requires the functional knowledge of creating new variables. By reading this book you are taking a step closer to expanding your metric portfolio. This chapter makes the case for adopting new metrics to equip you to sell the vision.