Analyzing product engagement: segmentation, activation, retention, loops, and choosing the right metric
Product engagement is a measure of the depth of activity within a specific time period for a given product. It is a key metric for growth, as it impacts retention, acquisition, monetization, and defensibility.
Product engagement measures of how users interact with your product. Standard product engagement metrics include depth, breadth, and frequency of interaction over time, and understanding this behavior allows you to maximize your application usage.
Measuring product engagement is crucial to helping companies understand how well they engage customers and improve their products to keep customers more engaged.
You can use product analytics to measure how well you are engaging your users, and then use that data to inform your growth strategy.
How to Analyze Engagement
Product engagement is a measure of the depth of activity within a specific time period for a given product. It is a key metric for growth, as it impacts retention, acquisition, monetization, and defensibility.
There are three levels of engagement that can be measured: total engagement, engagement per active, and engagement state segmentation. After diagnosing engagement and selecting a high-level strategy, the next step is to align the metrics and measure engagement.
To do this, segments of users can be placed into different engagement buckets, and the average engagement per active user can be calculated. Finally, by understanding the absolute, percentage, and percentage difference from the average engagement for each segment, actionable insights can be gleaned about how to improve product engagement.
Cohorts & Segmentation
Cohorts can be used to understand how total engagement (measured by total features used) trend over time. Segmentation can be used to understand how different engagement states (measured by power, core, and casual users) trend over time.
You should perform cohort analysis on active users and retention to understand how product engagement evolves over time. You should perform segment analysis on power users, casual users, and core feature usage to help identify trends in user behavior over time. You should compare data across different attributes (i.e., product categories or segments) to identify if certain attributes are performing better than others. Lastly, you should regularly create action plans based off of the analyses you perform.
Starting with Activation
Activation is not just about onboarding. The aha moment is a key part of activation, and it’s important to determine what it is for your product. It can help you connect your higher-level goals to actions that users take within your product.
Activation is the process of getting a user to the point where they are using your product regularly. This means they've taken the action a repeated number of times within a certain time period. It's important for engagement because it's the first step in getting a user to stick around and use your product. If they don't activate, they're not going to stick around.
Habit & Aha Moments
The habit moment is the point at which a user has established a habit around the core value proposition. The aha moment is the point at which the user has experienced the core value prop for the first time. It can help you connect your higher-level goals to actions that users take within your product. The first value exchange most often happens with the first transaction in an activation stage. For two-sided marketplaces, it might be different for buyers and sellers.
The habit moment is important because it's indicative of long-term retention.
The aha moment is important because it's the point at which users first experience the core value prop. To measure this, we look at the first number of times the user does the action within an initial time period. This metric should highly correlate with the habit moment and long term retention metric.
How Do You Define Aha Moments?
The first step is to make sure you understand your personas and use cases. Then, you want to make sure you understand the natural frequency of how often they would take the desired action. After that, you want to make sure you have a retention metric in place. And finally, you want to define your habit moment and habit metric.
Discovering Your Engagement Game
Amplitude describes product engagement models as the game your business plays and how it keeps score. There are three engagement games that all digital products play: the attention game, the transaction game, and the productivity game.
Products playing the attention game optimize for the amount of time users spend in-product, while companies playing the transaction game help customers confidently make purchase decisions more frequently.
Predominantly in B2B SaaS, the productivity game aims to create an easier, faster, and more reliable way for the user to complete tasks or workflows.
It is essential to understand which kind of game you are playing – what are the rules and how will you keep score. Products playing any of these games need to define the right metrics of success for their game in order to measure their success.
Defining The Right North Star Metric
The North Star Metric is a single metric that captures your product strategy and its impact on customers and business results. To define your North Star Metric, you need to understand what factors influence it, and these factors can be aligned with your product vision to create a metric that's meaningful to your company.
The North Star Framework is a business strategy framework that helps organizations achieve their three critical goals: 1) increase revenue, 2) increase profits, and 3) increase customer satisfaction. The framework has a simple structure and is easy to use.
In any product-led company, the Product North Star serves three important functions. It provides your organization with clarity and alignment on what it is striving for and what may be sacrificed. Second, it showcases the product team's impact and progress to the rest of the company, resulting in greater support for strategic product initiatives. And lastly, but most importantly, it creates accountability to an outcome.
If your North Star is based on a metric that doesn't represent how customers use your product to get value, you run the risk of optimizing for the wrong measure and, as a consequence, achieving the wrong outcome.
If your company and team assumes you are playing a different game, you will design and ship features that move metrics, but are not ultimately the right ones to build a sustainable and growing business.
How do you measure Engagement?
There are a variety of ways to measure engagement, but the three most common are frequency, intensity, and feature spread. Frequency is the number of times an activity is done in a given time period. Intensity is the amount of time spent on an activity in a given time period. Feature spread is the number of features used in a given time period.
Total engagement is a high-level view of total engagement and measures the total engagement within a specific time period for your entire active user base. To measure total engagement, you need to track:
- Total features used
- Frequency
- Intensity
Engagement intensity is the amount of time spent on a product in a given time period. It can be measured in different ways, such as total time spent, average time spent, or median time spent.
How can you analyze Engagement?
A team can use engagement analytics to measure how effectively it is deepening engagement with its customers or users. This allows a team to track whether it is successfully moving people along the engagement spectrum, from initial engagement through to deeper levels of engagement, in order to create more value for the business.
The distribution of current engagement states, how engagement buckets trend over time, and how many users are transitioning between states are all important metrics to track in order to measure the success of an engagement strategy. By understanding these metrics, a company can determine whether they are making progress in increasing or improving engagement.
What Are Engagement Loops?
The Engagement Loop is an interaction framework that helps you understand how customers experience your product and how you can create value for them early on.
Engagement loops are a key part of an engagement strategy. They are the experiences that we put in front of our users to keep them coming back to our product.
There are three types of engagement loops: core, supplemental, and moderate. You want to think about how to increase the number of supplemental and moderate loops, while also optimizing the core loop.
A core loop is the central experience that a user has with our product. It's the most important and the most frequent.
A supplemental loop is an experience that augments or enhances the core loop. It's not as important as the core loop, but it's more important than a moderate loop.
A moderate loop is an experience that's less important than the core and supplemental loops. It's less frequent and less impactful.
How do you Activate a New Use Case?
There are five different steps that we move through when activating someone on a new use case or new feature: signal, real estate, message, activation flow, and destination.
First, we need to identify a signal in the data that an individual user is a good candidate for a new feature or a new use case.
Next, we need to figure out what the potential real estate is to pull them in and engage them in a flow to transition them to another feature or use case.
After that, we need to come up with a message or hook that gets them to engage.
Then, we need to create an activation flow that will successfully lead back to them associating that trigger with the product again.
Lastly, we need to consider frequency and intensity as meaningful engagement strategies that must be considered contextually.
Identifying Engagement Opportunities
Start by understanding the different mindsets among users in different engagement states. Understand the pathways that are necessary for users to transition between mindsets. Then identify data signals that indicate which users are in which mindset. Last, understand the business value of the opportunity for each pool of users in each mindset.
Transitioning users to a new state will vary depending on the product, user, and business. However, there are a few key things to keep in mind:
- Understating the mindsets among the different engagement states is key to identifying the right opportunity.
- Pathways are the steps a user needs to take to make the transition between states.
- Data signals help us understand how many users are in each mindset and the business value of the opportunity.
- Value is the size of the opportunity among the group of users.
Measuring Engagement for B2C and B2B Apps
Common ways to measure engagement include measuring how often users open the app, how much time they spend using it, how many tasks they complete, or how frequently they interact with it.
DAU is the number of daily active users. WAU is the number of weekly active users. MAU is the number of monthly active users. DAU, WAU, and MAU are all important for measuring engagement because they give us a snapshot of how many people are currently active, how many people were active in the past, and how many people could potentially be active in the future.
DAU:MAU stands for Daily Active Users:Monthly Active Users. It's a calculation of how many active users you have on a given day, compared to the number of active users you have over the course of a month. This calculation is important because it gives you a sense of how engaged your active users are. A high DAU:MAU ratio means that your users are more engaged, while a low DAU:MAU ratio means that your users are less engaged.
The L28 histogram is a graph that shows the distribution of a user's engagement over the last 28 days. This graph is important because it helps to understand the distribution of a user's engagement. This information can be used to help understand how to improve a user's engagement.