A simple structure to think about any metrics for any product.
I talked about the principles of creating product metrics in my earlier blog. Typically, you should answer:
- What do you want to measure?
- Why do you want to measure?
- What will you do next after measuring?
- What are the consequences of the metric?
Choosing the right product metrics can be challenging but also rewarding.
Many product managers struggle to identify the right success metric for their product. Most folks rely on google to tell the most used metrics and apply them for their product. Google may know some commonly used metrics, but it does not know your product, so be careful there.
In this blog post, I will share a framework for product managers to help identify the right product metrics for their product, without having the need to search the internet.
TL/DR
If you are like me and prefer watching a video over reading a blog, then you can view this video. Just a note, the video and blogs may contain different content since both are produced at different times.
This blog is part of a 4 part series on product metrics.
Blog 1: Principles of product metrics
Blog 2: Thinking about lead indicators instead of lag indicators
Blog 3: Structure for product metrics
Blog 4: Graphs for product metrics
There is a simple 6 point approach to capture your product metric. 90% of the metrics that you wish to capture for your product will fit into this 6 point approach.
So here it goes. Just identify how you will measure the:
- Breadth of product usage.
- Depth of product usage.
- Frequency of product usage.
- Usability of the session.
- Quality of the session.
- Correlations
Looks confusing? Don’t worry. By the end of the blog, I hope this structure becomes the only way of thinking when it comes to metrics.
Let’s assume you are a product manager of an e-commerce website to define some of these metrics.
Breadth of product usage.
Your first metric should depict how do you measure the breadth of product usage. Breadth metrics help understand the top of the funnel.
In the context of e-commerce, breadth would mean many things.
If I take amazon.com as an example, then breadth would mean:
- # of users that landed on home page.
- # of users that performed search.
- # of searches performed.
- Which browser did the user land from?
- Mobile or desktop count from where users landed.
- User demography.
- Where did the user land on the home page, i.e. from email, search, direct url, partner websites, referral links etc.
- Monthly active users or daily active users.
Here the definition of active is important. What do you mean by active? Users who land on amazon.com 10 times a month. Users who make at least 1 purchase a month.
Depth of product usage.
This metric indicates user’s depth of engagement with your product.
Depth helps understand the level of interaction that users can do with your product.
In the context of e-commerce portal, depth would mean the entire purchase funnel.
- How many users landed on the portal?
- Of that number how many users clicked on the item details page?
- Of that number how many users added items to the cart?
- Of that number how many users made the purchase?
As a product manager your challenge should be to reduce the funnel size so that users can get to closure as early as possible.
I recommend you to see the end to end funnel while buying this awesome book on metrics on Amazon. Click on the link below and explore the breadth and depth of Amazon purchase funnel.
Frequency of product usage.
Frequency of product usage refers to how often the product is accessed. This is also referred to as session-based metrics.
Frequency in the context of e-commerce portal would mean what all did a user do when he or she landed on the portal?
- A user purchased a pair of jeans by searching for them on the portal.
- In the same session, the user purchased perfume by clicking on the recommended items.
- The user also added a watch to the wish list.
In some other session a user simply clicked on Buy now and completed the purchase. The two sessions are totally different.
Sessions add a component of session analysis to the depth metrics. The following are common metrics that are measured for ecommerce portals:
- Session metrics by device type
- Session metrics by traffic source
- Session metrics by location
Usability of the session.
Now, imagine you have breadth, depth, and session data. One element that is completely missing in our metrics is the element of time.
The time component answers how much time the user takes to accomplish a task. All your metrics that say amount of time taken to … are all having a time component to their metrics.
In the context of e-commerce portals, any metric that helps understand the cycle time of a process fall in this category.
For example, why do you think Amazon introduced the BUY NOW button? Explore it for yourself.
Click on the above link and try to make the purchase by clicking on Add to Cart.
Now try to make the same purchase by clicking on Buy Now. You will understand the whole essence behind reducing cycle time.
Quality of the session
So now we have 4 components in our telemetry.
- Breadth of usage.
- Depth of usage.
- Session information for breadth and depth.
- Time taken for each session.
All of this is great however it still does not tell the quality of your breadth, depth, session, or time-based usage of the product.
What if I took 20 minutes to make a purchase but I enjoyed it since Amazon helped me find and compare products and I ended up landing a great deal.
What if I took 30 seconds to make a purchase, but I saw purchase errors on my credit card. That would make me never return to the platform.
Here we are talking about getting the quality of each session by the user on the product.
This is often measured in many products using net promoter score, customer dissatisfaction score, customer satisfaction score etc.
Correlation
Correlation is very simply. Now that you have all these metrics, how do you correlate them to derive something meaningful. For example,
- If you have # of purchases and # of sessions, then you can find Sales Conversion Rate = (# of purchases / # of sessions)*100.
- If you have Total products purchased and Total price, you can find Total revenue = Price * Product purchased.
- If you have Total Revenue and Total number of orders, then you can find Average order value = Total Revenue / Total number of orders
- Customer Lifetime Value = Average Value of a Purchase x # of Times the Customer Will Buy Each Year x Average Length of the Customer Relationship (in Years)
- Customer Acquisition Cost = Amount Spent on Marketing / # of New Customers
- Shopping Cart Abandonment Rate = (# of Completed Purchases / # of Shopping Carts Created) x 100
- Returning Customer Rate = (# of Return Customers / Total # of Customers) x 100
- Click Through Rate = (# of Clicks / # of Views/Impressions) x 100
Reflections
To summarize, what this blog explains is that when you think metrics, think of these 5 core elements:
- Did you capture the depth of product usage?
- Did you capture the breadth of product usage?
- Do you have session data for breadth and depth?
- Do you have a time component associated with your metrics?
- Finally, do you have qualitative data to sum up the whole thing?
- Now with all these metrics in place, how do you correlate them all to find meaningful metrics.
In the video, I also talk about financial metrics etc. but that was making this blog too long so I skipped it :)
Congratulations if you are reading this line. Stats show that only 22% of the readers will read till the end of this blog. Which means you are an outlier. And outliers make it big in life. So congrats.
If you’ve found value in my content and would like to support its continuation, I invite you to consider making a contribution at Patreon.
Your generosity enables me to keep creating and sharing meaningful content with you.
You can follow me on youtube or medium for regular updates on my own journeys of product management, business management, and leadership.
