Analytics is the foundation of growth by measuring and providing valuable insights into customer behavior, product performance, and overall business performance.
How robust and functional is your analytics and instrumentation system?
Let’s start by noting that analytics is typically not a specific (vertical) feature and you might be wondering why I’m including it here as one. Analytics is actually not an option but a necessity, and it is such a critical part of growth, and it covers so much more than just Web analytics, that I am including it here again to reinforce its importance again.
At each product and feature level if your analytics are not set up properly and effectively your growth will not be optimized.
Ecommerce analytics refers to the process of collecting, analyzing, and interpreting data related to ecommerce activities with the intention to use the interpretations to make decisions.
The level of internal expertise and power of analytics tools we use is a competitive advantage in ecommerce. It can be argued that we will not be able to find a high growth ecommerce company that does not master analytics on the tool front as well as team skills.
What to measure
As we undertake setting up, running and improving our analytics systems it’s important to understand the difference between KPIs and metrics.
Key Performance Indicators (KPIs) and metrics are measurements used to track and improve the performance of various aspects of a business. They do however differ in their purposes, scope, and importance.
KPIs are focused on measuring the success of specific strategic objectives or high-priority goals within an organization and are closely tied to the mission and vision.
KPIs are used to evaluate the most critical aspects of performance, which are essential for the success of the business or project.
Metrics are quantitative measures used to track and evaluate a wide range of operational aspects of a business, project, or process. Metrics typically roll up to specific organization KPI’s.
Metrics in general are more descriptive than actionable, meaning they provide information about performance without necessarily guiding decision-making or improvement actions.
However, some metrics can be converted into KPIs if they become critical to the organization's success and are primarily used for monitoring, tracking, and understanding performance trends over time.
Some examples of KPIs and related metrics:
-
KPI: Increase our Conversion Rate
-
Metrics: Conversion Rate, Conversion Rate by Traffic Source.
-
-
KPI: Improve our Average Order Value (AOV)
-
Metrics: Average Order Value, AOV by Product Category, AOV by Customer Segment
-
-
KPI: Grow our Revenue
-
Metrics: Total Revenue, Revenue by Product, Revenue by Geographic Region
-
Also important to consider is that KPIs can change according to organizational goals and how OKRs (Objectives and Key Results) or other methods of setting organizational targets might guide us. However, the actual metrics do not always change when KPIs or OKR change for the organization.
Another way to think about the inter-relationship of KPIs and metrics is to think of KPIs as horizontal (cross organizational goals) and metrics as vertical (specific actions or behaviors).
Platform and tools
The first tool product managers and analytics teams typically start on their analytics journey is Google Analytics (GA).
As the company grows it can outgrow GA’s capabilities and need more options and controls such as Mixpanel, Segment and Heap among others.
A key determination for selecting the best tool for your company is automation and recommendation options. For example the amount of time it takes to understand the actual interaction of users with your website directly influences how your team members spend their valuable time.
Generative AI will also become a key factor in how analytics tools process data and recommend options in what’s called “Idea Generation” in a way current tools are not capable of.
Mobile analytics
Different analytic tools may be needed for different touch points, for example for Websites, mobile apps and newer forms of interaction.
Mobile apps where a growing percentage of user interaction is moving are controlled mainly by Apple App Store and Google Play Store and each have their own analytics tools and capabilities.
In addition to click or tap-generated data, there is also user movement data which is useful in optimizing the screens. These tools are called heatmap software and are an additional tool to the analytics stack.
Experimentation and A/B testing
Another important aspect of product managers’ involvement in analytics is the experimentation and on-page testing (called A/B testing).
Product teams work closely with analytics teams and tools to plan and track experiments on multiple aspects of user interaction with their website and apps.
Some people only think of AB testing when the topic of experimentation is discussed, which is an incomplete understanding. Full scale experimentation includes multiple views into how our products are performing from top to bottom.
Even though we have digital products our optimizations should not just focus on the user-interface (UI) side of our products. That is of course important but there is a higher, product and business level of optimization which is important.
For example if we are looking at providing an affiliate program, the top level experiments assessing the value and performance of the program are different from tool or UI related performance measures such as acquisition, engagement and conversion on different steps of the customer journey.
There are popular and free A/B testing tools including Google Optimize, Optimizely and Amplitude as well as business experimentation tools available on the market.
A key aspect of experimentation tools is how well it plays with your other tools which is determined by the ease and width of API integrations. Graphic visualization, automated notification and team collaboration capabilities also play a role in how easy the tool can deliver value to the organization.
Experiment design
In order to have a robust experimentation system we need to design experiments that deliver the data and results we need to make correct decisions.
Designing and running AB tests as well as business and product level experiments is a key skill product managers need to master. This mastery comes from education and experience in running experiments and learning from them on a regular basis.
The performance of experimentation programs can be measured by how closely experiments designed and run by a team actually deliver the optimizations and business decisions that positively contribute to business outcomes.