How To Measure Product Recommendation Performance On Shopify Store?

Ellie Ho
Jun 7, 2024
3 min read

Product recommendations and product bundles are crucial elements for eCommerce upselling and cross-selling. Research shows that website visitors with an appropriately implemented recommender system can spend five times higher than usual. To fully leverage the potential of product recommendations, online merchants need first to measure their performance and optimize them with data-driven decisions.

This blog post will help you decide whether your recommender system works well from a business angle.

Why Do You Need To Evaluate Your Product Recommendations?

A recommendation engine is nothing new in the eCommerce world. Since Amazon applied it and became the leading marketplace, online stores have been reverse-engineering Amazon's strategies to gain equivalent success. However, businesses are not created the same. There is no such thing as a one-size-fits-all strategy for a successful product recommendation system.

Moreover, there is an extensive range of recommendation types you can choose to display. From statistic-based models like Newest Arrivals, Trending Products, Most Viewed, and Bestsellers to personalized models like Recently Viewed. Boost AI Search & Discovery also allows Shopify merchants to enable AI-powered and rule-based recommendations for Frequently Bought Together and Related Items (including Alternative and Complementary Products). You can also manually select the recommended items for a target product using the Manual model of Boost.

Try Free Now

When you spend time and effort analyzing the performance of recommendation widgets or recommended products, you will know which works best for your customers and amplify the effect. The evaluation also brings problems or issues with your product recommendation to light so you can make necessary changes.

Furthermore, the advancements in artificial intelligence and machine learning (AI/ML) technology and its application in eCommerce have pushed the price for AI/ML product recommendations. Higher investment is expected to generate a better return. Therefore, merchants should measure the quantitative benefits of the recommendation engine to ensure that no resources are wasted.

ecommerce product recommendation analytics

Which Metrics Should You Keep An Eye On?

When it comes to measuring the performance of product recommendations, there are quite many angles to look at. For example, an engineer will focus on the accuracy and relevancy of recommendation models. In this article, we will stand on a business perspective. After all, what merchants want to know the most is, do product recommendations lift revenue and increase conversions, right?

Below are some useful metrics to help you, as a store owner, gain insight into how customers are engaging with their recommendations and how they are impacting their conversion rates.

Click-Through Rate (CTR)

The click-through rate is the percentage of users who click on a recommended product after viewing it. This metric is a good indicator of how engaging the product recommendations are. A low click-through rate may indicate that the recommendations need to be more relevant to the user or that they need to be presented in an appealing way. On the other hand, a high click-through rate shows that the recommendations are resonating with the audience.

Conversion Rate

The conversion rate measures the percentage of users who make a purchase after viewing a recommended product. This is a crucial metric as it directly impacts revenue. If product recommendations are not leading to conversions, they do not fulfill their purpose. By tracking the conversion rate, merchants can identify the most effective recommendation types and adjust their strategy accordingly.

Revenue per Visitor (RPV)

The revenue per visitor metric measures the average revenue generated per visitor interacting with a recommended product. This metric is especially useful if merchants have a high website traffic volume. By tracking the RPV, merchants can identify which recommendations generate the most revenue and adjust their strategy accordingly.

Average Order Value (AOV)

The average order value measures the average value of orders that include a recommended product. By tracking the AOV, merchants can identify which recommendations are leading to more significant purchases and adjust their strategy accordingly.

By tracking these metrics, merchants can gain insight into how their product recommendations are performing and identify areas where they can make improvements.

Besides these abovementioned metrics, the best product recommendation tools often give you more. For example, the Recommendation analytics of Boost AI Search & Discovery shares the percentage of recommended items in cart or in order. This shows how much, on average, recommended products match the customers' buying intent.

Measuring Product Recommendation Effectiveness Then What?

After you have the data on your hands, you can identify which recommendation widgets on which pages are performing well or poorly. But don't stop that. You can take a closer look to uncover more behavioral patterns of your customers and have more ideas to optimize not only the product recommendation but also other elements of online product discovery.

using recommendation analytics to analyze customer behavior

Analyzing Customer Behavior

Product recommendation analytics can help reveal customer behavior. By tracking how customers interact with your widgets (how many of them view or click recommended products) and which products they ultimately purchase, you can gain insight into which items are the most favorable.

A heat mapping software like Hotjar is helpful for tracking where customers are clicking on your website. Recommendation analytics in the Boost app also offer Click count and Click rate. Another approach is tracking which products customers add to their cart after interacting with a recommendation. By analyzing this data, you can identify which recommendations are generating the most clicks or conversions and which are being ignored.

Identifying Opportunities for Growth

By measuring the performance of product recommendations, merchants can identify opportunities for growth and expansion. For example, if a specific product recommendation type generates high click-through and conversion rates, consider placing it on more web pages. On the other hand, if a particular product category is not performing well, merchants may want to reevaluate their strategy and consider removing those products from their recommendations.

A/B Testing

Experiments are necessary to verify the insights you get from product recommendation analysis. A/B testing involves creating two versions of your widget and randomly showing one version to a group of users and the other version to another group. By comparing the performance of the two versions, you are confirmed which version is more effective.

ab testing with product recommendation widget

When conducting A/B testing, it is essential only to test one variable at a time. For example, test the placement of the widget on the page or the wording of the recommendation. By only testing one variable at a time, you can ensure that any changes in performance can be attributed to that one variable.

Improving the Customer Experience

Measuring the performance of product recommendations is about more than just increasing sales. It is also about improving the overall shopping experience for customers. By presenting personalized recommendations relevant to the customer's interests, merchants can enhance their experience and make them feel valued.

Moreover, by analyzing customer behavior, merchants can gain insight into which recommendations are most effective and adjust their strategy accordingly. This can lead to increased customer engagement and brand loyalty, which can ultimately drive long-term success for the business.

Use Boost To Recommend Products And Measure Product Recommendation Performance

Our app offers eight product recommendation types: statistic-based, personalized, rule-based, manual, and AI-driven models. We don't limit the number of widgets you can create and place on a web page, so Shopify merchants can easily carry out testing with many product recommendation types.

Boost's Recommendation Analytics is an insightful dashboard with four main reports:

  • Overview: showing most essential metrics like Total Revenue, Conversion Rate, and Click Rate
  • Page performance: evaluating the performance of web pages that have a product recommendation widget
  • Widget performance: evaluating the performance of product recommendation widgets in particular
  • Product performance: evaluating the performance of recommended items in particular
recommendation analytics boost ai search and discovery

Besides, Boost users can investigate the App Impact dashboard to see Order count, AOV, and Revenue per visitor broken down for the Recommendation feature.

Interesting to know:


Boost AI Search & Discovery offers all merchants a 14-day free trial and a 30-day money-back guarantee. So don't wait and install the app now!

boost ai search and discovery banner
Ellie Ho
Content Marketing Specialist
June 17, 2024
6 min read