Expert Opinion: AI & CRO | TAG Webinar Roundup

Key takeaways from the TAG webinar

AI compresses the work around a test, not the test itself. Alon was clear that AI doesn't replace what TAG does - it accelerates specific stages. Research that used to take a strategist two days now takes half an hour to an hour; the same goes for analysis, reporting and early-stage design. But the test itself still needs real time and real traffic to reach a reliable result, no amount of AI speed changes that.

Statistical significance is still a traffic and order-volume problem, and AI can't shortcut it. Alon put a real number on it: generally around 3,000 orders a month is needed to run structured A/B testing and reach statistical significance. Below that, TAG recommends larger, more deliberate changes tested over longer periods (period-over-period comparison) rather than smaller iterative tests that would never gather enough data to mean anything.

Synthetic shoppers can point at a problem, but they can't prove a fix works. TAG has trialled AI-driven synthetic shopper tools and found them useful for catching obvious issues - an unclear layout, a confusing hierarchy - but generic and averaged out by design. Justin was direct about the risk: these tools need to be validated against real customer data before being trusted, because synthetic shoppers lack the context of what's actually changing in a market and can't predict genuinely unpredictable human behaviour.

A segment-level win can be invisible in a site-wide average. One personalisation test, showing returning visitors the products they'd already viewed, lifted conversion by 28% for that specific returning-customer segment. Justin's point wasn't just the result itself; it was that this kind of win would never surface in a topline, all-traffic conversion number. Measuring only the site average risks missing exactly the kind of result AI-assisted personalisation is best placed to find.

Personalisation has a line, and crossing it erodes trust rather than building it. Alon's gut check for the team: what would a good shop assistant reasonably say to a returning customer, versus what would feel invasive? "Welcome back, still thinking about that item?" is useful. Referencing exact demographic guesses or how many times someone viewed a product in a week reads as surveillance, not service - and TAG treats that instinct as a genuine test filter, not just a nice-to-have.

Expert quote

"AI definitely doesn't replace what we do, and it doesn't do the work. What it does do is help us accelerate - specifically around research, design and build. But your test still needs time, it still needs real traffic to work." Alon Abraham, Co-founder & Head of Strategy — TAG

Testing a 'sure win' on the product page

TAG shared a test that illustrates why following best practice isn't the same as testing it: moving a USP block (shipping, returns, brand trust signals) up the page, directly below the add-to-cart button, in place of a product recommendation carousel. Every instinct on the call said this should win - clearer, more prominent trust signals right where the customer is deciding to buy. It lost.

Rather than treating that as a dead end, Justin's read was that the result raises more useful questions than it answers: did moving the recommendation carousel down reduce product discovery? Were the specific USPs shown - like a shipping threshold - actually working against the brand once customers noticed them clearly for the first time? The test didn't hand them an answer, but it ruled out an assumption the team would otherwise have kept making - and did so without any lasting cost, since a losing test can simply be reversed.

What this means for eCommerce teams

Use AI to speed up the unglamorous parts of a CRO programme - synthesising reviews and support tickets, drafting analysis, generating a wider pool of test hypotheses - rather than treating it as a way to skip testing altogether.

Check whether you actually have enough order volume to run structured A/B tests. If you're well under roughly 3,000 orders a month, larger, less frequent changes tested period-over-period will tell you more than a string of small, inconclusive A/B tests.

If you use synthetic shopper tools, validate their output against real customer data before acting on it, treat them as a way to catch obvious problems quickly, not as a substitute for testing with real traffic.

Look at your test results by segment, not just as a site-wide average. A meaningful win for returning customers, or for one traffic source, can be completely invisible in a topline number.

Before shipping a personalisation feature, ask whether it reads as helpful or as invasive. If it would feel strange coming from a human shop assistant, it will feel stranger coming from your website.

TAG's perspective: AI accelerates the process, evidence still decides

What set TAG's session apart wasn't excitement about AI's potential - if anything, their tone was the most measured of the series. Alon and Justin's throughline was that AI has genuinely changed the speed of CRO work without changing anything about what makes a test result trustworthy: real customers, real traffic, and enough volume to reach statistical significance. The clearest evidence of this discipline was the fashion brand example Alon described - a full site redesign built by "best practice," which tanked conversion because it was based on what a good website is supposed to look like rather than on what that brand's actual customers respond to. TAG's answer to that failure wasn't more AI - it was more testing. That's the distinctive position running through the whole session: AI has widened what a small CRO team can attempt, but it hasn't changed what counts as proof.

About TAG

TAG is a digital growth agency running structured CRO and experimentation programmes for eCommerce and digital brands. Their 50-person Melbourne and Sydney team spans CRO, UX/UI, web development, CRM and personalisation, with every change tested and results measured to statistical significance. Speakers: Alon Abraham, Co-founder & Head of Strategy, and Justin Sommerville, CRO Specialist.

Expert Opinion: AI & CRO | TAG Webinar Roundup

Shopping shouldn’t feel like a chore

Related Posts

Level Up Your eCommerce Game With Targeted Email Marketing

Expert Opinion: AI & CRO | Brancoy Webinar Roundup

The Psychology Behind E-Commerce Product Recommendations