How to A/B Test Catalogue Ad Creative on Meta

Branded catalogue ad designs are easy to like and hard to prove. You see your products with your logo and price, and they look better than the plain photos. Whether they sell better is a separate question, and the only way to answer it is a fair test.

This post covers how to set one up on Meta, what to measure, and the mistakes that make a creative test meaningless.

What you’re actually testing

There are two common questions:

Designed images vs plain product photos. Does adding a branded design to catalogue images improve results at all?
Design A vs design B. Which of two designs does better, for example product-led against price-led?

Answer the first before the second. If designed images don’t beat plain photos for your products and audience, choosing between designs is the wrong question.

The catalogue problem

In a normal ad test you’d make two ads with different images. Catalogue ads don’t work like that: each product has one main image in the catalogue, and the ad shows whatever the catalogue holds. You can’t put two different main images on the same product in the same catalogue.

The cleanest way round it is two catalogues:

Catalogue A with one set of images, for example your design.
Catalogue B with the same products and the same IDs, with the other set, for example the original photos.

Everything else stays identical: products, titles, prices, availability. Make sure your pixel and Conversions API events are connected to both catalogues, so both can match events to products.

Setting up a fair split test

Meta’s split testing divides the audience into groups that don’t overlap, so the same person doesn’t see both versions. Its developer guidance is clear on how to run one:

Define your KPIs first. Decide what you’re measuring before the test starts.
Decide your confidence level in advance. Meta notes that tests with larger reach, longer schedules or higher budgets tend to give more statistically significant results.
Change one variable only. If two ad sets differ in creative and placements, you can’t tell which made the difference.
Keep the groups comparable, especially when you’re comparing volumes like number of conversions.

In practice, for a catalogue creative test:

Create two ad sets, or two ads, that are identical apart from the catalogue they use.
Same audience, same placements, same optimisation, same budget split.
Use the same product set definition in each catalogue, for example all products, or the same best sellers.
Run them as a split test, so Meta divides the audience between them.

What to measure

Pick one primary metric before you start, and stick to it:

Cost per purchase, if your goal is volume at a target cost.
Return on ad spend, if order values vary a lot between products.

Watch these as context, not as the verdict:

Click-through rate. A design can attract more clicks and still sell less, for example if it promises a saving the product page doesn’t make obvious.
Add-to-basket rate. Useful for spotting where in the journey a design helps or hurts.

How long to run it

At least a full week, so weekdays and weekends are both included.
Until you reach the number of conversions you decided on in advance. A result from a handful of purchases is noise.
Not over a sale period, unless that’s what you’re testing. Black Friday traffic behaves differently, and a design that wins in a sale might not win in March.

Don’t stop the test the moment one side pulls ahead. Early leads often shrink.

Mistakes that ruin a test

Different products in each catalogue. If one catalogue is missing products, you’re testing the range, not the creative.
Prices out of sync. If one catalogue refreshes more often than the other, one shows stale prices.
Changing things mid-test. Editing budgets, audiences or the design itself resets the comparison.
Too many variants. Two or three clearly different designs give a clearer answer than seven similar ones.
Ignoring Meta’s own creative changes. Meta’s Advantage+ creative options can change how catalogue images appear. Keep these settings the same in both arms.

After the test

If the designed images win, keep them, and test design against design next. If they lose or there’s no clear difference, that’s useful too: put your effort into the product data and the photos themselves.

Either way, write down what you tested, when, and the result. A test you can’t remember is a test you’ll run again.

Running this with AI Shopping Feeds

In AI Shopping Feeds, each product in a feed gets one rendered image, from the feed’s main design or one of up to four extra designs chosen by rule or date, and each feed can sync to its own Meta catalogue. For a design vs plain photos test, set up two feeds from the same source: publish a design on one and leave the other with original photos, and sync each to its own catalogue. For a design vs design test, publish a different design on each.

Because both feeds come from the same source, they share products and IDs. Give them the same refresh schedule so prices stay in step too. The DPA Creative Editor lets you preview a design on real products before publishing, so you can check both versions before the test starts. Our post on DPA templates suggests designs that differ enough to be worth testing.

Frequently asked questions

How do I A/B test catalogue ad images on Meta?

Each product has one main image per catalogue, so the cleanest test uses two catalogues with the same products and IDs, one with each set of images, and a Meta split test between two ad sets or ads that are identical apart from the catalogue. Meta's split testing divides the audience so the groups don't overlap.

How long should a catalogue creative test run?

Long enough to reach the number of conversions you decided on before starting, and at least a full week so every day of the week is included. Meta's guidance says tests with larger reach, longer schedules or bigger budgets tend to give more statistically significant results.

What should I measure in a catalogue ad creative test?

The metric that matches your goal, usually cost per purchase or return on ad spend, decided before the test starts. Click-through rate is useful context, but a design that wins clicks and loses on purchases hasn't won.

How to test catalogue ad creative properly