Facebook recently rolled out a new A/B testing feature called “Split Testing”, which allows advertisers to test multiple campaign variables at once to produce more granular and actionable performance learnings than setting up a manual A/B testing structure.
how it works
- The feature will split an audience into random, non-overlapping groups.
- The two (or three) groups are completely randomized to ensure a fair test.
- Each ad set is then given only one distinct difference for testing that will run the exact same creative. The only variables available for testing are: Audience, Delivery Optimization settings, and Placements.
- Once the variable is selected and set up, a minimum budget of $800 must be given to the test (the budget can be split evenly (50/50) or given a weighted budget split).
Screenshot via Facebook Ads Manager
A few additional considerations to remember while setting up your Split Test:
- A minimum of three days and a maximum of 14 days is required for campaign duration.
- The campaign cannot use any campaign objective. The success of the test must be measured on one of the following direct response objectives:
- App Installs
- Conversions
- Leads from lead generation ads.
This means Facebook will not allow A/B testing for campaigns set up for website clicks, video views, or any other awareness goal campaigns.
what this means for marketers
Ultimately, Facebook’s new Split Testing allows marketers to more easily find audiences and campaign targeting that drives more revenue or leads. It will reduce ad spend used for testing and deliver more definitive, statistically significant results. As these new testing features begin to scale, we should see an increase in ROAS for marketers on Facebook and Instagram.
The largest benefit of the A/B testing is that the two (or three) ad sets will be bidding on separate audience, which gives both groups an equal chance in the auction. Previously, marketers would either need to do a “run it and see how it performs” approach or testing between ad sets, which would often mean targeting the same individuals and creating a non-valid A/B test. Here are the things you should consider testing with this new feature:
Audiences
Spend less to test audiences by testing them under one campaign and learn which audience targeting performs best. Have you ever wondered which audience between Nike fans or Adidas fans are better aligned with your brand? Now you can test these things quickly and easily.
Delivery Optimizations
Learn which optimization setting brings in the most revenue or leads. Settings include optimizing for: daily unique reach, conversions, link clicks, or impressions. Or you may want to test optimizing for a conversion window of 1 day vs. a conversion window of 7 days. One thing we have learned from Facebook is that as you shift your optimization strategy, the audience shifts. An audience will start out as a large pool, and as you select your optimization strategy, Facebook will determine who to serve your ads too. Facebook’s approach to this kind of audience refining has often been unclear but now this feature will allow advertisers to test various settings and learn what works best for them.
Placements
Facebook recommends testing custom placements against automatic placements only; however, this is a unique opportunity to directly test mobile vs. desktop or test different devices and operating systems.
Lastly, when running these tests, make sure you’re not using the audience in any other Facebook campaigns because it may cause overlapping audiences and affect testing results. Make sure you also have a clear objective for your test, which can simply be a hypothesis of what you think will happen. Then once your test is complete, use your learnings to carry over into future campaigns and keep on optimizing.
Creative Testing
What Facebook has not released is a tool to easily A/B test creative, something the industry has been anxiously awaiting. While this functionality is not currently available, we predict it is in the near future. In the meantime, be sure to test creative at different times to give each ad version a fair chance. While it may not eliminate time variables, it will make sure creative is served to gather enough data and will eliminate overlapping/competing ad sets.
Source: Facebook Advertiser Help Center – Split Testing, Facebook Advertiser Help Center – A/B Testing
