A/B Testing
A/B testing allows you to send one email with a single varying component (or variable) to test which version your audience engages with most. Engagement is calculated by open rates or click-through rates over a period of time.
IMPORTANT LIMITATIONS
- If you are conducting an A/B test:
- You must send to data extensions. A/B testing does not function with reports and campaigns.
- You can only send to ONE data extension at a time.
- You can only exclude ONE data extension at a time.
From Name
- eComm
- CU Boulder Chancellor
- Melanie Jones, eComm Training Manager
- Chancellor Phil DiStefano, CU Boulder
Have you completed a From Name A/B Test? Share your success so others can benefit.
Considerations A From Name A/B Test should not be completed repeatedly. From Names have other implications on deliverability that benefit from consistency.
Send Time
- 9:30 AM
- Sunday evening
- 2:30 PM
- Monday morning
We noticed a consistent uptick in engagement for our gift planning newsletter audience (mostly above age 55) when we sent at 7pm on Sunday evenings.
Considerations Send Time A/B Tests might be limited based your audience. For example, sends to large audiences (over 18,000) can not be sent during business hours due to potentially overloading servers. For those, test a sendtime before 8AM vs after 5 PM.
Subject Lines & Preheaders
- Register today: CU Staff Appreciation 2018 event
- A/B Testing with Marketing Cloud
- Should we save you a seat at Elitch’s?
- Is anyone reading your emails?
We tested subject lines with for an alumni birthday card and learned mentioning a gift is the way to go:
- It's Time to Celebrate YOU!
- Open Rate=16.5% | Click Rate=2.73%
- A Birthday Gift from UCCS!
- Open Rate=21.96% | Click Rate=4.32%
Considerations Subject Line & Preheader A/B Tests are a great feature to continually test.
Content Areas & Entire Emails
- White call-to-action button
- Call-to-action button: 'Register'
- Gold call-to-action button
- Call-to-action button: 'Register Now'
Our team tested a long newsletter against one with an introductory paragraph and a link to 'Read Full Newsletter'. Now we have a better indicator of who is really interested in reading the newsletter. Even better, we had a 200% increase in clicks on stories below the newsletter.
- 26 Illuminating Content Marketing A/B Tests to Run
- How To Create Killer CTAs Using A/B Testing
- A/B test your calls-to-action (CTAs)
Considerations The possibilities are endless when it comes to Content Area or Entire Email A/B Tests - from comparing a completely different email look to getting as granular as the shade of the button and everything in-between.
Summary 1-Pager
Use the roadmap below as a guide to performing A/B Tests in Marketing Cloud. Note the arrows across the top (indicating the level of effort to perform the test) along with the horizontal arrow on the left side (indicating the order tests should be performed).
This 1-pager summary serves as a great checklist of the different tests you've completed and which to plan for next. It's important to view A/B testing as a whole by combining the successes of all indidual A/B tests as one. Together those small steps equate to a big impact.
