A/B Testing, sometimes called split testing, allows you to experiment with different user experiences to optimize your campaigns over time. With A/B Testing, you can test and compare different variations of a campaign to determine which copy will yield the strongest performance based on your goals. This article explains how to set up an A/B Split in your experience flow and track its performance using Attributes.
Before you begin
Identify the goal for your A/B Test. For example, you might want to determine whether or not engagement with an Intro screen leads to higher completion rates.
Plan the different variations of your experience flow you want to test. You could create two paths off of your A/B Split, sending 50% of traffic to an Intro screen and the other 50% of traffic to the first question screen instead.
Have access to the builder map for your experience.
Consider using Attributes to track the performance of each pathway beyond basic reporting.
Set up the A/B Split
You can place an A/B Split anywhere you want within your experience.
Right click on the '+' node in the builder map where you want to add the split.
Select 'Split Traffic' from the menu.
Select A/B test.
Adjust the percentages to determine the traffic split for each pathway (e.g., 50% for each).
Label each pathway clearly to identify the different variations you are testing.
If you want to test more than two options, select 'Add another branch'.
Click 'Create'.
Connect the A/B paths to their respective screens in your experience flow.
Map Attributes for Tracking
Using Attributes is a simple way to track and compare the performance of one pathway over another in your A/B Test.
Map Attributes onto the screens immediately connected to the various branches in your A/B traffic split.
Use descriptive Attribute Values to label the pathways for clarity in reporting. For example:
On the screen following the branch that bypasses the Intro screen, use "No Intro" as the Attribute Value label.
On the Intro screen connected to the branch that includes it, use "Intro" as the Attribute Value label for the button on that screen.
After you publish your experience, any user who engages with it will be assigned the corresponding Attribute value based on the path they took through the A/B split.
Preview and Publish
Preview your experience to verify that the A/B split and attribute mapping are configured correctly.
Publish your experience to deploy the A/B test.
Monitor Performance of A/B split
After launching your experience, you can analyze the results to determine which variation performed best based on your goals.
Navigate into the reporting dashboards.
Monitor the results for your A/B split by accessing the 'Audience Insights' tab.
Review the data associated with the Attributes you mapped to understand the performance metrics for each pathway.
If you are unsure how to interpret your A/B Test data, reach out to our Support team for assistance.
Frequently Asked Questions
What is the best way to name Attributes to best understand the data from an A/B Test?
The Attribute Values you map should clearly describe the pathway or variation they represent. For example, using "No Intro" for the path without an Intro screen and "Intro" for the path that includes one makes the data easy to understand.
I set the traffic split on my A/B Test to send 50% of traffic to one pathway and 50% of traffic to another pathway, but it looks like traffic is skewing more towards one pathway over another. Why does this happen?
The A/B Split works the same way that a coin flip works. Each time you flip a coin, there is a 50% chance that you will get heads and a 50% chance that you will get tails. Each time you flip the coin, the chances of getting heads or tails resets again and has nothing to do with whether you got heads or tails on your last flip. In the same way, each time that a user reaches the A/B split, their chances of being sent down one pathway or another is 50/50, but that chance has nothing to do with any other prior user session. As the data set that you collect reaches a significant volume, you will see the trend between both pathways get closer to 50/50 but it may not ever be a perfect split.