Using A/B Tests In Workflows

A/B testing helps validate hypotheses and understand the preferences and behavior of the target audience

Built-in functionality allows testing:

Using workflows, you can test the effectiveness of various elements of triggered messages, such as text, calls to action, images, or sending via different channels.

Use the Split block from the Conditions blocks for split testing.

Workflow editor Conditions blocks list with Split highlighted in the sidebar and a Split block connected to Start/Stop configuration, showing a 50/50 split

Let's consider an experiment using the example of an abandoned cart:

  • The first message contains a discount on items abandoned in the cart:
Abandoned cart email showing a 10% discount offer, the Samsung Galaxy S21 product with a discounted price, and a Buy button
  • The second message contains a timer with a countdown until the end of the discount.
Abandoned cart email showing a 10% discount offer with a countdown timer, the Samsung Galaxy S21 product, and a Buy button

The goal of this test is to check whether a timer counting down the time until the end of the promotion influences the purchase decision.

Creating a Workflow for Testing

  1. Go to Triggers → Workflows, and click the New workflow button.
Workflows page with the Automation menu, Workflows tab, and New workflow button highlighted
  1. Enter its name and tags (optionally).
Workflow editor titled Abandoned Cart with an A/B test tag, showing the Start block and Start/Stop configuration panel
  1. By default, the Start block already starts the workflow. Then add blocks:
  • Split, where contacts are automatically distributed between workflow branches 50/50. To change the ratio, drag the slider in the block settings to the left or right.
Workflow editor Split block settings panel with the 1 part and 2 part fields highlighted, both set to 50%
  • Email for A and B branches; select the message in the block parameters.
Workflow editor with a Split block branching into two Email blocks at 50% each, and the Message field highlighted showing Abandoned cart discounts
  • End.

The workflow will look like this:

Workflow editor titled Abandoned Cart with A/B test tag, showing a Split block branching into A and B Email blocks at 50% each, connected to End
  1. Click Save and exit.Ā 

After launching theĀ workflow, the system will randomly distribute contacts between two branches in a 50-50 ratio. Each contact will receive only one message, according to the branch they fall into.

Setting Up Workflow Launch and Checking Results

  1. Go to Triggers → Workflows and click Start/Stop configuration in the field of the created workflow.
Workflows list with the Start/Stop configuration link highlighted for the Abandoned Cart workflow tagged A/B test
  1. Enable the Start Configuration slide button, select Regular, and a dynamic segment for the campaign.
Start/Stop configuration panel with Start configuration enabled, Regular selected, Segment set to Abandoned Cart, and Once a day selected
  1. Configure launch conditions and click Apply.
  2. Activate the workflow.
Workflows list with the play icon highlighted to activate the Abandoned Cart workflow

Later, you will be able to check and compare the test results.

To do this:

  1. Click three dots in the field of the created workflow and select View campaign report.
Workflows list dropdown menu with View campaign report highlighted for the Abandoned Cart workflow
  1. Choose a period and compare the indicators for each message.
Reports page filtered by Abandoned Cart with Today selected, showing Opened, Clicked, and Click-to-open rate metrics highlighted for both test campaigns

Did this page help you?