⭐Creating an A/B Test
Got a great idea for your site? Creating and launching your first test is incredibly easy and only takes a few minutes. We'll walk you through each step, from draft to launch, below. If you’re new to A/B testing or want some test inspiration, see Ideas to Get You Started.
For examples of popular tests and their setup instructions, see the pages nested under Ideas to Get You Started.
Create a test
To get started, click the button that says "Create a test" on the Tests homepage. This brings you into the Test Draft page, where you can choose what part of your store to test and which shoppers to test it on. If you want to create a test with a pre-made, high-converting section, navigate to Lift Assist instead (available only on Advanced and Pro plans).
The Test Draft page is broken down into the following sections:
Original (A): The current version of your site, where your test will run. Think of your original as the theme, template, or page you want to optimize.
Variant (B): The edited version of your site that you want to test. Think of your variant as the new theme, template, or page you want to test.
Traffic Split: The proportion of visitors that will be directed to the variant instead of the original.
Test Goal: The conversion metric that determines the winning version of your test.
Targeting: Each of the targeting options determines which site visitors participate in your test.
Select your original
When selecting your original, think about what you want to test and where it lives on your Shopify store. For example, changes to product descriptions happen within product templates, whereas changes to navigation bars happen at a theme-level.
Test a template or page
Template tests are used to test changes to a single page, like your homepage or a landing page, or a group of similar pages, like product pages. Selecting this option opens up the template drawer, filled with every template in your live theme. Like in Shopify, templates from your theme are split into categories: Homepage, Collection pages, Product pages, Landing pages, etc.
FAQ - Assigned Pages
In order for a template to be the original (the starting point) for your test, it must be assigned to at least one active page. Each template can be assigned to any number of pages in your Shopify theme. For example:
You might have a template called “Default Product” that is assigned to 10 products.
When you select “Default Product” as your original, all 10 product pages assigned to that template will be included in your test.
Therefore, whenever a visitor clicks on one of these products, they will enter your test and be sent to either the original or the variant.
To test individual pages, reassign pages to a new template in Shopify.
Test a theme or global element
Theme tests are used to test changes to sitewide elements, like navigation menus and mini carts, and you can also test apps and entirely new themes. Selecting this option opens up the theme drawer, filled with every theme that’s on your Shopify site.
FAQ - Theme Compatibility
For a theme to be compatible with your store, it must use the same templates that are currently assigned to pages on your site. For example:
In your live theme, you might have “Product X” assigned to “Product X Template.”
In your theme library, you might have a theme that is missing “Product X Template.”
Because “Product X” would not have an assigned template in this theme, it is incompatible to test against your live theme.
Select your variant
Now it’s time for the main event: implementing the change you want to test. There are three ways to create a variant in Shoplift: (1) duplicating your original and making edits to the copy, (2) selecting a template or theme that’s already been made, or (3) creating a variant with a Lift Assist™ test recommendation. We'll cover each option below.
Template and page tests
Option 1: Duplicate your original template
If you choose to create a variant by duplicating your original, Shoplift will create a copy of the template you originally selected to test and add this to your test as the test variant. You can then make edits to this duplicate in the Shopify Editor to test anything you want, such as new copy, images, or layouts.
Please note that while all themes and templates are supported for duplication, the degree to which you can edit your template in the Shopify Theme Editor will depend on your theme. Read more about Theme Compatibility.
Option 2: Select a template from your theme
If you've already created a template that you want to test against your original, you can select it from your theme and it will be added to your test draft as your variant. A popular use case for this is for testing pages created with third-party landing page builder apps, like Replo, Shogun, or Pagefly. Read more about Testing third-party pages and templates.
Option 3: Use a Lift Assist™ recommendation
Another option for creating a test variant is to use a Lift Assist™ test recommendation. Lift Assist™ is Shoplift's recommendation engine which dynamically provides theme sections to extend your store's functionality and give you the tools you need to run powerful tests otherwise not capable with your theme.
Theme and global element tests
For theme and global element tests, the only option for choosing a variant is to select a theme that’s already been made. Therefore, to test a copy of your original, you must duplicate your live theme in Shopify and make changes to that duplicate theme. Then, in Shoplift, you will be able to select that duplicate theme.
Duplicating your theme in Shopify:
In Shopify, navigate to your Online Store. Click on the ellipsis by your live theme, and select “Duplicate” to create the variant theme for your test. We recommend renaming this duplicate theme with “Shoplift Test” at the beginning, for clear communication within your team.
After creating a duplicate theme in Shopify, click on the “Customize” button on this new theme to enter the Shopify Editor. Be sure not to click the “Customize” button on your live theme.
In the Shopify Editor, make the changes you want to test. When you’re done, save your changes and head back to Shoplift.
In Shoplift, click “Create a Test” to enter the Test Draft page.
In the Test Draft page, click “Test a theme or global element,” and select your live theme.
To select your variant theme, click “Select variant theme” and select the theme you just prepared for testing.
Adjust your traffic allocation
By default, each variant is set to receive 50% of total test traffic, but you can change this to any distribution that you want if you'd like to more tightly manage the traffic exposure of specific tests.We recommend running tests with the default allocation of 50% for both the original and variant, but there are certain cases where you may choose to allocate more or less traffic depending on your strategy.
We recommend running tests with the default allocation of 50% for both the original and variant templates, but there are certain cases where you may choose to allocate more or less traffic depending on your strategy.
Choose a primary goal
Once you've selected a template and created a variant, you can define a primary goal for your test. Shoplift uses five key metrics to define success for tests:
Average order value: The average revenue generated by each purchase.
Conversion rate: The percentage of visitors who make a purchase.
Revenue per visitor: The average revenue generated by each visitor.
Clickthrough rate: The percentage of visitors who proceed to other pages on your site.
Add-to-cart rate: The percentage of visitors who add an item to their cart.
The primary goal you select will be used to determine the statistical significance of your test, but all metrics will still be included in the report. For more information on each of these metrics, view our documentation on Goals and reporting metrics.
Target specific devices
Next, you can choose to run your test on all devices, or limit your test traffic to a specific device type. To limit your test to desktop or mobile traffic only, select the option for "Desktop" or "Mobile".
Target new or returning visitors
In addition to providing device segmentation options, Shoplift also provides segmentation capabilities based on new and returning website visitors. Segmenting your tests based on new and returning visitors can be a powerful way to optimize your store for these major cohorts of users, particularly for subscription-based businesses.
Please note that new and returning visitor segmentation becomes more accurate over time as data is collected from our tracking script. To ensure a minimum threshold of accuracy, this segmentation option is not available for tests until we've collected data for a minimum of 14 days from initial installation of the app. After 14 days, you will be able to segment your tests based on visitor type, and our segmentation and reporting will increase in accuracy the longer the app is installed.
Target custom audiences segments
If you have more advanced use cases for your test, you can also create custom audiences in Shoplift. Audiences are created and managed on the draft page and are highly customizable by constructing rulesets based on UTM parameters, URL strings, referring domains, and geographic parameters like country.
For a comprehensive guide on creating audiences, see Audience Targeting.
Last updated
Was this helpful?
