Create a Test
Got a great idea for your site? Creating and launching your first test is incredibly easy and only takes a few minutes. We'll walk you through each step, from draft to launch, below.
Last updated
Got a great idea for your site? Creating and launching your first test is incredibly easy and only takes a few minutes. We'll walk you through each step, from draft to launch, below.
Last updated
If you’re new to A/B testing or want some test inspiration, see What to Test for high-impact test ideas.
To get started, click "Create a test" on the Tests homepage. This brings you into the Test Draft page, where you can choose which part of your store to test and which shoppers to include in the test.
The Test Draft page is broken down into the following sections:
Original (A): The current version of your site, where your test will run. Think of your original as the theme, template, or page without any changes.
Variant (B): The edited version of your site that you want to test. Think of your variant as the new theme, template, or page you want to test.
Traffic split: The proportion of visitors that will be directed to the Original (A) and the Variant (B).
Test goal: The metric that determines the statistical significance of your test.
Targeting: The various targeting options, such as devices, visitor types, and audiences, which determine which site visitors participate in your test.
Shoplift enables merchants to easily run A/B tests on their Online Store templates or on their Online Store themes. Depending on the what you want to test, one type may be more suitable than the other.
Test a template or a page: Great for testing changes to a single page or a group of pages, like adding a new section to your homepage or a group of similar product pages.
Test a theme or a global element: Great for testing changes to elements that appear across your entire website, like navigation menus, headers/footers, mini carts/cart drawers, or new apps.
Don't know which to pick? Click "Help me decide" under the two test options.
Once you've decided on a type of test, you'll be prompted to select your Original (A). This is the unedited template or theme that you would like to test a change against. In other words, it is the current location on your site where your test will take place.
Template testing allows you to test homepages, landing pages, product pages, and other template types against one another. Selecting this option opens the template drawer, containing every template in your live theme. Like in Shopify, templates from your theme are categorized by type: Homepage, Collection pages, Product pages, Landing pages, etc.
Template tests will run on every page assigned to your original template to maintain a consistent shopping experience for visitors participating in your tests. Therefore, if you launch a new product that is assigned to a template being tested, it will automatically be included in the test. If you want to test a single page with template tests, see Guide: Testing Individual Pages.
FAQ: What are "assigned" pages?
In Shopify, each page (or distinct URL of your store) is "assigned" to a template, giving these pages their unique design and functionality. Because the original template is the starting place for your test, it must be "assigned" to at least one live page (otherwise, website visitors wouldn't see it). For more info, see Shopify Help Center.
Theme testing allows you to test global elements, new flows, and entire themes by testing two themes against one another. When you select this option, a drawer will open containing your live theme and every theme in your Theme Library. Your original theme must be live at the time of test launch.
FAQ: Why are some themes "missing templates?"
Some themes might be missing templates that are currently assigned to pages on your site by your live theme. These templates are required for your site to function normally. To resolve this issue, see Theme Compatibility.
After you've selected your Original (A), you're ready to select your Variant (B). There are three ways to create a variant in Shoplift, depending on the type of test you've chosen:
Duplicate: Test a variant based on your original template (template tests only)
Select: Test against an existing template or theme
Create with Lift Assist™: Test a recommendation from our library (template tests only)
If you choose to create a variant by duplication, Shoplift will create a pixel-perfect copy of your original template and add it as the test variant (creating an A/A test). Then, click "Edit in Shopify" to make changes to the variant in the Shopify Theme Editor. For more information on the Shopify Theme Editor, see Edit Your Variant.
FAQ: How can I duplicate theme tests?
If you'd like to test a copy of your original theme, you'll need to duplicate your theme in Shopify. Then, in Shoplift, you can to select that duplicate theme and make the changes you want to test from the Shopify Theme Editor. See our guides for specific instructions on setting up common theme tests. By the way, Shopify allows up to 20 themes or 100 themes (Shopify Plus), so be sure to delete outdated themes.
Something not right? While all themes and templates are supported for duplication, the degree to which you can edit your template in the Shopify Theme Editor depends on your theme. Read more about Theme Compatibility.
If you've already created a template or theme that you want to test against your Original (A), you can select it from your library as your Variant (B).
FAQ: How do I test third-party page builder pages?
For template tests, a popular use case for selecting an existing template is testing pages created with third-party page builder apps, such as Replo, GemPages, or Pagefly.
Lift Assist™ is Shoplift's recommendation engine, dynamically providing theme sections to extend your store's functionality. This feature pairs premade, highly-customizable theme sections with high-converting test strategies and is available on Advanced and Pro plans.
By default, each variant is set to receive 50% of total test traffic, but you can change this to any distribution that you want if you'd like to more tightly manage the traffic exposure of specific tests.
We recommend running tests with the default allocation of 50% for both the original and variant templates, but there are certain cases where you may choose to allocate more or less traffic depending on your strategy.
Shoplift uses five key metrics to define success for tests:
Average order value: The average revenue generated by each purchase.
Conversion rate: The percentage of visitors who make a purchase.
Revenue per visitor: The average revenue generated by each visitor (recommended).
Clickthrough rate: The percentage of visitors who proceed to other pages on your site.
Add-to-cart rate: The percentage of visitors who add an item to their cart.
The test goal you select will be used to determine the statistical significance of your test, but all metrics will still be tracked and included in the report. For more information on each of these metrics, view our documentation on Test Metrics.
You can choose to run your test on all devices, or limit your test traffic to a specific device type. To target your test to desktop or mobile traffic only, select the option for "Desktop" or "Mobile".
For more information, see Device Targeting.
In addition to providing device segmentation options, Shoplift also provides segmentation capabilities based on new and returning website visitors. Segmenting your tests based on new and returning visitors can be a powerful way to optimize your store for these major cohorts of users, particularly for subscription-based businesses.
For more information, see New and Returning Visitor Targeting.
FAQ: Why do I have to wait 14 days for visitor segmentation?
To ensure a minimum threshold of accuracy, this segmentation option is not available for tests until we've collected data for a minimum of 14 days from initial installation of the app. After 14 days, you will be able to segment your tests based on visitor type, and our segmentation and reporting will increase in accuracy the longer the app is installed.
If you have more advanced use cases for your test, you can also create custom audiences in Shoplift. Audiences are created and managed on the draft page and are highly customizable by constructing rulesets based on UTM parameters, URL strings, referring domains, and geographic parameters like country.
For a comprehensive guide on creating audiences, see Audience Targeting.
If you need help conducting tests with Shoplift, our support team is always available to assist. Reach out from within the app using the live chat widget in the bottom right of the screen, or email help@shoplift.ai to get in touch.
