Changelog
Week of July 7, 2024
Added additional logic to our testing script to improve speed, performance, and reliability. Test actions like launch, pause, and end are now represented on the storefront much more quickly.
Week of June 23, 2024
Added additional logic to our theme preview detection to reduce impact on admin workflows when developing and previewing non-test themes.
Week of June 16, 2024
We fixed an issue wherein when a test was paused, events performed during the paused period by visitors associated with the test prior to the paused period would continue to be attributed to the test during the paused period.
Week of June 9, 2024
We've implemented additional redundancies and fallbacks into our tracking approaches to ensure that visitors will continue to be tracked and attributed to tests in the event that tracking data could get cleared out by various browsers.
Week of June 2, 2024
Due to popular demand, we no longer auto-end tests when our safety checks detect a theme incompatibility when running template and theme tests, but will pause them instead.
We implemented these safety checks to ensure that tests never have an adverse affect on storefronts if theme files become out of date between variants.
Now, if an incompatibility that could cause an issue is detected on an active test due to a recent theme or template change, we will pause the test and provide an interface for troubleshooting and resolving the issue. Once resolved, the test can be resumed.
Fixed an issue where the test list would load indefinitely when sorting by Test Progress.
Fixed an issue where duplicating older tests would provide an infinite loading experience due to the template or theme no longer being available. We now provide a proper error message.
We've made a number of improvements to our in-app screenshots of templates and themes to ensure they are as up-to-date as possible.
Week of May 26, 2024
We've made various fixes and improvements to the speed of user-driven testing actions like launching, pausing, and ending tests so that they go live more quickly on storefronts.
Week of May 19, 2024
Added a loading state to the theme drawer when loading themes to better represent the process of loading themes from your store.
Fixed an issue where test variant creation would fail if there were no products on a store.
Week of May 12, 2024
Fixed an issue where variant creation could occasionally become hung up by other background processes in the app.
Week of May 5, 2024
Updated our method of detecting theme previews to be more reliable for theme tests.
Implemented a fix for rare instances where our cart events API could occasionally fail to send add to cart events.
Week of April 28, 2024
We've added a new "Theme Compatibility" modal that will allow merchants to better understand why a theme may not be compatible with their store for a theme test, so that they can effectively troubleshoot and resolve any compatibility issues.
We've implemented improved bot detection in our tracking script to exclude more bots from test data.
We've added some additional protections to our tracking script so that if multiple instances of the script are incorrectly present, the script will only execute once.
On some themes, we were incorrectly reinjecting our script hourly during our integrity checks, which unnecessarily cluttered the Shopify Admin theme changelog. This is now resolved.
We've fixed an issue where tooltips could persist in the UI even when scrolled out of view.
We've fixed an issue where removing an audience from drafts would not work in some instances.
We've added improved scroll functionality to our template and theme drawers so that more rows can be viewed concurrently.
Fixed a UI issue where extremely long theme and template names could spill out of the card in the template and theme drawers.
Fixed an issue where deeplinks to the Shopify Editor will now correctly update to the latest live theme on template tests.
Added a "refresh preview" button on template and theme cards on drafts and reports so that merchants can easily request updated test images on an as-needed basis.
Improved script execution timings on geo-targeted tests.
Fixed an issue where an erroneous parameter on our deeplinks to the Shopify Editor could prevent the ability to preview templates on other products.
We've updated our default goal for all tests from conversion rate to revenue per visitor, since it is a more accurate, universal metric by which to measure business performance.
Week of April 14, 2024
Our screenshot functions now hook into theme webhooks to provide even more timely updates to images.
We've fixed issues where some compatibility information banners were incorrectly triggering on reports.
We've cleaned up where, when, and how informational banners trigger and show throughout our app, to make our UI more intuitive and provide less cognitive load, so you can focus on what matters!
We've killed the exit confirmation modal on the test draft page. All drafts are saved automatically, so this was redundant.
When pressing "back" in your browser from the draft, active report, or ended report pages, you will now correctly return to the relevant tab in the tests list.
Week of April 7, 2024
We've released our biggest update to test functionality yet: Theme Testing!
Merchants can now test one theme against another to run a host of powerful tests that previously were outside of the scope of template tests, like testing global elements like navigation menus, mini carts, theme app embeds, and more!
Theme tests can run alongside template tests as well as other theme tests and are mutually exclusive by default.
Merchants can draft theme tests on unpublished themes to publish at a later date.
We've implemented a host of default protections to keep your store safe when running theme tests, including robust compatibilty checks and comprehensive UI solutions to troubleshoot and solve them.
We've also implemented a more intuitive traffic slider component on the test draft page, to more easily allocate traffic to each variant.
Week of March 31, 2024
Fixed an issue where our liquid tracking script would not inject on some themes if a pre-existing script was not detected.
Week of March 24, 2024
We've added a timestamp to our liquid tracking script and script data metafields so merchants and developers can easily tell when they were last updated.
Fixed an issue where our script updates for the purposes of traffic balancing could occasionally fail to run.
Occasionally, merchants could end up with multiple scripts in the <head> of their theme.liquid due to development workflows. We now proactively remove any duplicate instances of our script.
Fixed an issue with certain percentage values on reports when enormous datasets were presented. This was a frontend issue only, underlying data is unaffected.
Week of March 17, 2024
Fixed an issue on the Lift Assist template browser where we'd incorrectly show a "must select a template with a single assignment" tooltip when selecting a variant.
We now properly truncate the visitor usage percentage in tooltips when a plan is approaching a usage overage.
Fixed an issue where the "Set me up" button for Brand Styles would not hide once Brand Styles were set up.
Fixed the "Select an asset" button on the Lift Assist page to correctly pull up the test browser.
We now disable duplication for OS1 templates that cannot be converted to JSON, since section settings are scoped globally when making edits in the Editor on OS1 templates.
We now indicate when a tested template includes Shopify Markets contexts, so that similar contexts can be created for test variants
We now correclty hide context templates from the template drawer when selecting a variant.
Fixed an issue where large template tests running on several thousand pages would exceed an internal limit for recognized pages to include in the test.
Fixed an issue where our script was not executing on markets subpaths.
We've made additional tweaks to our plan prices and featuresets to better reflect the value of each plan:
Core 50k monthly and annual plans have a new, reduced price.
We've upgraded the visitor limit on our Advanced 500k plan to be 1M.
We've upgraded the visitor limit on our Pro 250k plan to be 1M.
We've upgraded the visitor limit on our Pro 500k plan to be Unlimited.
Lift Assist is now available to all Advanced plans in addition to Pro plans.
Week of March 10, 2024
We've made big improvements to our script so that it can no longer contain outdated data when publishing new themes that are based on older branches.
Week of March 3, 2024
Week of April 7, 2024
We've introduced our biggest upgrade to testing since we launched nearly a year ago: theme testing!
Merchants can now test one theme against another, and this feature is available to all merchants on all plans.
Theme tests can run alongside template tests, and merchants can even launch multiple theme tests simultaneously. All tests are properly isolated so that visitors and data cannot cross-contaminate tests.
Merchants can draft theme tests on unpublished themes, and launch them as soon as the theme goes live.
Active theme tests are responsive to merchants publishing new live themes and will update accordingly.
We've implemented a host of protections to prevent incompatibility issues between live theme and tested variant themes, to make the experience as seamless as possible.
Week of March 31, 2024
We've added a timestamp to our script in the theme.liquid code to easily identify the most recent time the script has been updated, for version identification purposes
Occasionally, our script could end up with multiple instances on a theme.liquid file as a result of the merchant-development workflow. We've added a function to proactively remove duplicate scripts from the <head> in theme.liquid if a script has unintentionally been added multiple times.
Week of March 17, 2024
We've fixed an issue on the Lift Assist template browser where we'd incorrectly show the cross-template "must select a template with a single assignment" tooltip when selecting a variant
We've fixed the "Set me up" button for Brand Styles on the "How does it work?" Lift Assist drawer to correctly hide once Brand Styles are already set up
We've fixed the "Select an asset" button in the "How does it work?" Lift Assist drawer to correctly pull up the test browser
We've added a tooltip to the draft page when a template that contains contexts is selected, indicating that the contexts must be manually recreated on the variant.
Week of March 10, 2024
We've rolled out a new version of our script which leverages the power of metafields. Occasionally, merchants would encounter their theme liquid code being cached by Shopify, which could introduce issues with tests. By leveraging metafields our script now more effectively defends against this behavior.
Week of March 3, 2024
We've made a number of improvements and adjustments to our plans and pricing to be more flexible than ever, to more effectively accommodate stores of different sizes and provide more specificity regarding the various feature-needs of stores on each plan.
These plan adjustments are completely opt-in for all of our existing merchants. If you are happy on your current plan, then there is no need to change to the new plan structure.
Annual payment options are now fully supported on all plans, and include significant cost savings. This was a frequent request by merchants and we're excited to roll this out.
All plans are now feature and usage based. Previously, plans were entirely usage-based, which meant that smaller merchants might be paying for features that they don't need. For more information on the new pricing structure, visit our pricing page.
For very large stores (monthly traffic in excess of 1M visitors) we have a custom plan option, with which we'll work with you to identify the right fit of features, usage, and price. For inquiries, please contact sales@shoplift.ai.
We've made account suspension when a store hits usage overage more forgiving. You will now be able to more easily navigate your existing tests and reports during a state of overage.
Week of February 25, 2024
In cases where a store had exceeded its usage limits, we fixed an issue where there could be a time delay between a plan upgrade and the resulting resolution of overage-related account suspension.
Fixed an issue where our "Apply Variant" function was not applying to tested templates as expected.
Week of February 18, 2024
Added several new Lift Assist tests for Cart, Search, Blog, and Article templates.
Fixed an issue where case-sensitivity for UTM values in custom audiences was not applying as expected.
Fixed an issue where Lift Assist tests for new template types were incorrectly disabled in the Lift Assist browser.
Properly disabled the "Export to CSV" button in test reports if a test did not have any data to export.
We've made the error message more clear when a Lift Assist test was created on a template that already contained the maximum of 25 sections.
Week of February 11, 2024
Fixed an issue where Shopify Plus Partner Sandbox (development) stores were unable to proceed with installing the app due to a recent change in Shopify's API naming schema for this type of development store.
Fixed an issue where tests that were auto-ended due to theme incompatibility or missing assets would be missing an end date in the test report UI.
Fixed an issue where if a template had references to code that was incompatible with Shopify's API, our template duplication processes could stall indefinitely, preventing a test variant from being created.
Fixed a rare issue where if an order made in a non-store currency was delayed by Shopify the initial non-store currency revenue amount was being attributed to tests.
Week of February 4, 2024
We've expanded template testing! You can now run tests on cart, search, collection list, blog, and article templates.
Fixed an issue where tests would not correctly identify URLs with symbols (like ™) as valid test pages
Week of January 28, 2024
Added some performance improvements to our Bayesian analysis to allow it to run more efficiently on large datasets
Fixed an issue where our scanning and analysis of themes was triggering change history logs more frequently than necessary in the Shopify Admin
Improved when we interface with Shopify's APIs for test launch and test end actions to speed up both processes for users
Week of January 21, 2024
Our shoplift.js script will now only inject into the top of the <head> of your theme.liquid initially. If merchants or developers have reasons to modify the position of the script, we will not reposition it upon re-injection.
If merchants already leverage "view" parameters on certain website links to provide alternate template views to shoppers, we will now exclude those visitors from tests to ensure website functionality operates as originally desired.
We've made launching and ending tests a bit quicker by optimizing our processes for how we interact with themes.
Week of January 14, 2024
We've expanded the functionality of template testing! You can now test different types of templates against each other.
This capability enables merchants to conduct much-requested tests with ease: test homepages versus landing pages, product pages versus landing pages, and more!
Fixed an issue that would prevent referrer segmentation from working in certain instances where the browser could hand Shoplift a referring domain value in an unexpected format.
Week of January 7, 2024
Fixed an issue where reports for certain tests could occasionally become delayed.
Made some adjustments to when our Bayesian analysis runs for tests to improve scalability under large test loads.
Week of December 17, 2023
Our latest feature is here: Audiences! With this release, you can now run tests on all sorts of tightly-specified visitor segments that extend beyond mobile/desktop or new/returning visitor targeting.
Run multiple A/B tests on a single template and target each test to different audiences.
Create audiences by defining groups of website visitors with custom logic and rules, using UTM parameters, query strings, geography, and referring domains.
Leverage one-click segments with our pre-built "common audiences" for popular use cases like Facebook, Instagram, Google, TikTok, and more.
Save, reuse, and modify your audiences over time as you learn what works and what needs improvement.
Week of December 10, 2023
Fixed an issue where the "Test progress" tile in test reports could show a "trending neutral" state even when the trend was positive or negative.
Fix an issue where the "Test duration" tile in test reports could show an extended test runtime in the event that a test was ended from a paused state.
Week of November 26, 2023
We've made a handful of continued speed improvements on our testing script. Our script no longer needs to communicate with Shoplift's servers to run, so your tests should redirect as fast as computer-ly possible, and the whiteout function from our anti-flicker script is now imperceivable.
Fixed a rare issue where on some stores, due to pageload times, the Shoplift Pixel could execute quicker than our testing script, resulting in incorrect clickthrough rates for test variants.
Implemented a handful of fixes and improvements for countdown timers in Lift Assist sections, including proper timezone support.
Implemented a fix for when removing a variant from a test draft while the variant was still being created would result in an infinite loading state until the page was refreshed.
Expanded our time-to-scan window for Brand Styles from 30 seconds to 60 seconds to accommodate stores for which the scan would take particularly long due to various reasons.
Week of November 19, 2023
Screenshots of your store make our app pretty (because your stores are pretty), but sometimes they can take a while to load depending on the store. We've added some additional in-app messaging to inform merchants when this is the case.
Fixed an issue where full-page screenshots for test previews could occasionally drop content lower on the page if a website had certain lazy loading configurations.
Added improved loading states when creating Lift Assist tests to provide more feedback about progress.
Implemented the ability to run tests with a 0/100 traffic allocation.
Added timer-based Lift Assist tests for landing pages.
Week of November 12, 2023
Added a much-requested feature to edit variants on a live test. The variant card on the test report page now has a dropdown with an "Edit in Shopify" option, so merchants can make necessary edits to their active tests to correspond with sales and other external promotions. This allows merchants to ensure that their A version and B version are in sync with required changes that lie outside the scope of what is being tested.
Fixed an issue where occasionally, our screenshot service would return a blank image preview because it did not wait long enough for content to load.
Fixed an issue where if a tested page included a forward slash before the start of query parameters (like /products/red-hat/?utm_source=facebook) it would not be recognized as a valid visitor for tests.
Week of November 5, 2023
Fixed an issue where if a test was paused for long periods of time, our estimated time to significance calculations could present inaccurate dates.
Made an adjustment to account overage handling, wherein if an account goes over their allotted plan limits and tests are paused, upon overage correction the tests no longer auto-resume.
Fixed an issue on the test report page where our UI visualization of paused periods did not account for Daylight Savings Time.
Week of October 29, 2023
Fixed an issue with our Image Hero Lift Assist section where the benefits block was not responding to Mobile/Desktop only settings.
Added additional optimizations to our screenshot processes so that screenshots can be returned in shorter timeframes without dropping content, particularly for newly onboarded stores.
Fixed an issue with form validation on our onboarding contact forms that would throw errors before users had typed anything.
Week of October 22, 2023
Fixed an issue where stores with thousands of products might have to wait an unreasonable amount of time for templates to load.
Fixed an issue where screenshot updates could get hung up or delayed because screenshots were inadvertently being queued more frequently than necessary.
Fixed an issue on the test list page where due to rounding inconsistencies, the Lift reported on the test list page and the test report page would be differ by a few hundredths of a percent.
Fixed an issue with rounding inconsistency on the test report page. All values now round up or down to the nearest hundredth of a percent.
Fixed an issue in our Report CSV export where certain numerical values were formatted incorrectly
Week of October 15, 2023
Added an error handler to our Brand Styles import process to indicate if a failure to import occurred, to enable better troubleshooting of issues.
Fixed an issue where our Brand Styles snackbars for various edit functions would not auto-dismiss.
Fixed an issue in our CSV export of test data wherein conversion rate would include extra decimals.
Fixed an issue in Test Reports wherein the performance over time charts wouldn't display the store currency symbol correctly.
Week of October 8, 2023
New feature! We've released reporting for all major subscription apps, including Recharge, Skio, Loop, Prive, Subify, Yotpo, Ongoing Subcriptions, Ordergroove, Appstle, and Bold! Integration with these apps works out-of-the-box, with no prior setup required. To view subscription reporting for newly launched tests, simply go into your test report and change your view for different device, visitor, or channel tables to "Subscription" metrics.
Fixed an issue where launching the app from the Shopify Admin could take several seconds to load.
Fixed an issue where our "Probability to win" badges in test reports were changing color (indicating success or failure) prior to significance being achieved.
Week of October 1, 2023
New feature! We've released both Clickthrough Rate and Add-to-Cart Rate as available test goals. These new goals are perfect for merchants with lower traffic volumes or merchants who want to conduct tests for engagement-focused KPIs rather than purely revenue-driven strategies.
New feature! We've released a new test segmentation option - new and returning visitors - so you can better understand your shoppers. Segmenting tests by new or returning visitors is a great way to gain insight on how these distinctly different customer cohorts interact and engage with your store. Every test will now report on these visitor segments.
Fixed an issue where our Brand Styles importer would miss fonts on some websites.
Week of September 24, 2023
We've made some great improvements to our tracking script!
Our script now uses a snippet, so it's more concise and easier for developers to work with in their theme.liquid files.
We've also migrated all of our event tracking from our javascript snippet to the Shoplift Web Pixel, which means that all test-related events are now tracked server-side. This makes our javascript snippet faster than ever before, because it now handles just redirects and our anti-flicker solution.
Implemented adjustments to the way we ingest order data so that edits and updates to orders post-purchase (like upsells) are now reflected in test report data.
Adjusted the way our signup flow works so that merchants will now pick a plan best suited for them upon installing the app, rather than defaulting to the Starter plan and needing to manually upgrade shortly thereafter.
Fixed an issue on the Image hero Lift Assist section where the mobile setting "anchor at section bottom" would apply to desktop.
Week of September 17, 2023
Implemented improved coverage on order tracking by implementing a Shoplift Pixel via Shopify's web pixel API, which enables us to properly track and attribute orders made by payment methods which circumvent Shopify checkout.
Fixed an issue where the product template list was timing out for stores with several thousand products.
Fixed an issue where Lift Assist creation would stall out on the first attempt after starting up the app.
Fixed an issue where our pie charts on test reports were being resized inconsistently on different viewports.
Week of September 10, 2023
New feature: Quick Start Guide for new users! Users on trial will now have various cards that help them more effectively navigate various app features when getting started with Shoplift.
Fixed an issue where button styles were not being rendered correctly in the Brand Styles Editor.
Renamed "Landing pages" to "Pages" on template browser screens to be more in-line with Shopify naming conventions.
Added a "fold height" indicator to variant previews on desktop and mobile to indicate where the fold will occur on shoppers' devices.
Updated template preview toggles to enable easier switching between viewing original, variant, or a comparison.
Added an "Ended" tab to the Dashboard so merchants can view recently ended tests.
Fixed an issue where longer test names could cause a break in formatting in test lists.
Week of September 3, 2023
Fixed an issue where merchants with large numbers of collections were not having their template assignments returned correctly in the template browser.
Changed "Certainty" to "Progress" on test reports to provide merchants with a more semantic indicator of test significance, because "certainty" was misleading when it was less than 95%.
Removed "High confidence" as a test status, because merchants were inferring it to mean statistically significant. This has been changed to a "Nearing significance" status.
Ended tests now only have two statuses: Significant or Inconclusive.
Tests that are taking a long time to reach significance now have two new statuses: "Long timeline expected" (if the time to significance is estimated to take between 14 and 60 days, after the test has been running for 14 days) and "Significance unlikely" (if the time to significance is estimated to take greater than 60 days, after the test has been running for 14 days).
Fixed an issue where our line charts were not plotting points for a single day of data.
Fixed an issue where our line charts were not plotting points for the latest day of data.
Week of August 27, 2023
New feature: the new test creation experience is here! We've made a number of improvements to the workflows and UI.
Creating a draft no longer requires going through a lengthy flow. Get to the draft screen from the dashboard in one click, and add templates to your test from there.
Lift Assist is now integrated directly into this draft flow, and the full library of tests can be browsed from the draft screen.
Test drafts are now extremely flexible. If you change your mind about what you'd like to test, you no longer have to create an entirely separate draft. Easily swap out your original or variant templates with a few clicks.
We have improved how Lift Assist tests are organized, which are now categorized by different concepts.
We've greatly improved our time-to-creation for variants by breaking up various processes (variants are now typically created in between 1-2 seconds).
We've made improvements to our screenshotting methods to ensure that screenshots for your original and variant templates are always up-to-date on your drafted tests.
Week of August 13, 2023
New feature: we now support templates created with third-party pagebuilder apps like Replo, Shogun, Pagefly, Gempages, and more!
Third-party templates can be located in the template browser and added to tests like any other template in your theme.
We no longer limit the templates merchants can test on OS1 themes to liquid templates that we can programmatically convert to JSON. OS1 merchants can now test any liquid template on their theme.
We've updated how we generate filename suffixes for templates created with Shoplift to be more human-readable and more easily identified as a Shoplift template.
Fixed an issue where merchants could technically have two tests for the same template live at the same time by inadvertently exploiting the pause feature. Now, if a test is active on a template, and a test for the same template is paused, merchants cannot resume the paused test until the currently active test is paused or ended.
Fixed an issue where test duration would calculate incorrectly when paused periods occur.
Total test duration is now reported to the hour, rather than to the day.
We've hidden the percentage change badge from variant revenue in reporting, as this metric is affected by traffic weighting to each variant and did not make sense to provide an improvement percentage for.
Development stores can now install the app without a valid payment method and will be presented with a test charge. This enables merchants and agencies to take Shoplift for a spin on development environments for their production stores.
Week of July 30, 2023
New feature: you can now pause and resume tests!
Tests that are paused will continue to be organized under the "Active" test tab, but will have their significance badge replaced with a "Paused" badge.
Merchants who have gone over their plan limits will now have their tests automatically paused and resumed when plan limits are corrected.
We've added detailed variant session information to the pie chart tooltips on the reports pages.
Week of July 23, 2023
We've adjusted the way we handle testing existing templates from your theme. Previously, Shoplift would always create a duplicate of the template selected from the theme to prevent users from editing live templates with changes intended to be isolated to a test. This introduced certain issues for users with theme code that was dependent on the suffix of the original template selected from the theme. Now, when selecting a template from your theme to test as a variant, we do not duplicate the template, but rather load in the template selected as a variant to be tested against the selected original. This behavior is more inline with what users would expect.
Fixed a bug on the "Create your variant" step of test creation wherein the image containers would always show the empty state, regardless of whether a screenshot was available.
Fixed an issue on the Homepage tab of variant selection for OS1 stores, wherein for newly installed stores the tab might present OS1 homepages as incompatible.
Week of July 16, 2023
Removed the requirement for screenshots to be taken inline with draft creation, which was leading to error screens for a small percentage of users in the event that the screenshot service timed out on their theme/store.
Added screenshot empty states to the Variant selection step in the Create Test flow, which previously would return broken image links.
Fixed an issue with the Image hero section where mobile buttons were always set to "wide".
Fixed an issue where Multi-message bar and Timer bar would use the desktop size for text when set to "Large" on mobile.
Updated date labels on chart tooltips on the Test Report page to provide day of week and year.
Updated date labels on charts on the Test Report page to provide year.
Week of July 9, 2023
Fixed an issue where the full-page preview on the draft screen would not update until manually refreshing the page after creating a test draft.
Fixed an issue where Estimated Time to Significance in Test Reporting could provide very low "day" values for tests with small sample sizes.
Fixed an issue where a dialogue banner indicating that your template has changed might erroneously populate.
Fixed an issue when a Test was ended, the Report page would not update with the End Date until refreshing the page.
Fixed an issue where the values on the Test Report tables would round down to the nearest hundredth of a decimal point. Values now properly round up or down based on the thousandth decimal point.
Fixed an issue where the dropdown on the Drafts list said "View report" instead of "View draft".
Fixed an issue with Single Message Bar, Multi-Message Bar, and Timed Message Bar wherein they would not properly hide/show on devices according to the "Show on mobile only" setting.
Fixed an issue with Single Message Bar wherein "Mobile text size" would not update correctly.
Last updated