Changelog
New features, fixes, and improvements
Week of March 3, 2024
We've made a number of improvements and adjustments to our plans and pricing to be more flexible than ever, to more effectively accommodate stores of different sizes and provide more specificity regarding the various feature-needs of stores on each plan.
These plan adjustments are completely opt-in for all of our existing merchants. If you are happy on your current plan, then there is no need to change to the new plan structure.
Annual payment options are now fully supported on all plans, and include significant cost savings. This was a frequent request by merchants and we're excited to roll this out.
All plans are now feature and usage based. Previously, plans were entirely usage-based, which meant that smaller merchants might be paying for features that they don't need. For more information on the new pricing structure, visit our pricing page.
For very large stores (monthly traffic in excess of 1M visitors) we have a custom plan option, with which we'll work with you to identify the right fit of features, usage, and price. For inquiries, please contact sales@shoplift.ai.
We've made account suspension when a store hits usage overage more forgiving. You will now be able to more easily navigate your existing tests and reports during a state of overage.
Week of February 25, 2024
In cases where a store had exceeded its usage limits, we fixed an issue where there could be a time delay between a plan upgrade and the resulting resolution of overage-related account suspension.
Fixed an issue where our "Apply Variant" function was not applying to tested templates as expected.
Week of February 18, 2024
Added several new Lift Assist tests for Cart, Search, Blog, and Article templates.
Fixed an issue where case-sensitivity for UTM values in custom audiences was not applying as expected.
Fixed an issue where Lift Assist tests for new template types were incorrectly disabled in the Lift Assist browser.
Properly disabled the "Export to CSV" button in test reports if a test did not have any data to export.
We've made the error message more clear when a Lift Assist test was created on a template that already contained the maximum of 25 sections.
Week of February 11, 2024
Fixed an issue where Shopify Plus Partner Sandbox (development) stores were unable to proceed with installing the app due to a recent change in Shopify's API naming schema for this type of development store.
Fixed an issue where tests that were auto-ended due to theme incompatibility or missing assets would be missing an end date in the test report UI.
Fixed an issue where if a template had references to code that was incompatible with Shopify's API, our template duplication processes could stall indefinitely, preventing a test variant from being created.
Fixed a rare issue where if an order made in a non-store currency was delayed by Shopify the initial non-store currency revenue amount was being attributed to tests.
Week of February 4, 2024
We've expanded template testing! You can now run tests on cart, search, collection list, blog, and article templates.
Fixed an issue where tests would not correctly identify URLs with symbols (like ™) as valid test pages
Week of January 28, 2024
Added some performance improvements to our Bayesian analysis to allow it to run more efficiently on large datasets
Fixed an issue where our scanning and analysis of themes was triggering change history logs more frequently than necessary in the Shopify Admin
Improved when we interface with Shopify's APIs for test launch and test end actions to speed up both processes for users
Week of January 21, 2024
Our shoplift.js script will now only inject into the top of the <head> of your theme.liquid initially. If merchants or developers have reasons to modify the position of the script, we will not reposition it upon re-injection.
If merchants already leverage "view" parameters on certain website links to provide alternate template views to shoppers, we will now exclude those visitors from tests to ensure website functionality operates as originally desired.
We've made launching and ending tests a bit quicker by optimizing our processes for how we interact with themes.
Week of January 14, 2024
We've expanded the functionality of template testing! You can now test different types of templates against each other.
This capability enables merchants to conduct much-requested tests with ease: test homepages versus landing pages, product pages versus landing pages, and more!
Fixed an issue that would prevent referrer segmentation from working in certain instances where the browser could hand Shoplift a referring domain value in an unexpected format.
Week of January 7, 2024
Fixed an issue where reports for certain tests could occasionally become delayed.
Made some adjustments to when our Bayesian analysis runs for tests to improve scalability under large test loads.
Week of December 17, 2023
Our latest feature is here: Audiences! With this release, you can now run tests on all sorts of tightly-specified visitor segments that extend beyond mobile/desktop or new/returning visitor targeting.
Run multiple A/B tests on a single template and target each test to different audiences.
Create audiences by defining groups of website visitors with custom logic and rules, using UTM parameters, query strings, geography, and referring domains.
Leverage one-click segments with our pre-built "common audiences" for popular use cases like Facebook, Instagram, Google, TikTok, and more.
Save, reuse, and modify your audiences over time as you learn what works and what needs improvement.
Week of December 10, 2023
Fixed an issue where the "Test progress" tile in test reports could show a "trending neutral" state even when the trend was positive or negative.
Fix an issue where the "Test duration" tile in test reports could show an extended test runtime in the event that a test was ended from a paused state.
Week of November 26, 2023
We've made a handful of continued speed improvements on our testing script. Our script no longer needs to communicate with Shoplift's servers to run, so your tests should redirect as fast as computer-ly possible, and the whiteout function from our anti-flicker script is now imperceivable.
Fixed a rare issue where on some stores, due to pageload times, the Shoplift Pixel could execute quicker than our testing script, resulting in incorrect clickthrough rates for test variants.
Implemented a handful of fixes and improvements for countdown timers in Lift Assist sections, including proper timezone support.
Implemented a fix for when removing a variant from a test draft while the variant was still being created would result in an infinite loading state until the page was refreshed.
Expanded our time-to-scan window for Brand Styles from 30 seconds to 60 seconds to accommodate stores for which the scan would take particularly long due to various reasons.
Week of November 19, 2023
Screenshots of your store make our app pretty (because your stores are pretty), but sometimes they can take a while to load depending on the store. We've added some additional in-app messaging to inform merchants when this is the case.
Fixed an issue where full-page screenshots for test previews could occasionally drop content lower on the page if a website had certain lazy loading configurations.
Added improved loading states when creating Lift Assist tests to provide more feedback about progress.
Implemented the ability to run tests with a 0/100 traffic allocation.
Added timer-based Lift Assist tests for landing pages.
Week of November 12, 2023
Added a much-requested feature to edit variants on a live test. The variant card on the test report page now has a dropdown with an "Edit in Shopify" option, so merchants can make necessary edits to their active tests to correspond with sales and other external promotions. This allows merchants to ensure that their A version and B version are in sync with required changes that lie outside the scope of what is being tested.
Fixed an issue where occasionally, our screenshot service would return a blank image preview because it did not wait long enough for content to load.
Fixed an issue where if a tested page included a forward slash before the start of query parameters (like /products/red-hat/?utm_source=facebook) it would not be recognized as a valid visitor for tests.
Week of November 5, 2023
Fixed an issue where if a test was paused for long periods of time, our estimated time to significance calculations could present inaccurate dates.
Made an adjustment to account overage handling, wherein if an account goes over their allotted plan limits and tests are paused, upon overage correction the tests no longer auto-resume.
Fixed an issue on the test report page where our UI visualization of paused periods did not account for Daylight Savings Time.
Week of October 29, 2023
Fixed an issue with our Image Hero Lift Assist section where the benefits block was not responding to Mobile/Desktop only settings.
Added additional optimizations to our screenshot processes so that screenshots can be returned in shorter timeframes without dropping content, particularly for newly onboarded stores.
Fixed an issue with form validation on our onboarding contact forms that would throw errors before users had typed anything.
Week of October 22, 2023
Fixed an issue where stores with thousands of products might have to wait an unreasonable amount of time for templates to load.
Fixed an issue where screenshot updates could get hung up or delayed because screenshots were inadvertently being queued more frequently than necessary.
Fixed an issue on the test list page where due to rounding inconsistencies, the Lift reported on the test list page and the test report page would be differ by a few hundredths of a percent.
Fixed an issue with rounding inconsistency on the test report page. All values now round up or down to the nearest hundredth of a percent.
Fixed an issue in our Report CSV export where certain numerical values were formatted incorrectly
Week of October 15, 2023
Added an error handler to our Brand Styles import process to indicate if a failure to import occurred, to enable better troubleshooting of issues.
Fixed an issue where our Brand Styles snackbars for various edit functions would not auto-dismiss.
Fixed an issue in our CSV export of test data wherein conversion rate would include extra decimals.
Fixed an issue in Test Reports wherein the performance over time charts wouldn't display the store currency symbol correctly.
Week of October 8, 2023
New feature! We've released reporting for all major subscription apps, including Recharge, Skio, Loop, Prive, Subify, Yotpo, Ongoing Subcriptions, Ordergroove, Appstle, and Bold! Integration with these apps works out-of-the-box, with no prior setup required. To view subscription reporting for newly launched tests, simply go into your test report and change your view for different device, visitor, or channel tables to "Subscription" metrics.
Fixed an issue where launching the app from the Shopify Admin could take several seconds to load.
Fixed an issue where our "Probability to win" badges in test reports were changing color (indicating success or failure) prior to significance being achieved.
Week of October 1, 2023
New feature! We've released both Clickthrough Rate and Add-to-Cart Rate as available test goals. These new goals are perfect for merchants with lower traffic volumes or merchants who want to conduct tests for engagement-focused KPIs rather than purely revenue-driven strategies.
New feature! We've released a new test segmentation option - new and returning visitors - so you can better understand your shoppers. Segmenting tests by new or returning visitors is a great way to gain insight on how these distinctly different customer cohorts interact and engage with your store. Every test will now report on these visitor segments.
Fixed an issue where our Brand Styles importer would miss fonts on some websites.
Week of September 24, 2023
We've made some great improvements to our tracking script!
Our script now uses a snippet, so it's more concise and easier for developers to work with in their theme.liquid files.
We've also migrated all of our event tracking from our javascript snippet to the Shoplift Web Pixel, which means that all test-related events are now tracked server-side. This makes our javascript snippet faster than ever before, because it now handles just redirects and our anti-flicker solution.
Implemented adjustments to the way we ingest order data so that edits and updates to orders post-purchase (like upsells) are now reflected in test report data.
Adjusted the way our signup flow works so that merchants will now pick a plan best suited for them upon installing the app, rather than defaulting to the Starter plan and needing to manually upgrade shortly thereafter.
Fixed an issue on the Image hero Lift Assist section where the mobile setting "anchor at section bottom" would apply to desktop.
Week of September 17, 2023
Implemented improved coverage on order tracking by implementing a Shoplift Pixel via Shopify's web pixel API, which enables us to properly track and attribute orders made by payment methods which circumvent Shopify checkout.
Fixed an issue where the product template list was timing out for stores with several thousand products.
Fixed an issue where Lift Assist creation would stall out on the first attempt after starting up the app.
Fixed an issue where our pie charts on test reports were being resized inconsistently on different viewports.
Week of September 10, 2023
New feature: Quick Start Guide for new users! Users on trial will now have various cards that help them more effectively navigate various app features when getting started with Shoplift.
Fixed an issue where button styles were not being rendered correctly in the Brand Styles Editor.
Renamed "Landing pages" to "Pages" on template browser screens to be more in-line with Shopify naming conventions.
Added a "fold height" indicator to variant previews on desktop and mobile to indicate where the fold will occur on shoppers' devices.
Updated template preview toggles to enable easier switching between viewing original, variant, or a comparison.
Added an "Ended" tab to the Dashboard so merchants can view recently ended tests.
Fixed an issue where longer test names could cause a break in formatting in test lists.
Week of September 3, 2023
Fixed an issue where merchants with large numbers of collections were not having their template assignments returned correctly in the template browser.
Changed "Certainty" to "Progress" on test reports to provide merchants with a more semantic indicator of test significance, because "certainty" was misleading when it was less than 95%.
Removed "High confidence" as a test status, because merchants were inferring it to mean statistically significant. This has been changed to a "Nearing significance" status.
Ended tests now only have two statuses: Significant or Inconclusive.
Tests that are taking a long time to reach significance now have two new statuses: "Long timeline expected" (if the time to significance is estimated to take between 14 and 60 days, after the test has been running for 14 days) and "Significance unlikely" (if the time to significance is estimated to take greater than 60 days, after the test has been running for 14 days).
Fixed an issue where our line charts were not plotting points for a single day of data.
Fixed an issue where our line charts were not plotting points for the latest day of data.
Week of August 27, 2023
New feature: the new test creation experience is here! We've made a number of improvements to the workflows and UI.
Creating a draft no longer requires going through a lengthy flow. Get to the draft screen from the dashboard in one click, and add templates to your test from there.
Lift Assist is now integrated directly into this draft flow, and the full library of tests can be browsed from the draft screen.
Test drafts are now extremely flexible. If you change your mind about what you'd like to test, you no longer have to create an entirely separate draft. Easily swap out your original or variant templates with a few clicks.
We have improved how Lift Assist tests are organized, which are now categorized by different concepts.
We've greatly improved our time-to-creation for variants by breaking up various processes (variants are now typically created in between 1-2 seconds).
We've made improvements to our screenshotting methods to ensure that screenshots for your original and variant templates are always up-to-date on your drafted tests.
Week of August 13, 2023
New feature: we now support templates created with third-party pagebuilder apps like Replo, Shogun, Pagefly, Gempages, and more!
Third-party templates can be located in the template browser and added to tests like any other template in your theme.
We no longer limit the templates merchants can test on OS1 themes to liquid templates that we can programmatically convert to JSON. OS1 merchants can now test any liquid template on their theme.
We've updated how we generate filename suffixes for templates created with Shoplift to be more human-readable and more easily identified as a Shoplift template.
Fixed an issue where merchants could technically have two tests for the same template live at the same time by inadvertently exploiting the pause feature. Now, if a test is active on a template, and a test for the same template is paused, merchants cannot resume the paused test until the currently active test is paused or ended.
Fixed an issue where test duration would calculate incorrectly when paused periods occur.
Total test duration is now reported to the hour, rather than to the day.
We've hidden the percentage change badge from variant revenue in reporting, as this metric is affected by traffic weighting to each variant and did not make sense to provide an improvement percentage for.
Development stores can now install the app without a valid payment method and will be presented with a test charge. This enables merchants and agencies to take Shoplift for a spin on development environments for their production stores.
Week of July 30, 2023
New feature: you can now pause and resume tests!
Tests that are paused will continue to be organized under the "Active" test tab, but will have their significance badge replaced with a "Paused" badge.
Merchants who have gone over their plan limits will now have their tests automatically paused and resumed when plan limits are corrected.
We've added detailed variant session information to the pie chart tooltips on the reports pages.
Week of July 23, 2023
We've adjusted the way we handle testing existing templates from your theme. Previously, Shoplift would always create a duplicate of the template selected from the theme to prevent users from editing live templates with changes intended to be isolated to a test. This introduced certain issues for users with theme code that was dependent on the suffix of the original template selected from the theme. Now, when selecting a template from your theme to test as a variant, we do not duplicate the template, but rather load in the template selected as a variant to be tested against the selected original. This behavior is more inline with what users would expect.
Fixed a bug on the "Create your variant" step of test creation wherein the image containers would always show the empty state, regardless of whether a screenshot was available.
Fixed an issue on the Homepage tab of variant selection for OS1 stores, wherein for newly installed stores the tab might present OS1 homepages as incompatible.
Week of July 16, 2023
Removed the requirement for screenshots to be taken inline with draft creation, which was leading to error screens for a small percentage of users in the event that the screenshot service timed out on their theme/store.
Added screenshot empty states to the Variant selection step in the Create Test flow, which previously would return broken image links.
Fixed an issue with the Image hero section where mobile buttons were always set to "wide".
Fixed an issue where Multi-message bar and Timer bar would use the desktop size for text when set to "Large" on mobile.
Updated date labels on chart tooltips on the Test Report page to provide day of week and year.
Updated date labels on charts on the Test Report page to provide year.
Week of July 9, 2023
Fixed an issue where the full-page preview on the draft screen would not update until manually refreshing the page after creating a test draft.
Fixed an issue where Estimated Time to Significance in Test Reporting could provide very low "day" values for tests with small sample sizes.
Fixed an issue where a dialogue banner indicating that your template has changed might erroneously populate.
Fixed an issue when a Test was ended, the Report page would not update with the End Date until refreshing the page.
Fixed an issue where the values on the Test Report tables would round down to the nearest hundredth of a decimal point. Values now properly round up or down based on the thousandth decimal point.
Fixed an issue where the dropdown on the Drafts list said "View report" instead of "View draft".
Fixed an issue with Single Message Bar, Multi-Message Bar, and Timed Message Bar wherein they would not properly hide/show on devices according to the "Show on mobile only" setting.
Fixed an issue with Single Message Bar wherein "Mobile text size" would not update correctly.
Last updated