Docs
  • INTRO
    • Welcome to Shoplift
    • What to Test
    • Quickstart
      • Install Shoplift
      • Create a Test
      • Edit Your Variant
      • Launch Your Test
  • TEST
    • Template Testing
      • Overview
      • Guides
        • Guide: How to Use Shopify Magic to Generate New Theme Blocks for Testing
        • Guide: Testing Individual Pages
        • Guide: Testing One-Click Payment Methods
        • Guide: Testing App Blocks
        • Guide: Testing Different Template Types
      • Theme Compatibility
      • Template Management
      • Switching Themes
      • Shopify Markets
      • Shopify Translate & Adapt
    • Theme Testing
      • Overview
      • Guides
        • Guide: Testing a Navigation Menu
        • Guide: Testing a Mini Cart
      • Theme Management
      • Theme Development
      • Switching Themes
    • URL Testing
      • Overview
      • Guides
        • Guide: Testing URLs
        • Guide: Testing Product Properties
    • JavaScript API Testing
      • isHypothesisActive
      • setAnalyticsConsent
      • getVisitorData
    • Lift Assist™
      • What is Lift Assist™?
      • How To Use Lift Assist™
      • Brand Styles
    • Audience Targeting
      • Devices
      • New and Returning Visitors
      • Custom Audiences (UTMs, Referring Domains, and Geo-targeting)
    • Test Management
      • Test Compatibility
      • Mutual Exclusion
      • QA Checklist
      • Scheduling Tests
      • Pausing Tests
      • Implementing Winning Tests
    • Performance
      • Pagespeed
      • Shopify Caching
      • CDNs and Edge Delivery
  • ANALYZE
    • Reports
      • Overview
      • Metrics
        • Goals & Reporting Metrics
        • Metric Views
      • Test Progress
      • Statistical Significance
      • Channel Groups
    • Tracking
      • Visitor Attribution
      • Web Pixel
      • Liquid Snippet
      • Anti-Flicker
      • Analytics Discrepancies
  • integrate
    • Integrations and Tooling
      • GA4 (Beta)
        • Integrate GA4 (Shopify)
        • Integrate GA4 with Google Tag Manager
        • Integrate GA4 with Elevar
        • Analyze Test Data In Google Analytics
        • Disabling the GA4 Integration
      • Subscription Apps
      • Page Builder Apps
      • Reporting Tools
      • Heatmapping Tools
      • Custom Integrations
  • ADMIN
    • Plans and Billing
      • Subscription Plans
      • Uninstalling Shoplift
    • Users and Permissions
      • Managing Team Access
      • Managing User Notifications
    • Privacy and Compliance
      • Customer Data and GDPR
  • PARTNERS & DEVELOPERS
    • Shoplift Partner Program
    • Testing on Development Stores
  • SUPPORT
    • Frequently Asked Questions
    • Get Help
    • Give Feedback
Powered by GitBook

Contact Us

  • Get help
  • Give feedback
  • Request a feature

Helpful Links

  • Shoplift
  • Shopify App Store
On this page
  • What is Mutual Exclusion?
  • When Mutual Exclusion Is Enforced
  • When Mutual Exclusion Is Not Enforced
  • Why doesn't Shoplift mark same-funnel tests as conflicting?

Was this helpful?

  1. TEST
  2. Test Management

Mutual Exclusion

Shoplift supports running multiple experiments simultaneously. In certain cases, when tests have the potential to interfere with each other — for example, by targeting the same page — we apply mutual exclusion logic to preserve the integrity of your results.

What is Mutual Exclusion?

Mutual exclusion ensures that a visitor is only enrolled in one of several potentially conflicting tests. This prevents interaction effects and attribution errors that can occur when a single visitor is exposed to overlapping experiments.

When Mutual Exclusion Is Enforced

Shoplift automatically enforces mutual exclusion in the following scenarios:

  • Multiple template tests targeting the same template: If more than one test targets the same template, Shoplift ensures that each visitor is enrolled in only one of those tests.

  • Theme tests relative to all other test types: Theme tests are mutually exclusive with all other tests (URL, API, and template), including other theme tests. Visitors in a theme test will not be included in other experiments.

  • URL tests targeting the same page: When multiple URL tests affect the same URL, visitors are assigned to only one test to prevent overlap.

  • Template and URL tests on the same page If both a template and a URL test affect the same page, Shoplift assigns the visitor to one test only.

  • Automatic API tests relative to theme and template tests Automatic API-based experiments are excluded from running alongside theme or template tests.

In cases where mutual exclusion is enforced, visitors who match the participation conditions of multiple tests will be allocated equally to each test.

When Mutual Exclusion Is Not Enforced

In scenarios where tests are unlikely to interfere with each other in a material way, Shoplift allows concurrent participation:

  • URL tests targeting different pages These can run simultaneously without conflict.

  • Template tests on different templates Since the templates do not overlap, mutual exclusion is not necessary.

  • Manual API tests, except when a theme test is active Manual API tests may run concurrently with most other test types.

  • URL and API tests on the same page These are not considered conflicting and may run in parallel.

In instances where Shoplift proactively prevents tests with the potential for overlapping visitor segments from cross-contaminating datasets, it is important to know that your tests may reach statistical significance more slowly because the total potential traffic of each test is reduced, because it must be divided between each test.

Why doesn't Shoplift mark same-funnel tests as conflicting?

Shoplift doesn’t automatically flag tests in the same funnel or buyer journey as “conflicting” because sound statistical practices ensure each test’s results remain valid even when experiments overlap. In essence, robust sample sizes and significance thresholds act as a safeguard, so simultaneous tests don’t skew each other’s outcomes.

  • Even distribution neutralizes interactions: With proper A/B test design, traffic is randomly split for each experiment. When multiple tests run along one funnel, all combinations of variations occur across users, meaning any interaction effect is spread evenly rather than biasing one group. This randomization ensures no test gets a systematic advantage or disadvantage from another – potential influences cancel out with enough sample size.

  • Statistical significance filters out noise: Shoplift requires rigorous criteria (95% confidence and sufficient sample size) before declaring a winner. This high bar controls for false positives and treats minor cross-test effects as statistical noise. In practice, small overlaps won’t trigger a win by chance – only truly significant improvements shine through. The result is that overlapping tests act independently, with the 95% significance threshold effectively washing out any random interaction noise.

  • Proven in high-scale testing: Industry experts and data scientists agree that overlap concerns are often overstated. In fact, companies like Facebook safely run thousands of simultaneous experiments (even on the same user journey) by relying on strong analytics to isolate each effect. With large sample sizes and proper stats, concurrent funnel tests rarely conflict in any meaningful way.

This data-driven confidence means you can run more tests in parallel to speed up optimization, knowing that the platform’s statistical safeguards will keep your results trustworthy and free from cross-test contamination.

PreviousTest CompatibilityNextQA Checklist

Last updated 1 day ago

Was this helpful?