Mutual Exclusion

Shoplift supports running multiple experiments simultaneously. In certain cases, when tests have the potential to interfere with each other — for example, by targeting the same page — we apply mutual exclusion logic to preserve the integrity of your results.

What is Mutual Exclusion?

Mutual exclusion ensures that a visitor is only enrolled in one of several potentially conflicting tests. This prevents interaction effects and attribution errors that can occur when a single visitor is exposed to overlapping experiments.

When Mutual Exclusion Is Enforced

Shoplift automatically enforces mutual exclusion in the following scenarios:

• Multiple template tests targeting the same template: If more than one test targets the same template, Shoplift ensures that each visitor is enrolled in only one of those tests.

• Theme tests relative to all other test types: Theme tests are mutually exclusive with all other tests (URL, API, and template), including other theme tests. Visitors in a theme test will not be included in other experiments.

• URL tests targeting the same page: When multiple URL tests affect the same URL, visitors are assigned to only one test to prevent overlap.

• Template and URL tests on the same page If both a template and a URL test affect the same page, Shoplift assigns the visitor to one test only.

• Automatic API tests relative to theme and template tests Automatic API-based experiments are excluded from running alongside theme or template tests.

In cases where mutual exclusion is enforced, visitors who match the participation conditions of multiple tests will be allocated equally to each test.

When Mutual Exclusion Is Not Enforced

In scenarios where tests are unlikely to interfere with each other in a material way, Shoplift allows concurrent participation:

• URL tests targeting different pages These can run simultaneously without conflict.

• Template tests on different templates Since the templates do not overlap, mutual exclusion is not necessary.

• Manual API tests, except when a theme test is active Manual API tests may run concurrently with most other test types.

• URL and API tests on the same page These are not considered conflicting and may run in parallel.

In instances where Shoplift proactively prevents tests with the potential for overlapping visitor segments from cross-contaminating datasets, it is important to know that your tests may reach statistical significance more slowly because the total potential traffic of each test is reduced, because it must be divided between each test.

Why doesn't Shoplift mark same-funnel tests as conflicting?

Shoplift doesn’t automatically flag tests in the same funnel or buyer journey as “conflicting” because sound statistical practices ensure each test’s results remain valid even when experiments overlap. In essence, robust sample sizes and significance thresholds act as a safeguard, so simultaneous tests don’t skew each other’s outcomes.

• Even distribution neutralizes interactions: With proper A/B test design, traffic is randomly split for each experiment. When multiple tests run along one funnel, all combinations of variations occur across users, meaning any interaction effect is spread evenly rather than biasing one group. This randomization ensures no test gets a systematic advantage or disadvantage from another – potential influences cancel out with enough sample size.

• Statistical significance filters out noise: Shoplift requires rigorous criteria (95% confidence and sufficient sample size) before declaring a winner. This high bar controls for false positives and treats minor cross-test effects as statistical noise. In practice, small overlaps won’t trigger a win by chance – only truly significant improvements shine through. The result is that overlapping tests act independently, with the 95% significance threshold effectively washing out any random interaction noise.

• Proven in high-scale testing: Industry experts and data scientists agree that overlap concerns are often overstated. In fact, companies like Facebook safely run thousands of simultaneous experiments (even on the same user journey) by relying on strong analytics to isolate each effect. With large sample sizes and proper stats, concurrent funnel tests rarely conflict in any meaningful way.

This data-driven confidence means you can run more tests in parallel to speed up optimization, knowing that the platform’s statistical safeguards will keep your results trustworthy and free from cross-test contamination.