Mutual Exclusion

Unlike other platforms which require manually creating and managing exclusion rules between concurrent tests, Shoplift handles mutual exclusion out of the box. This means that you can run multiple tests simultaneously with confidence, as any tests which might have significant interaction effects are mutually exclusive by default.

What is Mutual Exclusion?

Mutual exclusion ensures that a visitor is only enrolled in one of several potentially conflicting tests, or tests which might have significant interaction effects, thereby compromising data integrity.

Mutual exclusion maintains data integrity across tests running simultaneously to prevent interaction effects and attribution errors that can occur when a single visitor is exposed to multiple experiments.

How Mutual Exclusion is Enforced

Shoplift leverages a sophisticated logic layer to ensure tests which might have significant interaction effects are mutually exclusive. This logic layer enforces a variety of exclusion rules depending on which tests you run simultaneously.

When a visitor lands on your website, takes specific actions, or views specific pages that would enroll them in a test, Shoplift will randomly assign them to a specific test in which they can participate.

When Mutual Exclusion is Enforced

Tests are evaluated for mutual exclusion based on their entry criteria, which occur either globally or conditionally.

Global evaluation occurs when one or more tests are active wherein visitors can be assigned to the test on any page, like Theme tests, Price tests, or Automatic API tests.

Conditional evaluation occurs when one or more tests are active wherein visitors are assigned to the test on specific pages or after taking specific actions, like template tests, URL tests, and Manual API tests.

In cases where a global test and a conditional test are mutually exclusive, visitors are evaluated for test assignment upon site entry. Visitors assigned to tests with global entry criteria will participate in the test immediately. Visitors assigned to tests with conditional entry criteria will have a "reservation" created. This means that they will be excluded from the global test and will only be included in the conditional test when the entry conditions are met.

Example: Theme Test and Price Test Exclusion

Let's say you are running a theme test and a price test simultaneously. Visitors can participate in theme tests upon entering the site on any page, while visitors can only participate in price tests once they've scrolled a tested price into view.

In this case, when a website visitor lands on the site, evaluation for test assignment will happen immediately. If the visitor is assigned to the theme test, they will be immediately included in the theme test, and excluded from the price test. If the visitor is assigned to the price test, a "reservation" will be made, and the visitor will be included in the price test only when they encounter a tested price on their journey.

Test Type	Assignment	Participation	Entry Criteria
Theme	Global	Immediate	Any page load
Automatic API	Global	Immediate	Any page load
Manual API	Conditional	Deferred	Any defined criteria
Template	Conditional	Deferred	Specific pages
URL	Conditional	Deferred	Specific page
Price	Conditional	Deferred	When price is viewed

How many tests can I run at the same time?

There is no limit to the amount of tests that you can run at the same time. However, as you increase the amount of tests you are running simultaneously, if the tests are mutually exclusive, the total potential sample size of each test diminishes, since the sample is distributed across exlusive tests. This means that statistical significance may take longer to reach.

Why doesn't Shoplift mark same-funnel tests as conflicting?

Shoplift doesn’t automatically flag tests in the same funnel or buyer journey as “conflicting” because sound statistical practices ensure each test’s results remain valid even when experiments overlap. In essence, robust sample sizes and significance thresholds act as a safeguard, so simultaneous tests don’t skew each other’s outcomes.

• Even distribution neutralizes interactions: With proper A/B test design, traffic is randomly split for each experiment. When multiple tests run along one funnel, all combinations of variations occur across users, meaning any interaction effect is spread evenly rather than biasing one group. This randomization ensures no test gets a systematic advantage or disadvantage from another – potential influences cancel out with enough sample size.

• Statistical significance filters out noise: Shoplift requires rigorous criteria (95% confidence and sufficient sample size) before declaring a winner. This high bar controls for false positives and treats minor cross-test effects as statistical noise. In practice, small overlaps won’t trigger a win by chance – only truly significant improvements shine through. The result is that overlapping tests act independently, with the 95% significance threshold effectively washing out any random interaction noise.

• Proven in high-scale testing: Industry experts and data scientists agree that overlap concerns are often overstated. In fact, companies like Facebook safely run thousands of simultaneous experiments (even on the same user journey) by relying on strong analytics to isolate each effect. With large sample sizes and proper stats, concurrent funnel tests rarely conflict in any meaningful way.

This data-driven confidence means you can run more tests in parallel to speed up optimization, knowing that the platform’s statistical safeguards will keep your results trustworthy and free from cross-test contamination.