Use Cases

This guide covers the most common scenarios where an API test is the right choice, along with setup tips for each.

When to use an API test

API tests are the right choice when you need custom JavaScript to implement the change you want to test. The key question: can this change be made in Shopify's Theme Editor, or does it require code?

• Requires custom JavaScript → API test

• Can be done in the Theme Editor → template test or theme test

Notification bars and banners

Test whether adding a site-wide notification bar or promotional banner increases engagement or revenue.

Example: You want to test a sticky banner at the top of every page announcing free shipping on orders over $50. The banner needs to appear on all pages, dismiss when clicked, and not reappear for that visitor during the session. An automatic API test adds the banner via JavaScript for variant visitors while control visitors see the store without it.

Test ideas:

• Free shipping threshold banner vs. no banner

• Urgency messaging ("Sale ends tonight!") vs. value messaging ("Save 20% sitewide")

• Sticky banner that scrolls with the visitor vs. static banner at the top

• Banner with a countdown timer vs. static text

Setup tip: Use an automatic API test for site-wide banners. Since the banner should appear on every page, automatic execution is the simplest approach. Make sure your script checks whether the banner element already exists before adding it, so it doesn't duplicate on AJAX page navigations.

Popup and modal tests

Test whether a popup or modal improves email signups, reduces bounce rate, or drives more purchases.

Example: You want to test an exit-intent popup offering 10% off a visitor's first order in exchange for their email address. The popup should only trigger when the visitor moves their cursor toward the browser's close button. A manual API test gives you control over exactly when the popup appears, since the trigger logic depends on visitor behavior.

Test ideas:

• Exit-intent popup vs. no popup

• Discount popup (10% off) vs. free shipping popup

• Popup after 30 seconds on site vs. popup on exit intent

• Full-screen takeover vs. small corner popup

Setup tip: Use a manual API test for behavior-triggered popups. Your developer writes the popup code as the variant script and implements the trigger logic (exit intent, scroll depth, time delay) in the theme code using isHypothesisActive().

Third-party script variations

Test different configurations of third-party tools to find what drives the best results.

Example: You use a product recommendation widget on your product pages, and you want to test whether showing "Customers also bought" recommendations performs better than "You may also like" recommendations. An automatic API test can swap the recommendation config for variant visitors.

Test ideas:

• "Customers also bought" vs. "You may also like" recommendation logic

• Chat widget on all pages vs. chat widget only on product pages

• Reviews displayed as a carousel vs. reviews displayed as a list

• Different social proof notification formats or frequencies

Setup tip: Automatic API tests work well when you're changing a script configuration that should apply globally. If the third-party script only runs on specific pages, consider a manual API test so you can trigger only where needed.

Dynamic UI modifications

Test JavaScript-driven changes to existing page elements without rebuilding templates.

Example: You want to test whether adding a "Bestseller" badge to your top 10 products increases their conversion rate. An automatic API test can inject badge elements onto the relevant product cards across collection and search results pages.

Test ideas:

• "Bestseller" or "Staff Pick" badges on top products

• Sticky add-to-cart button that follows the visitor as they scroll

• "X people are viewing this" live visitor counter on product pages

• Auto-applied discount code that shows in a banner when items are added to cart

Setup tip: When modifying existing page elements, test your script carefully against the live theme. DOM selectors can break if your theme updates or if other apps modify the same elements. Use preview mode to verify the script works across page types and devices.

Conditional and behavior-triggered tests

Test experiences that activate based on specific visitor actions or conditions.

Example: You want to test whether showing a "Still deciding?" message with a discount code after a visitor has spent 60 seconds on a product page improves conversion. A manual API test lets your developer trigger the message only after the time condition is met.

Test ideas:

• Discount offer after spending 60 seconds on a product page

• "You've viewed this 3 times" message on return product page visits

• Scroll-triggered content reveal (show social proof after scrolling 50% down the page)

• Upsell prompt when a visitor adds a specific product to their cart

Setup tip: Manual API tests are built for this. Your developer uses isHypothesisActive() at the moment the condition is met, then runs the variant code only for assigned visitors. This keeps the test clean — visitors who never meet the trigger condition aren't counted.

Complex integration tests

Test experiences that depend on data from external systems.

Example: You use a customer data platform (CDP) that segments visitors into loyalty tiers. You want to test whether showing a personalized loyalty message ("Welcome back, Gold member! Here's your exclusive offer") improves repeat purchase rate. A manual API test can check the CDP for the visitor's tier, then trigger the variant experience only for Gold-tier visitors.

Test ideas:

• Personalized loyalty messaging based on customer tier

• Different product recommendations based on browsing history from your CDP

• Localized promotional content based on IP-derived location data from a geolocation API

• Dynamic pricing or offer display based on data from your marketing automation platform

Setup tip: Manual API tests are the right choice here. Your developer writes the integration logic to fetch data from the external system, then calls isHypothesisActive() to determine which experience to show. Make sure the external data fetch doesn't significantly slow page load.