API Guides

Guides

Recipes for automations built on the Shoplift API. Each guide takes you from an empty file to a working workflow, and each one is a starting point: adjust the thresholds, schedules, and destinations to fit your program.

Automated test alerts in Slack

Tests fail quietly. A traffic split drifts, volume dries up, or a variant starts losing, and nobody notices until the weekly review. This guide builds a monitor that checks every active test on a schedule and posts to Slack only when something needs attention.

What you'll build

A scheduled script that, every hour:

1. Lists your tests via GET /tests and filters to active status.

2. Pulls live stats for each via GET /tests/:id/results.

3. Checks each test against alert conditions.

4. Posts an alert to a Slack channel via an incoming webhook when a condition trips.

What to alert on

Conditions worth starting with, all available in the results response:

• Split drift: controlSplit and variantSplit should stay near your configured traffic split. A persistent gap suggests a sample ratio problem worth investigating.

• Low volume: totalSessions growing slower than expected means the test will take far longer to conclude than planned.

• A variant trending badly: a low variantProbabilityWin sustained over days may justify ending the test early rather than continuing to send traffic to a loser.

• Stalled progress: progress tells you how far through the required sample the test is. A test at 5% progress after three weeks is a scheduling conversation, not a statistics one.

Set thresholds conservatively at first. An alert channel that cries wolf gets muted within a week.

Build it with Claude

You don't need to write this script yourself. With your API key and a Slack incoming webhook URL available, give Claude Code a prompt like:

Using the Shoplift API reference at docs.shoplift.ai, write a script that runs
hourly, pulls all active tests and their results, and posts to my Slack webhook
when any test has a traffic split more than 5 points off its configuration,
fewer than 100 new sessions in the last 24 hours, or a variant win probability
below 20% for a test past 30% progress. Include the test title and a link to it
in each alert. My API key is in .env.

Claude will read the reference and write the script, and you schedule it with whatever you already use for cron jobs or scheduled workflows. Ask Claude for setup instructions for your environment if you aren't sure.

Extend it

• Daily digest instead of alerts. Post one summary each morning covering every active test, alert or not.

• Auto-pause on failure. For hard failures, go beyond alerting: call POST /tests/:id/pause and note the pause in the Slack message. Paused visitors see the original experience, so this is a safe reflex for a badly broken test.

Rate limit note

The API allows 60 requests per minute. Monitoring N active tests costs roughly N+1 requests per run: one list call, plus one results call each. Under about 50 active tests, an hourly run fits comfortably in a single minute's budget. Above that, pace the results calls.

Scheduled post-test reports

A reporting engine that detects when a test crosses a milestone (7 days live, 14 days live, or ended) and delivers a written summary to Slack or email.

Structure:

1. Poll GET /tests for status and startAt/endAt transitions.

2. Pull GET /tests/:id/results.

3. Have Claude turn the payload into a plain-language summary covering lift, win probability, sample progress, and a recommendation.

Two output formats are worth building: a technical version for the testing team, and a client-ready version for stakeholders.

Automated testing roadmap report

A weekly report on the state of your testing program, built entirely from the API. Pull all tests, then surface:

1. Completed tests with a winner where implemented is No, your unrealized wins.

2. The draft queue, ordered by priority and impact-to-effort ratio.

3. Anything active but stalled.

Output it as a Slack post or a document.

For what the priority, impact, effort, and implemented fields mean and how to maintain them, see test planning. This guide only reads them.

Every guide in this section requires the API. Anything that's really about managing tests, such as what the fields mean, when to set them, or dashboard workflows, lives in the test management documentation instead.

Still need help?

Contact the Shoplift support team if you have questions about building on the Shoplift API.