Skip to main content

Offer A/B testing

Prosper planEnterprise plan

Offer A/B testing lets you compare incentives or messaging under controlled conditions. Use a control and one or more variants to test a specific hypothesis about what customers will accept and which intervention produces better retention outcomes.

A/B test dashboard

ProsperStack assigns traffic to the variants and compares their acceptance rates. Consider acceptance alongside retained value and longer-term performance when deciding which offer to keep and what to test next.

Creating an A/B test​

To create an offer A/B test, you need at least two offers. Navigate to Offers and select A/B tests, then click Create A/B test.

Creating an A/B test

Enter a unique Name, select an offer as the Control and select another offer as Variant 1. You can also add an optional Variant 2. Click Save.

Sticky assignment​

By default, an A/B test uses sticky assignment: once a subscription is assigned a variant, it will keep seeing that same variant every time it enters the test. This keeps the experience consistent for a given customer across multiple cancellation attempts.

If you'd rather have each cancellation attempt independently assigned a variant at random — rather than sticking to whatever the subscription saw previously — uncheck Sticky assignment when creating or editing the test.

Variant weights​

Enterprise plan

A/B test variant weights

Weights control how likely a customer is to see a variant in an A/B test. For example, you can use weights to roll out a new offer to a small percentage of customers to validate effectiveness before committing to a full launch.

Using an A/B test in a cancellation flow​

Use an A/B test in the flow

To activate an A/B test, select the test in the offer rules editor in a cancellation flow. When the rule containing an A/B test is matched, the subscriber will be presented an offer from the A/B test, chosen according to the variant weights and the test's assignment setting. ProsperStack will track the results and update your A/B test metrics in real-time.

Ending an A/B test​

Ending an A/B test

When your A/B test has collected enough data to determine a winning offer, you can end the test by clicking the End test button. If a statistically significant winner can be determined automatically, it will be pre-selected. Confirm the winning offer to end the test.

Once a test is complete, only the winning offer will be presented in any cancellation flows using the A/B test.