Offer A/B testing

Prosper planEnterprise plan

Offer A/B testing lets you compare incentives or messaging under controlled conditions. Use a control and one or more variants to test a specific hypothesis about what customers will accept and which intervention produces better retention outcomes.

ProsperStack assigns traffic to the variants and compares their acceptance rates. Consider acceptance alongside retained value and longer-term performance when deciding which offer to keep and what to test next.

Creating an A/B test

To create an offer A/B test, you need at least two offers. Navigate to Offers and select A/B tests, then click Create A/B test.

Enter a unique Name, select an offer as the Control and select another offer as Variant 1. You can also add an optional Variant 2. Click Save.

Variant weights

Enterprise plan

Weights control how likely a customer is to see a variant in an A/B test. For example, you can use weights to roll out a new offer to a small percentage of customers to validate effectiveness before committing to a full launch.

Using an A/B test in a cancellation flow

To activate an A/B test, select the test in the offer rules editor in a cancellation flow. When the rule containing an A/B test is matched, the subscriber will be presented a random offer from the A/B test. ProsperStack will track the results and update your A/B test metrics in real-time.

Ending an A/B test

When your A/B test has collected enough data to determine a winning offer, you can end the test by clicking the End test button. If a statistically significant winner can be determined automatically, it will be pre-selected. Confirm the winning offer to end the test.

Once a test is complete, only the winning offer will be presented in any cancellation flows using the A/B test.