Flow A/B testing

Prosper planEnterprise plan

Flow A/B testing lets you run controlled experiments on complete cancellation experiences. Compare a control with one or more variants to determine whether changes to questions, messaging, logic or interventions improve retention performance.

ProsperStack assigns traffic to the variants and compares their retention rates, giving you evidence to decide which experience to keep and what to test next.

Creating an A/B test

To create a flow A/B test, you need at least two cancellation flows. Navigate to Flows and select A/B tests, then click Create A/B test.

Enter a unique Name, select a flow as the Control and select another flow as Variant 1. You can also add an optional Variant 2. Click Save.

Assignment

By default, an A/B test uses sticky assignment: once a subscription is assigned a variant, it will keep seeing that same cancellation flow every time it enters the test. This keeps the experience consistent for a returning customer and avoids skewing results if the same subscription starts a cancellation session more than once during the test.

If you'd rather have each cancellation attempt assigned a flow independently at random, uncheck Sticky assignment when creating or editing the test.

Variant weights

Enterprise plan

Weights control how likely a customer is to see a variant in an A/B test. For example, you can use weights to roll out a flow variant to a small percentage of customers to validate effectiveness before committing to a full launch.

Routing cancellations to an A/B test

To activate an A/B test, select the test in the flow routes editor. When the rule containing an A/B test is matched, the subscriber will be routed to a cancellation flow chosen from the A/B test in proportion to the variant weights. With sticky assignment enabled, a subscription is routed to the same flow on every subsequent match; otherwise a flow is chosen at random each time. ProsperStack will track the results and update your A/B test metrics in real-time.

Ending an A/B test

When your A/B test has collected enough data to determine a winning cancellation flow, you can end the test by clicking the End test button. If a statistically significant winner can be determined automatically, it will be pre-selected. Confirm the winning flow to end the test.

Once a test is complete, the winning flow will be selected for any sessions that are routed to the A/B test.