Reversioning

A reversion marks a point in time where your experiment effectively restarts. After a reversion, results are measured from that date forward. Earlier data is preserved in the underlying record but no longer rolls up into the default dashboard view.

Reversioning exists because some changes to a template invalidate the comparison you’ve been running. Continuing to mix pre-change and post-change data would produce misleading metrics.

What a Reversion Does

When a reversion is recorded against a template, two things happen:

1. The effective start date moves forward. The results dashboard, ranking algorithms, and statistical significance calculations default to using the most recent reversion date as the experiment’s start date. Older data isn’t deleted; it’s just no longer the default window.
2. Users are reshuffled. If the template uses persistent user bucketing, a new salt is generated. The same user may now land in a different bucket than before, which is what makes the post-reversion experiment a true fresh start.

Note

Historical assignments aren’t rewritten retroactively. The reshuffle applies to new requests going forward. Old sends keep the variant they were originally served.

When Reversions Happen

Reversions occur in two situations.

Automatically When You Ship a Winner

When you ship a variant via Milestones & Shipping, the winning variant becomes the new control and a reversion is recorded automatically. This is the expected path: you’ve concluded one round of experimentation and are starting the next one against a new baseline.

Manually When You Change the AB Split

If you edit a template’s AB split configuration while persistent bucketing is enabled (switching split methods, changing the control/treatment ratio, or toggling persistence on), JustAI will detect the change on save and prompt you with a dialog:

⚠️ AB Split Configuration Change Detected

Changing the bucketing method will reassign users to new buckets. This effectively starts a new experiment and should be logged as a reversion.

You’ll see three options:

Option	What it does
Save & Reversion	Saves the new split configuration and records a reversion event. The results dashboard will anchor to today’s date going forward.
Save Without Reversion	Saves the configuration change but does not record a reversion. Pre- and post-change data will be combined in the dashboard.
Cancel	Discards the change.

Tip

We recommend reversioning whenever you change user splits. Changing the split ratio or bucketing method mid-experiment shifts which users see which variants, and mixing the pre-change and post-change populations can bias your results: the same variant is effectively being measured against two different audiences. A reversion gives you a clean baseline from the change forward.

When to Choose Each Option

Choose Save & Reversion when:

• You’re materially changing how the experiment runs (split ratio, bucketing method, unit of diversion).
• You want a clean break between the old configuration and the new one in your metrics.
• The previous comparison is no longer apples-to-apples.

Choose Save Without Reversion when:

• The configuration change is minor or corrective (e.g. fixing a misconfigured salt that hadn’t taken effect yet).
• You want continuity in the dashboard and accept that the metrics blend two slightly different setups.

Related Resources

• Configuration — AB split methods and persistent bucketing
• Milestones & Shipping — How shipping a winner creates an automatic reversion
• Reading Results — How the default date range is selected on the dashboard