Configuration

The Configure tab on a template controls how traffic is split between variants, what thresholds determine a winner, and how the template connects to your ESP. Defaults are inherited from your org settings but can be overridden per template.

AB Split Configuration

AB split determines how users are divided between your control group and experiment variants. Navigate to Configure > Integration to set this up.

Split Methods

JustAI supports three split methods:

Internal Random (Recommended)

JustAI handles the split internally. Each API request is randomly assigned to either the control group or a variant based on the configured split ratio (e.g. 80/20 experiment/control).

• Simplest to set up — no changes needed in your ESP beyond the API call
• Supports comprehensive metrics comparison within JustAI
• Enables continuous experimentation with a holdout group

Internal Persistent

Same as internal random, but users are bucketed consistently based on their user ID. A given user always sees the same variant across requests.

• Recommended for most use cases, especially multi-touch journeys
• Prevents the same user from seeing different variants across sessions
• Required for Milestones (Ship & Iterate)

Tip

Enable Persistent User Bucketing unless you have a specific reason not to. It produces cleaner experiment results and is required for shipping workflows.

External

Traffic splitting happens in your ESP rather than in JustAI. Your ESP sends different user groups to different JustAI template endpoints.

• Use only if you have a technical requirement for ESP-side splitting
• Limits some JustAI analytics features (no unified control/variant comparison)
• Requires additional ESP configuration

Caution

External splitting is a legacy option. New templates should use internal splitting whenever possible.

Control Split Ratio

The control split ratio determines what percentage of traffic sees the original (control) copy vs. experiment variants. Common configurations:

Split	Control	Experiment	Use Case
90/10	10%	90%	Default — maximize learning speed
80/20	20%	80%	More conservative holdout
50/50	50%	50%	Traditional A/B test

The experiment portion is further divided among active variants by JustAI’s ranking algorithms, which shift traffic toward better-performing variants over time.

Control Variant

The control variant is the baseline copy that your experiment measures against. It’s typically the content currently live in your ESP. When you create a template, the control is set during the setup process.

You can update the control at any time from the Configure tab. When you ship a winner using Ship & Iterate, the winning variant automatically becomes the new control.

Statistical Significance Thresholds

These thresholds determine when JustAI considers a variant a statistically confident winner.

Setting	Description	Default
P-value threshold	The probability threshold below which a result is considered significant. Lower values require stronger evidence.	0.05
Minimum sample size	The minimum number of sends a variant needs before it’s eligible for significance testing.	1,000

These defaults are set at the org level in Settings > General and can be overridden per template. Raising the minimum sample size reduces the risk of false positives but increases the time to reach a conclusion.

Note

A variant must pass both the p-value threshold and the minimum sample size before JustAI marks it as a statistically significant winner.

Integration Settings

Integration settings connect your template to the specific touchpoint in your ESP. These are configured during template creation but can be updated here.

Settings vary by ESP but typically include:

• ESP selection — Which integration to use (Braze, Iterable, Customer.io, etc.)
• Touchpoint mapping — The specific campaign, canvas, or journey step this template serves
• Channel — Email, push, SMS, or other channels supported by your ESP

Follow the inline instructions in the Configure tab for ESP-specific setup steps.

SLA Alerts

SLA alerts notify you when a template’s performance falls outside expected bounds. Configure alerts to catch issues like:

• Significant drops in your key metric compared to the control
• Unexpected changes in traffic volume
• Variants that haven’t received traffic within a configured window

Alert thresholds are set at the org level and can be customized per template.

Related Resources

• Setup Guide — Initial template creation and ESP connection
• Org Settings — Configure organization-level defaults
• Ranking Algorithms — How JustAI allocates traffic across variants