Combine results
Evaluate the results of several experiments together, for example a repeated test.
Combine the results of the experiment you have open (the base experiment) with other experiments of the same project, for example when you repeated a test or ran it on different traffic. ABlyft adds up the data when the page loads. Nothing is saved, and the original results of all experiments stay unchanged.
Which experiments can be combined
You can add experiments that:
- belong to the same project,
- share at least one goal with the base experiment,
- have results in a period that overlaps with the base experiment (they do not need to start or end on the same day),
- you have access to.
You can combine up to five experiments with the base experiment. If no experiment qualifies, Combine Results is disabled.
Combine experiments
- Open the Results of the base experiment.
- Click Combine Results in the top right.
- Under Experiments to combine, choose an experiment. Click Add experiment to add more. An experiment that is already chosen is not offered again.
- Map the variations. Original is always combined with Original. The other variations are suggested by their order (Variation 1 with Variation 1, and so on). Choose Not combined to leave a variation out.
- Keep Limit the date filter to the common period of all experiments switched on. Outside the common period only some of the experiments contribute data.
- Click Combine.
How the results are combined
- Participants, conversions and values of the mapped variations are added up. Conversion rate, improvement and certainty are then recalculated.
- The statistical method and the decision settings of the base experiment are used. The methods of the other experiments are ignored.
- All filters (date range, device types, statistical method) and the charts work on the combined data.
- A goal that only some experiments have is calculated with the data of these experiments only. Its card shows "Not combined · only …"; combined goals show "Combined · N experiments".
- A variation without a mapping keeps the data of the base experiment only and is marked Not combined.
The Combined Results box above the Summary lists the experiments with their periods, the evaluation period, the statistical method and everything that is not combined.
Combined results are indicative
ABlyft does not declare a winner or loser for combined results. Statistical State shows "indicative", coloured green or red when the combined data would meet the rules for a winner or loser. The remaining duration and the SRM check are not shown. Visitors who took part in more than one of the experiments are counted once per experiment, which can overstate the certainty. Only experiments in a shared exclusion group have no overlapping visitors. A different traffic split between the experiments can also bias the improvement.
Share, edit or remove a combination
- Copy link in the Combined Results box gives a URL with the combination and the current filters. Team members with access to the project open exactly this view.
- Edit combination opens the window again to add, change or remove experiments.
- The × next to an experiment removes it. Remove combination returns to the normal results of the base experiment.
While results are combined, Interpret, Share, Reset and Download CSV are hidden, because they only refer to the base experiment. A share link always shows the results of a single experiment.