Create an experiment
Create an experiment, edit its settings and understand the experiment page and its statuses.
This page explains how to create an experiment, what you can configure in its settings, and what the experiment page shows. For a complete walk-through from an empty account to a running experiment, see Your first experiment.
Creating and editing experiments requires the role member, admin or owner. See Roles & permissions.
Create an experiment
- Open your project and go to the Experiments page.
- Click new experiment (for the first experiment: Create your first experiment).
- Fill in the form Create a new experiment and click Create experiment.
| Field | Description |
|---|---|
| Name | A clear, descriptive name, for example Product page: bigger buy button. Must be unique in the project. |
| Enable FullStack for this experiment | Only visible if your project uses FullStack. Adds the experiment to the FullStack config file and enables the FullStack variation type. See FullStack. |
| Example URL | A page of your website where the experiment is visible, for example https://example.com/home. It is not the targeting. ABlyft uses it to build the preview link and to open the Visual Editor. Required, unless FullStack is enabled. |
| Description | Optional note, for example the hypothesis. It is shown below the experiment name. |
ABlyft opens the new experiment. Its status is Draft: it does not run and nobody sees it.
Example URL is not the targeting
Where the experiment really runs is defined by pages. The snippet must be installed on the example URL, otherwise the Visual Editor cannot work on it.
The experiment page
The experiment page is your control center. At the top you find the name, the description and these buttons:
| Element | What it does |
|---|---|
| Save (pink, pulsing) | Appears when you changed something on the page, for example audiences, pages or goals. |
| Status button (Experiment is in draft and similar) | Opens a menu to change the status. See Statuses. |
| Results | Appears as soon as the experiment has results. See Interpret results. |
| Cog icon | Opens the experiment settings. See Edit the settings. |
The main area contains these cards:
| Card | Question it answers | More |
|---|---|---|
| Variations | WHAT changes do you want to test? | Variations |
| Audiences | WHO should participate? | Audiences |
| Pages | WHERE should it run? | Pages |
| Goals | WHAT should be tracked? | Goals |
The right column contains:
| Card | Content |
|---|---|
| Status | The current status, plus tags and, if used, FullStack or ClientSide and the number of exclusion groups. The icon buttons open the Experiment Summary and the activity history. |
| Environments | "Restrict to environments". See Environments. Shows No restrictions set if none is assigned. |
| Schedule | "Start/end conditions". Shows No schedule set until you click Create. See Scheduling. |
While the status is Preview, the Preview box is shown above the cards. See Preview & QA.
Statuses
An experiment is a Draft, in Preview, Running or Paused. What each status means, the requirements for preview and start (at least one variation, page and goal) and how to switch are described in Start, pause & finish. Archived experiments are shown as Archived; see Clone & archive.
Edit the settings
Click the cog icon on the experiment page to open the settings. They are grouped into sections. Click Save when you are done.
General
| Field | Description |
|---|---|
| Name | The experiment name. Must be unique in the project. |
| API Name | Only needed for advanced users. Letters, numbers, dashes and underscores; must be unique in the project. It is created from the name when you create the experiment. |
| Enable FullStack for this experiment | Only visible if your project uses FullStack. |
| Description | Optional description. |
| Example URL | A page where the experiment is visible. Required and must be a valid URL. |
Tags
Tags help you to categorize experiments. Type a tag and press Enter. You can reorder tags, and ABlyft suggests the tags that other experiments of the project already use. Tags are shown in the Status card.
Metadata
Metadata are additional key/value pairs that you can store with an experiment, for example a ticket number. The section only appears if your project defines metadata fields: go to Project → Settings → Metadata and add field names under Experiment Metadata. Field names must be alphanumeric.
In the experiment, add rows with a Key (selected from your fields) and a Value (up to 64 characters). Each key can be used once.
Traffic Split & Distribution
This section defines how many visitors take part and how they are distributed.
- Overall traffic allocation (%) is the share of eligible visitors that is included in the experiment (1-100 %, default 100 %).
- The table lists each variation with its Traffic share. The total must be between 99 % and 100 %.
When you add or delete a variation, ABlyft automatically distributes the traffic evenly again; a remainder goes to the baseline variation. Adjust the shares afterwards if you need an uneven split. More in Traffic allocation.
Statistics
| Setting | Description |
|---|---|
| Statistical method for binary metrics | Applies to goals such as "conversions". |
| Statistical method for continuous metrics | Applies to goals such as "revenue" or "value". |
| Override project defaults | When switched on, you can set the decision rules for this experiment instead of using the project defaults. |
The new experiment starts with the default methods of your project. For the differences between the methods, see Statistical methods.
Experiment Level Codes
Code that applies to the whole experiment and therefore to every variation: JS code, Reset JS code and CSS code. The fields are described in Custom JavaScript.
The field Integrations run mode overrides how integrations (for example analytics tools) are run: Asynchronous or Synchronous. By default (Inherit from integration (default)) each integration decides.
Delete an experiment
On the settings page, the trash icon at the top deletes the experiment. If you only want to hide it, archive it instead, see Clone & archive.