# Results look implausible

URL: https://docs.ablyft.com/guides/troubleshooting/implausible-results/

> No conversions, an uneven split or numbers that seem too good to be true. Find the cause.



Sometimes the numbers in your results do not look right: a goal has no conversions, the visitors are split unevenly, or
one variation wins by a huge margin after a day. This page helps you decide whether it is a measurement problem or just
normal behavior.

## Too early to tell

Small numbers fluctuate a lot. Early in an experiment, a big difference between variations is usually chance.

* Let the experiment collect enough visitors and conversions. The [decision rules](https://docs.ablyft.com/guides/results/statistics-methods/#decision-rules)
  define when a result counts as a success.
* Use the [Pre-test calculator](https://docs.ablyft.com/guides/results/pre-test-calculator/) to estimate how many visitors you need.
* Do not stop an experiment the moment it first looks significant, see [Interpret results](https://docs.ablyft.com/guides/results/interpret-results/).

## A goal has no conversions

If a goal stays at zero, the goal is probably never triggered. Check by goal type.

| Goal type    | Typical cause                                                                                                                                 |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------- |
| **Pageview** | The page of the goal does not match the URL, or the goal is not assigned to the page. The goal is only counted while its page is active.      |
| **Click**    | The **CSS selector** does not match the element that is clicked. Selectors that depend on generated class names stop working after a release. |
| **Custom**   | The **API name** in the call `ablyft.push()` differs from the **API name** of the goal (it is case sensitive), or the call is never reached.  |
| **Revenue**  | Same as custom, and the call sends no value or a wrong value.                                                                                 |

Verify each goal live, without leaving ABlyft:

1. Open your site in Preview mode with the **quality assurance tool** (see [Preview & QA](https://docs.ablyft.com/guides/experiments/preview-and-qa/)).
   Goals are not sent to the results in preview mode, but you can see whether they fire.
2. Open **Live Log** in your project to see the events that really arrive from real visits.

See [Verify measurement](https://docs.ablyft.com/guides/goals-and-measurement/verify-measurement/) and
[Goal types](https://docs.ablyft.com/guides/goals-and-measurement/goal-types/).

> **Goals only count for visitors in an experiment:** A goal is only sent for visitors who were assigned to a variation. A visitor who was not included, for example because
> of the traffic allocation, does not produce conversions.

## Conversions look too low or too high

* **Counting method.** With **Every**, one visitor who clicks ten times counts ten times. Make sure the method fits the
  question you ask, see [Counting method](https://docs.ablyft.com/guides/goals-and-measurement/goal-types/#counting-method).
* **Goal fires more often than you think.** For example a click goal on an element that is also clicked by scripts, or a
  custom goal called in a loop.
* **Revenue values.** Check that your page sends the amount as a plain number, without currency sign or thousands separators,
  and consistently (same currency, with or without tax). Also check the **value filter** of the goal, see
  [Value filters](https://docs.ablyft.com/guides/goals-and-measurement/goal-types/#value-filters).
* **Your own visits.** Visits by your team count like any other visits. Exclude yourself with
  the [opt-out](https://docs.ablyft.com/developers/consent-and-security/storage-and-privacy/#opt-out) (once per browser) or use the preview mode,
  which does not count goals.

## Goals on another domain

Visitors are assigned to a variation in their browser. A goal can only be counted if the page on which it happens knows
the assignment.

* On a **subdomain** of the same website this works out of the box.
* On a **different domain** (for example a separate checkout on `pay.example.org`), enter that domain in
  **Settings → Project Settings** under **External tracking domains**. ABlyft then passes the assignment on through a URL
  parameter when visitors move over. The snippet must be installed on the other domain as well.

Without this, conversions on the other domain are lost, and results look too low. See
[Snippet settings](https://docs.ablyft.com/developers/installation/snippet-settings/).

## The split between variations is uneven

If you expect 50/50 but see, for example, 60/40 with thousands of visitors, you may have a **sample ratio mismatch (SRM)**.
Small deviations are normal chance. A strong deviation points to a technical problem, and the results cannot be trusted.

ABlyft can warn you, see [Check the traffic split (SRM)](https://docs.ablyft.com/guides/results/interpret-results/#check-the-traffic-split-srm).

Typical causes:

* **A variation loads slower or breaks.** If the code of a variation has an error, redirects slowly, or crashes in some
  browsers, those visitors leave before they are counted.
* **Changes during the run.** Changing traffic percentages or the audiences while the experiment runs shifts the split.

Check variations in other browsers and on mobile devices, and see [Changes are missing](https://docs.ablyft.com/guides/troubleshooting/changes-missing/).

## Overlapping experiments

A visitor can be in several experiments at the same time on the same page. If the experiments change the same area or
influence the same goal, they distort each other. Put such experiments into an **exclusion group**, so that a visitor
only takes part in one of them. See [Exclusion groups](https://docs.ablyft.com/guides/experiments/exclusion-groups/).

## Other effects on the numbers

* **Bots** are filtered by their browser identification only, so some automated visits may remain.
* **Seasonal effects** such as campaigns, holidays or a sale in the middle of the run. Let the experiment run for full
  weeks.
* **Changes on your website** during the experiment, such as a new release, can change behavior for all variations.
* **Results update with a delay.** Events are processed regularly; very recent visits can take a short moment to appear
  in the results. The **Live Log** shows events earlier.

## Still unsure?

Contact ABlyft support and include the experiment, the goal in question and, if possible, a screenshot of the
**Live Log**.

## Next steps

* [Interpret results](https://docs.ablyft.com/guides/results/interpret-results/)
* [Verify measurement](https://docs.ablyft.com/guides/goals-and-measurement/verify-measurement/)
* [Debugging](https://docs.ablyft.com/developers/debugging/) for developers