A/B tests

An A/B test shows two or more versions of something to different visitors at the same time, then measures which version gets more of the result you care about. You make a version A and a version B, leadmaps shows each one to a different slice of your visitors, and it counts how often each version leads to the thing you want, like a click, a signup, or a purchase. At the end you do not have to guess which one is better. You can see it in the numbers, and leadmaps even tells you whether the difference is real or just luck.

Think of it like a taste test at a market stall. You hand half the passers by version A of a recipe and the other half version B, then you count how many of each group come back for more. Because the two groups are otherwise the same crowd on the same day, the only thing that differs is the recipe, so the result tells you which recipe people actually prefer. An A/B test is that taste test for your website.

The reason to test instead of just deciding is that opinions are cheap and often wrong. A headline you love might do worse than a plain one. A button color you barely thought about might lift signups. Testing replaces “I think” with “I measured”, on your real visitors, with your real goal.

What you see

The page lists your tests and, for each one, the result so far.

A page title that reads “A/B tests”.
A list of your tests. Each test shows its name, whether it is running or finished, and a quick read of how it is going.
For a single test, the detail view shows:
- The versions. Each version (A, B, and more if you added them) listed by name, for example “Old headline” and “New headline”.
- The result for each version. How many visitors saw it, how many of them reached your goal, and the rate (the share who reached the goal). The rate is what you compare, because it is fair even when the versions did not get exactly the same number of visitors.
- Which version is ahead. leadmaps marks the version that is currently winning on your chosen goal.
- Whether the difference is real. A plain read of confidence, telling you whether the gap between the versions is big and steady enough to rely on, or whether it could still just be chance. This is the part that stops you from calling a winner too early.
A way to create a test: name it, set up its versions, pick the goal, and choose how to split the traffic.

A “version” is one of the things you are comparing. The “goal” is the result you are measuring, like a purchase. The “rate” is the share of visitors who reached the goal. “Winning” means a version has a higher rate. “Real, not chance” means the gap is large enough, over enough visitors, that you can rely on it.

How to read it

Compare the versions by their rate, then check whether the difference is real before you act.

Here is a worked example. Say you are testing two headlines on your home page, with the goal “started a signup”. After a week leadmaps shows:

Version A, “Track your sales in real time”: 5,000 visitors, 250 signups started, a 5.0 percent rate.
Version B, “See where your money comes from”: 5,000 visitors, 325 signups started, a 6.5 percent rate.
Leading version: B.
Confidence: high, the difference is very likely real.

What does this tell you? Both headlines were shown to a similar number of visitors, so the comparison is fair. Version B turned 6.5 percent of its visitors into signups, against 5.0 percent for version A. That is a 1.5 point lift, which on this traffic is about 30 percent more signups from the same visitors. And because leadmaps reports high confidence, the gap is big and steady enough that you can rely on it, not a fluke of one good day. Your move is clear: make version B your permanent headline, then think about what made it work (it speaks to a benefit, “where your money comes from”, rather than a feature) and apply that lesson elsewhere.

Now picture a different ending. Say after only two days version B is ahead 6.5 percent to 6.2 percent, but confidence is low. The temptation is to declare B the winner and move on. Do not. A small gap over a short time is exactly the kind of thing that flips back and forth as more visitors arrive. The confidence read is leadmaps telling you “wait, this is not settled yet”. Let the test keep running until the gap is either clearly real or clearly tiny. Calling a winner too early is the most common way A/B tests lead people astray.

Customize it

You set up the versions, the goal, and how traffic is shared. Here is each control and when to use it.

Define the versions. Create the two or more versions you want to compare, and give each a clear name. Use two versions (a plain A versus B) when you want a clean, fast answer, which is the right choice most of the time. Add a third or fourth version only when you genuinely have several distinct ideas to weigh, and remember that more versions means each one gets fewer visitors, so the test needs more traffic or more time to reach a clear result.
Pick the goal. Choose the single result that decides the winner, for example “started a signup”, “completed a purchase”, or “clicked the main button”. Pick a goal that is close to what you actually care about. Testing for clicks is easy but can mislead, because a version can win more clicks yet fewer sales, so prefer a goal further down your funnel when you can measure it.
Set the traffic split. Choose how visitors are divided between the versions, for example an even 50 / 50 split for two versions, or an uneven split when you want most visitors on the safer version while a smaller share tries a riskier one. Use an even split for the fairest, fastest read. Use an uneven split when one version is unproven and you want to limit how many visitors see it while you check it is not harmful.

Use cases

Test two headlines. Create a test with version A as your current headline and version B as a new one, with the goal “started a signup”. Split traffic evenly and let it run until confidence is high. Action: keep whichever headline wins on the real rate, not on which one you personally preferred, then carry the lesson from the winning wording into your other pages.
Test a button. Compare two versions of an important button, changing one thing at a time, for example its text (“Get started” versus “Start free”) or its placement, with the goal being the click or, better, the action it leads to. Action: ship the version that lifts the goal, and because you changed only one thing, you will know exactly what made the difference and can reuse it.
Test a pricing layout. Show two versions of your pricing page, for example one with three plans and one with the middle plan highlighted, with the goal “completed a purchase”. Let it gather enough visitors, since purchases are rarer than clicks and need more traffic to reach a clear result. Action: adopt the layout that wins on real purchases, and resist judging it on early numbers before confidence is solid.

Tips

Change one thing at a time. If version B has a new headline and a new button and a new color, and it wins, you will not know which change did it. Test one difference per test so the result actually teaches you something you can reuse.
Wait for confidence before you call a winner. A small lead early on means almost nothing, because it swings around as more visitors arrive. Let the test run until leadmaps says the difference is real, or you will keep “winners” that were just luck.
Pick a goal that matters, not just the easy one. Clicks are simple to measure but can mislead, since a version can win clicks and lose sales. Whenever you can, measure the result closer to the money, like a purchase or a completed signup.
Give rarer goals more traffic and time. A goal that happens often, like a click, settles quickly. A goal that happens rarely, like a purchase, needs many more visitors before the result is reliable. Plan for a longer run when the goal is deeper in your funnel.
A test on a fresh site with little traffic will stay undecided for a while. Confidence comes from numbers, and numbers come from visitors. If your site is new or quiet, expect the result to take longer to settle, which is expected, not a fault.