What’s the Best Landing Page to Get Subscribers on Your Mailing List?

The Summer of Marketing email course (from OkDork, Noah Kagan’s blog) asked students to make the best lead magnet they could to get subscribers on their mailing list.

A lead magnet is a free guide, free course, free anything that you give away on your website in exchange for an email address. It’s a well established way to get people on your mailing list that you know are interested in what you talk about or sell.

Working in a design and conversion rate optimization (CRO) agency, I thought, we have to win this thing. Surely we can make an awesome looking and useful guide on CRO.

So I tried my best. We blocked off multiple hours that week to make a great looking guide of my top 10 tools for CRO from scratch.

And then, we got this email:

Yay!

Phew! Hours not wasted, and a bunch of traffic would be coming my way!

That was exciting, but due to when I was told and some fun weekend plans, I had only a few hours on a Saturday morning to make a landing page to convert this traffic into subscribers to my mailing list.

And, being a conversion rate guy, I couldn’t just make one landing page, I had to split test two versions. Time to act fast.

Making a Landing Page in a Few Minutes

To create a landing page and split test it, you could make the page in WordPress (if you use it) or hardcode it (if you’re a baller) and use Optimizely, VWO, or any other split testing service. But I didn’t have time for that.

I opted to use LeadPages for three reasons:

(Note, I know this sounds like an ad, but I’m not an affiliate, they give me no money, in fact, I pay them. I’m just being honest with what I use and why in the hopes that it’ll help you if you’re ever in a similar situation).

1. They have a bunch of great looking pre-made landing page templates, which you can rank by highest conversion rate so I have to do no coding or fiddling around, just replace stock info with my info.

2. They let me split test easily within their system. (Not as robust as Optimizely, but good enough for now)

3. They let me quickly deliver a lead magnet without having to fiddle with my Email Service Provider.

So I found this nice looking landing page, which seemed to be high on their “sort by conversion rate” list, and I like the color blue, so I went with it:

This is what we turned it into:

So now I had a landing page, time to split test it.

A/B Testing a Landing Page: What to Test?

A lot of people read about A/B testing and decide to do it (great first step!). Then they start testing little minutae like changing a button color, adding a word or two. Here’s the thing about split testing, little changes usually result in little results. (Usually is a key word. Yes, sometimes a little change makes a big result. But sometimes people also win the lottery.)

If you have no idea where to start there are a two basic fundamentals to think about:

1. Value proposition. This is the main reason people buy or click. What are you offering? Is it stated clearly? Is it believable? Is it actually valuable? Are there multiple value propositions for your customer? Which one are you emphasizing now?

2. Clear Call to Action. Are there 453 things that someone could click on, read about, or do on the page? What’s the 1 thing you really want them to do? Is that obvious?

You can test a million things but they are generally variants of those two things:

Headline test? Value prop
Button color? CTA
More or less explanation? Value prop and clear CTA
In my case, this wasn’t a web app with a complicated UX that I’m trying to optimize or reduce the number of clicks, it’s just a simple squeeze page.

So I opted to test both in one test.

I just took the original (“A”) above, and eliminated most of it to create the variation (“B”). This variation definitely tests Principle 2 (Clear CTA), since a bunch of info to read is gone, you basically have 2 options: click the button, or leave. It also tests Principle 1 in a more subtle way: If the variation does equally well or better, it would mean that “Who Am I?” and “Tools for Every Step” are not relevant to people thinking of downloading the guide. In other words, knowing who wrote this and knowing what tools are mentioned are not valuable enough to sway anyone.

My variation (“B”), no smiling face, no extra info.

The Survey: What did Readers Think would do Better?

Before revealing the results I decided it would be more fun to ask the very people that went through the test which one would do better. Of course I sent that email out on a Friday so the only 12 responded, but nonetheless, the results were amusingly even:

What’s interesting is that though it was dead even on which would do better, not a single person thought there may not be a winner. Certainly if the pages were that different, one would be a clear winner right?

The explanations for why people voted either way are also hard to argue against. There are various reasons why either page could do better and you guys basically hit on all of them:

For the long page:

• “It shows your picture – that’s the one that I saw and it made me gain trust in the offer, seeing the person behind it”.
• “Because people like to see informations about what they buy/download/… It’s also important to see the face of the person, it makes us feeling in a safe website”.

For the short page:

• “People have short attention spans”.
• “Because it’s a free download, and so people can just download to find out about it”.
  So which did better?

The Results

Here are the results of the A/B test, straight from LeadPages:

At first glance one might say “B” wins, but look at the margin of error on those conversion rates. The difference in the two conversions is not beyond their margins of error (+/- 5%), telling me that in fact the two conversion rates aren’t really all that different.

Here’s more proof of that: Using Evan’s Sample Size Calculator, I can ask how big my sample size needs to be to be sure the variation I’m seeing is within a certain confidence level (in this case 95%).

So I set the baseline conversion rate to what the original was converting at (at the time…26.6%) and asked, how many views would I need to be able to detect a 3% variation with 95% confidence:

Apparently 3,440 per branch, so 6,880 visitors total. Slightly higher than…180!

Yes, if you have enough of a sample size, inevitably some variation will eventually turn out to be a winner, with statistical significance, every time. The reality, of course, is that we live in a world where we don’t have infinite sample sizes. So we have to ask an important question: Did this test show a winner in the sample size range that’s reasonable for my amount of traffic?

In this case no.

How could this be?

Without jumping into the head of each person that went through the landing page, we don’t really know why, but we can make some guesses.

If I had to bet money on why, I’d say that it was because what’s below the main call to action on a simple squeeze page really isn’t that big of a deal. The headline (most read part) in both cases was the same, and the main CTA (button) was also identical. Did some people make a decision based on the extra info below? Maybe. But not enough to matter. Certainly not enough to make a big difference in the hundreds of visitors range. If we were cnn.com and had millions of visitors, sure, one variation would be a clear winner, but in this case, the difference between the original and variation just wasn’t big enough.

Does this mean long form sales pages are useless?

No! Another important lesson is that context matters. In this case, I was just giving away a free ebook, and you could get it from a button on the top. Just as one of the quotes in support of the short page said, people don’t need to be “convinced” to get it because giving away an email isn’t as big of a deal as giving away $100, or $1000, or more.

Generally, the more expensive the item, the more elaborate the “woo-ing” process needs to be with the customer. This is true offline too. Think about what it takes to buy a toothbrush. “Ok, here are the toothbrushes. I want….that one. Let’s go.”

Now what about a house? No one wants to read a 2,000 word sales page to buy a toothbrush, but if someone said “Quick, buy this house now, don’t think about it, just click. Do it. Buy it.” You would run away fast. That’s basically what a short squeeze page/landing page is.

How you can apply these results to increase YOUR conversions

Think about the two fundamentals mentioned above and how they apply to your site. What do your readers or customers really care about? Are they interested in how many pages your ebook is, or are they interested in whether it will actually work in helping them lose weight?

Now think about how clear your CTA is and how distraction free the whole page is. Where on the toothbrush-to-house spectrum is your product? How much woo-ing do you need to give your prospect?

After you determine this, think, “Who do I know that is absolutely killing it selling something similar to my product?” How long is their sales page? What value are they advertising? After all of this preparation, then begin testing and you’ll be off to a great start.

But first…

It’s valuable to get outside input. Think of one thing from this post that you could change or test on key conversion pages on your site and mention it in the comments. If you want, link to the page and ask the community for their input.