MailChimp or AWeber: A Conversion Showdown

mailchimp vs aweberEver wonder which email marketing service provider you should use? Or if you should switch from one to the other?

The two “big wigs” in the industry are AWeber (over 120,000 users) and its free competitor MailChimp (over 3 million users).

And I’ve decided to find out which is the best – once and for all.

How am I deciding?

I’m basing it off of the first step – do people actually end up on my list after the double opt-in?

I’ve heard rumors and have experienced myself a few unexplained missing confirmation emails from my current service provider – MailChimp.

I’m embarking on this experiment to find out which email service provider has better delivery rates at the most important step – the step where that casual visitor becomes an active member of your community.


December 2, 2013 Update

I ran the Google Content Experiment using Google Analytics to determine the winner – AWeber or MailChimp – for which email provider performs better at getting subscribers on your list from the first initial step – signing up & confirming their email.

The two pages were EXACTLY the same except for the which email provider they sent the subscriber’s email to. A very plain, default subscribe form.

You can view the exact results in this screenshot:

Comparing MailChimp to AWeber

The screenshot is of my Google Analytics account and clearly shows that AWeber ended up confirming more sign-ups than MailChimp.

What conclusion can we draw here? Hard to say exactly.

It could be that AWeber has better deliverability rates (aka is better at avoiding the spam bin or just getting to inboxes in general).

It could be that the default email from AWeber is more effective at getting people to confirm their email address.

Either way, from this one experiment anyway, it looks like AWeber may pay off in the long run in getting you more subscribers for your email list.

Curious about how Google Content Experiments work? Check out this post Analytics Advocate Daniel Weisberg.

Will I be making the complete switch from MailChimp to AWeber? I’ll certainly be experimenting with it :)

UPDATE: As some of the comments have pointed out – I wanted to make one point clear – MailChimp CAN and DOES convert opt-ins for you – I’ve used MailChimp from day one and have an actual newsletter list. The zero in the above screenshot is not to imply that MailChimp will NOT give you subscribers – that’s a wrong conclusion. It’s just that in this time period, AWeber converted more. I’m not the only one who’s come to the “maybe AWeber grows your email list faster” conclusion – check out this post from Think Traffic on the subject:

Over to You

What email marketing service provider do you use? AWeber? MailChimp? Something else? None? Do you think this experiment result will have any impact on who you use? Let me know in the comments!

11 Responses so far.

  1. Dylan Jones says:

    Hi Liz

    Interesting test but I think to make this a fair comparison you need to introduce some additional steps.

    If the forms are identical then the data entry rate should be identical but that data isn’t given. Is there any way you can connect your forms to an additional data entry store like Google Spreadsheets to see what emails are collected?

    For example, I use Squarespace as my CMS so I’m able to store emails in my inbox and also Google Spreadsheets as well as sending to Mailchimp.

    This is important because I’ve noticed that when someone registers for our site that has already registered (it happens quite often) MailChimp doesn’t recognise this as a new event (or conversion) in your case, it simply updates the original account.

    That may explain why you’re getting some drop in conversion but to get 0% is odd, something doesn’t look right, you should be getting at least some conversions off Mailchimp I would say.

    What I would do is track what happens on the user experience after the email has been registered.

    Can you create two pages, one for the MC sign up and the other for aweber, share them on the site and we’ll register for both and comment on the user experience.

    There could be something post-data entry which swings in awebers favour.

    • Liz Lockard says:

      Hi Dylan! Thanks for your comment. I agree that I would love to nail down the process a bit more to find the specifics why of why AWeber won – I like the idea of testing the “almost finished” page – I might run with that one in my next experiment. I will tell you though that I have gotten sign-ups through MailChimp previously on the page I tested and post – I did not want to reveal publicly which page I was testing to keep the experiment as impartial as possible. If you dig into Google Content Experiments you’ll see they use a method called “multi-armed bandits” to find the winner which often results in faster winner selection though still statistically valid.

    • Nat says:

      I agree with Dylan — you’re making a bold conclusion but omitting a lot of information… and something just feels off. More data is needed and a much larger sample. Please keep in mind that you used a tiny sample of people and it’s one experiment; this doesn’t constitute “proof” that Aweber works better and that MailChimp gets you no conversions — that’s just nonsense. Of course MailChimp opt-ins convert (my lists are proof and yours too).

      It may serve your readers better for you to highlight what a small sample you used and that these results do not constitute conclusive findings, so you don’t mislead people who don’t know enough about email management systems to question this.

      • Liz Lockard says:

        Hey Nat! Thanks for your comment. I’ll definitely continue to experiment but Google Content Experiments are statistically valid – and I agree that of course MailChimp does convert opt-ins – I was just trying to answer how many of them go into the ‘blackhole’ of never confirming the opt-in. More experiments happening as we speak :)

  2. Valerie says:

    Thanks, Liz – very interesting. I use Aweber and really like it. I’ve heard that it has one of the best deliverability rates in the market and it looks like your experiment confirms it.

  3. Nancy says:

    I’ve always heard Aweber is the way to go. I currently use MailChimp because it fits within my current operating costs to do business. I’ve noticed a lot of my subscribers who sign up on my website do not opt-in, and wonder if those emails were going into the black hole. Some of them were Gmail users, and recently Gmail changed its format, and I had to tell them to make sure to put “me” in the primary list. This post definitely makes me ponder more, and investigate more, and utilize those analytic tools. I may want to give the trial period of Aweber if it has one a try. Thank you, Liz.

  4. Hi Liz, I’m a Mailchimp user but some of my clients use Aweber so I am familiar with both. I wonder whether it would be possible to track in both services who signed up bit didn’t confirm?

    If not, then perhaps setting up a goal in Google Analytics and finding out who finished the double opt-in process? But you’re probably doing that already. Just thinking aloud this cold early morning here in the Philippines.

    And I replied to your email to this regarding the header that won’t display… error on line etc etc

  5. Danielle says:

    Hey Liz – nice comparison. As a web shop, we are usually the ones helping a client to decide on the best solution to put in place. We start with outlining what our clients are going to send… newsletters, sales, notes, etc.

    Next we look at how it will integrate with how their site is built. Some services are just a complete mess while others have a killer API to work with. And some don’t allow much customization while others allow you to go nuts and have total control.

    Finally we look at the client and their skill set. We want to make sure folks are comfy in the system and can use it easily.

    Just wanted to share another approach to choosing the right email marketing platform based upon the more technical side of things.


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>