Since Google recently opened the gates for its Website Optimizer to a wider audience, and because we have a paid alternative called MuVar for quite a while, I figured it’s time to compare both programs and see which one brings greater returns in terms of profitability and actual results.
First though, what is multivariate testing anyway?
Before I answer that question, ever heard of split testing, or A/B testing? It is an approach where you show one piece of “something” to 50% of your visitors and another piece of that “something” to the other 50%. After a while, you’ll see which version brings the greater returns (in terms of profitability, conversion rate, etc.)
So split testing is indeed a powerful thing when optimizing sales letters, sign-up pages, and the like. You have a number, a conversion rate, and you want to increase that number. Sure, the first thought would be to increase traffic then you often automatically increase sales or sign-ups. But what if traffic stays the same and you still can optimize the process?
That’s where testing comes into place. And while split testing is a well-known option, multivariate testing is more often than not a much better one because you get results much faster. What it does is it compares multiple variations with each other and shows you which versions or combinations convert better.
So let’s say you want to compare 3 different headline versions with each other, then at the same time you also have 4 different testimonials, and 3 different pictures, different font sizes, and on and on. You get the idea. It actually doesn’t matter what, just that you compare whatever you have in mind, and various options with each other at the same time (not just two.)
Lots of ideas and possibilities, and multivariate testing helps you find out which combination sells better or converts more visitors into subscribers, etc. You have different variables, plug them into the testing system, and compare them with each other to find THE combination.
Now, let’s take a closer look at the two options at hand, MuVar and Google Website Optimizer (gonna call it GWO from now on.) I’m going to continue with bullet points to illustrate the main elements…
- GWO gives you the opportunity of setting up multivariate as well as split tests while MuVar focuses on multivariate tests only. Google says that split tests may be better when your site gets less than 1,000 page views per week. I’m not sure about that. You need a considerable amount of traffic anyway in order to get statistically significant results. When you have only 100 page views per week any testing result is not as meaningful as with 1,000 or 10,000.
- Both MuVar and GWO provide you with video tutorials to walk you through the set-up process. GWO further explains the differences between split and multivariate testing, which does MuVar on its sales page. Google may have an advantage here since it offers an extensive online help (in many different languages!) “behind” the two video tutorials (and its integrated powerful search feature which we’re using every day, aren’t we?)
- The video tutorial MuVar provides walks you through the whole installation and configuration process, so does the GWO, too. But I found the lady in the GWO tutorial is rushing through all the steps within 14 minutes, while the gentleman in the MuVar tutorial takes more than 30 minutes and makes sure each and every step is explained thoroughly. Besides, he shows you a real computer screen instead of a Powerpoint presentation. In other words, I find the GWO tutorial a bit superficial. Yes, I do see what I can do, but I miss the real live examples and “walk-through” shown in the MuVar tutorials. Further, the Google lady speaks into the microphone instead of aside from it, which is the reason why you hear those annoying pop sounds when she’s virtually spiting into the mic. (I’m allergic against this; it’s something many video tutorialists do wrong, often unknowingly.)
- GWO is placed on your Google Account’s home page and is accessible from within your Google Analytics account. I like it when many of my daily tools are available from one place and I don’t have to go to multiple locations in order to get what I want. With MuVar, you need to log in to your site’s MuVar account. When you installed MuVar on, say, 5 sites you need to log in at 5 different locations.
- GWO provides you with reports that show you which combination gives the best results. MuVar gives you these kind of reports as well, but it does a very important task… it optimizes your page by itself without me having to lift a finger! This is a huge advantage and a great time saver. One can say GWO tests while MuVar tests and improves. It’s like having a human resource do that for you every single day, for free. (OK, there is a sort of “batch processing” thing in MuVar that you have to perform occasionally, going to a particular URL once a week or so. I did it myself already; it’s pretty straight forward and can be done within a couple minutes.)
- MuVar can only be used on HTML or HTM pages, whereas (as far as I can see) GWO can be used on PHP pages as well, is that right? They only show HTML extensions in their examples, but they do not explicitly state that you cannot use it in PHP, ASPX, or something like that. I don’t see a need for PHP when having a sales letter or a sign-up page, but more technically inclined Webmasters may have some arguments towards PHP and ASPX.
Bottom line, GWO is available for free while MuVar cost $300. I see the quality in what Google does since I use a couple of their tools myself every day. So the word “free” may not apply here when talking about quality. How often did it happen you had to pay for a certain service, then Google bought that company and made that service available free of charge from that point on. (Now, you may wish Google goes ahead and purchases MuVar, but let’s not go that direction for now…)
The biggest advantage MuVar has is its automation and “built-in human resource.” It optimizes whatever you wish to optimize by itself, without me having to do anything (except the occasional “batch processing” and perhaps adding more variables.)
I don’t know about you, but when there is a tool available that saves me a bunch of time I become VERY interested and excited, and I’m more than happy to pay for that excitement. There are lots of things I have to do during the day. And when there is a tool that helps to empty my desk, I’m very grateful for that.
And although all your GWO tests are available from one location, it does not take so much time to go to a few Web sites in order to manage your MuVar installations.
With all that being said, my vote goes to MuVar. I’m going to use it from now on and am really looking forward to substantially improve my business and conversion rates.
- The Monetary System Has Failed
- Answer Email Communication FAST
- 5 Ways To Market Your Site
- Mind Mapping For High-Speed Brainstorming
- One New Blog Every Second?
- AutoResponder Tracking To Insanity
- What Is Best For You — A Web Site Or A Blog?
- My Posh Video Productions (Duh!)
- Discipline For The Work-At-Home Business Person
Just to add to your list… GWO will only allow you to test 8 different elements at a time. If you’re testing a squeeze page, thats probably enough. But for a long sales letter, you’re going to struggle. I believe MuVar allows unlimited elements because of how it stores the data.
What part of Brazil do you live in? I’ll be in São Paulo for a week. I’ve been enjoying the videos you have released recently.
[…] https://www.hochstadt.com/muvar-vs-google-website-optimizer […]
This post has been featured in FullTiltBlogging.com’s Daily Blog Summary today. Great post!
MuVar does offer a unique process for multirate variable testing – but I think it’s short sighted to believe that only .html pages are necessary for testing.
I have several clients who’s site designs are with .php and .asp/x pages. To create a static page (even trying to use server side includes) would drastically change the visitor experience and greatly complicate the code that is currently working for them. These are sites with more than a few pages that utilize a content management system (CMS). Trying to get the same look and feel for an optin or sales page using html instead can be a challenging sell because the site was already designed once and now the business owner has to pay to have template designed again in html.
The only sites I have seen pointed to for MuVar examples all use the generic ‘blank page’ template that is James Brauch’s signature. Marketing departments about to let their site’s look and feel change for a single page. Yes, one can argue that increased conversion ‘might’ be able to justify the cost – but now we’re talking about doing a split test between a dynamic page that can be integrated with GWO and a MuVar page. Do you know how to set up that split test?
For now, if a client has a dynamic site (php or asp pages) I lean toward GWO – if they have static html pages then MuVar could be a good fit.
I’d like to see examples of MuVar sites that don’t have the blank page template – can you recommend any that aren’t in the Internet Marketing niche?
I bought Muvar before it was ever disclosed that it would not work on php, so I am one of the unhappy owners of it.
If you have secure pages on different servers (such as a hosted shopping cart) then forget about muvar. It is better geared to simple info product sales using the few main payment processors, if that is your cup of tea.
Google analytics has TONS of informational statistics, and combined with Google Optimizer, it is really very strong.
Michel Fortin publicly states that now, he almost exclusively uses and recommends Google Website Optimizer, and he works with the biggest names in the business.
For the sophisticated crowd that want bells and whistles, you should also be aware of SplitTestAccelerator.com, as well as kaizentrack.com. Those are for your own server. You will also find lots of hosted turnkey solutions as well.
It’s funny you’re blogging about this: I just posted a video at my site that goes into depth about GWO. I set up a test, step by step, in 11 minutes. It’s incredibly easy.
Don’t forget, there is a whole nother world out there that sells a lot more than just info products with sales pages. And most of those people use shopping carts that use PHP. That’s where Muvar fails. GWO CAN use PHP, but with some limitations. It will not necessarily capture all of the dynamic content.
And in fact, I actually use PHP on my landing pages (AKA sales pages) when using Adwords. I use it for dynamic keyword insertion into the title tag so my Quality Score goes up, big time. No dice with MUVAR. I had to change a lot of my landing pages, but it saves me a lot of time and project management. I manage maybe 50 sales and landing pages in my business, and MUVAR makes it so I don’t have to babysit it. Worth its weight in gold.
Don’t forget about the video I posted on my site.
>> I’d like to see examples of MuVar sites that don’t have the blank page template – can you recommend any that aren’t in the Internet Marketing niche?
MuVar allowed me to test 82 segments of my copy in this long-standing sales letter and automatically remove the duds. I just posted the details about this testing at:
So we can conclude that GWO is an option for those with an extensive shopping card system, dynamic content, and the like, while MuVar is for the large small business crowd?
I mean, the amount of time you save when using MuVar is HUGE. And as Gary said, with GWO you are restricted to 8 elements at a time. When I start to use it I am definitely going to test as many variables as I can… colors, sizes, nothingness, just about anything.
MUVAR – great idea – terrible execution
I put in 2 different headlines at the SAME time:
Version 1: 1044 impressions with 141 conversions
Version 2: 179 impressions, 31 conversions.
MUVAR thinks that version 1 is the winner.
Version 2 has a HIGHER conversion rate and when you run the numbers through a split test confidence measuring tool, version 2 is WINNING with a confidence measurement of 91%.
So…I’m forced to assume that MUVAR only accounts for NUMBER of conversions, which doesn’t necessarily have anything at all to do with conversion rate
Absolutely. Without MUVAR (with GWO), you have to babysit the test and check it often to see if there is a winner. When there is a winner, you make the changes and test more. Let’s say you value your time at $100 an hour (very low), at MUVAR’s current price, with only one landing page, you’ll achieve a positive ROI within a month or so.
I would like to caution you, Marcus, the more variables you test, the longer it takes to get a conclusive result. Sometimes you may want to test less headlines to get a good result and make the changes to increase sales, especially with GWO. Doesn’t really matter with MUVAR because it will do it for you – test as many variations as you want.