How I Used Optify and Google Experiments to Run a Landing Page A/B Test

On August 1, 2012 Google moved it’s old Google Website Optimizer into the Google Analytics product and renamed it Google Content Experiments. I recently ran an A/B test on an Optify Landing Page using Google Experiment. Here are the basic steps for the setting up, running and measuring your landing page test.

10 Steps for Running a Landing Page A/B Test with Google Experiments and Optify

Step 1 – Choose the variable to be tested

Every A/B test starts with the variable you would like to test. By its nature, an A/B test tests one variable, but you can run multiple A/B tests at once to test more variables. The rule of thumb is that each tested variable should be tested on a separate “cell” (or variation) which would mean that the number of variables is always one less than the number of test variations. For example, if you test one variable, you will have two variations of the asset – Control and Treatment 1. If you are testing two variables, you will have 3 variations – Control, Treatment 1 and Treatment 2.
In the test I was running, I was testing the variable “preview image.”

Step 2 – Set up the hypothesis

Your hypothesis should include the tested variable and the “decision rule” and should be clear and short. A good structure is “A [marketing piece] with [tested variable] will have a better [decision rule] than a [marketing piece] without [tested variable].” For example, “an email with a personal from line will have a higher open rate then an email with a generic from line.” In this case the marketing piece is an Email, the tested variable is the From Line and the decision rule is Open Rate.
In my test, the hypothesis was “A landing page with a preview image will have a higher conversion rate than a landing page without a preview image.”

Steps 3 – Define the “Decision Rule”

The Decision Rule is the metric that will determine the best performer – which of the variations performed best. It’s important to pick that metric BEFORE you run the test and not AFTER to avoid personal biases when determining the best performer. The Decision Rule should be simple.
In my test it was conversion rates as measured by the total number of forms submitted on each page over the total number of visits to that page.

Step 4 – Determine sample size, statistical significance, experiment timeframe

Sample size, statistical significance and timeframe are all related since they determine how much poor performance you are willing to endure while you are trying to figure out what you need to fix. It’s like running water through a pipe in order to find the leak; you need to find it and fix it as quickly as possible. But with A/B testing that timeframe is determined by the statistical significance you need to achieve which is determined mostly by the sample size and the results.
In my test I specified two weeks for the experiment and a minimum of 500 visits or achieving statistically significant results, whatever comes first.

Step 5 – Build the variations

Hold everything constant but the tested variable. The easiest way to go about this is to create the first variation (preferably the Control) and then simply clone it and change the one variable you want to test.
There are three ways to use Optify in this step:

1. Create your landing pages and embed the Optify forms in both of them.
2. Use Optify Landing Page to create the variations. If you do that, you will have to also create a landing page on your CMS to use as the facilitator of the content swap using Google Experiments. (Optify currently doesn’t support adding the Google Experiment snippet to our landing pages, so you will have to use your own CMS for the “Original” page).
3. Create your landing pages and just track their performance in Optify.

I used the second method and created the Control and Treatment pages – the landing pages I wanted to test – in Optify, and the “Original” page in WordPress. All three pages looked the same with the exception of the preview image (my tested variable).

Step 6– Setting up a Google Experiment

1. Go to your Google Analytics and click the “Standard Reporting” option at the top menu bar. On the sidebar, click the Content option and then Experiment
2. Click the “Create experiment” button
3. Put the URL’s of the landing pages you want to test. If you’re using Optify Landing Pages, make sure you put those pages in the Variation placeholders and your CMS page in the Original placeholder.
4. Select an Experiment objective – that’s your “decision rule,” the metric that will determine what page performed better. Google will serve the pages based on how well they perform on your experiment objective. If you are using Optify to determine the best performer, set an Experiment objective that will be the same for both pages (a Thank You page, for example). Keep the “Visitors included in the experiment” at 100%.
5. Add the experiment code to your Original page. I found this to be the most challenging step since Google is very specific about where the code has to be on the page. If you’re using Optify Landing Pages , you also need to make sure you add your Google Analytics account ID to those landing pages. You can find that field under Properties in the Optify landing pages application.
6. Once your pages are tested and working, Google will let you review the Experiment and launch it.

Step 7 – Drive traffic to your pages

After launching the experiment you now need to drive traffic to the pages to let the experiment run. The easiest way to get a lot of volume fast is to send an email to your house list (or a third party list that sends them to the landing page).
In my experiment we used email, social media and third-party content syndication vendors to drive people to the landing pages as well as website promotion.

Step 8 – Monitor the test and measure results

I checked the results on the landing pages at least once a day to see if I can announce the Best Performer as quickly as possible and use only the higher converting page.
I used this great excel template to know if my results were statistically significant so I can stop the test and use the bet performer. I got to statistical significant results after 468 visits to the page.

Step 9 – Declare the champion, stop the test and apply lessons

If you are using Google Experiments, once you know your best performer don’t forget to stop the and then apply the lessons you learned. In my case, my Champion was the landing page with the preview image and we made sure that all of our future landing pages would include a preview image.
An important, sometimes overlooked step in declaring a champion is documenting the results for future use. Not all tests are as clear as the one I ran, so remember to document your results and lessons so you don’t repeat your past mistakes.

Step 10 – Start a new test

One test is done, another one starts. Testing never ends and as marketers we always need to be testing our tactics, copy, creative, channels and messaging. So test away and good luck!