How to Improve Your Website With A/B Testing

How to Improve Your Website With A/B Testing

In recent years, competition on the Internet has grown fiercer, and many businesses are looking for anything that gives them that competitive edge. Online marketing methods such as SEO and PPC are a good way of improving a company’s ability to compete, but increased traffic alone doesn’t make for a successful business. When it comes to a website, it all comes down to number of conversions and thus, revenue generated; this is where a/b testing can help.

Improve Your Website With A/B Testing
Image credit: Darrenhester

What is an A/B Test?

A/B testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates. – Wikipedia

As the Wikipedia quote above describes pretty well, A/B testing is a method whereby an alternate version of an element within a website can be displayed to a group of users in order to track its effect on conversion rates.

Multivariate Testing

When using A/B testing it is recommended to make only single changes to an element in order to track its success in isolation of any other factors. What Multivariate testing does is allow you to track a wider range of changes in order to see which is the more effective combination of changes. Although it tends to return better results in the long run, it is more difficult to analyze any results returned.

How to Perform an A/B Test?

If you’re a whizz at coding then it’s possible to create your own A/B tests, but be sure to do the following:

Drop a Cookie
Image credit: esti-

  • Drop a Cookie: It’s always advisable that a user sees only one version of a site, even on return visits. If a user were to see the multiple versions of the site then it’s more difficult to determine which version led to a conversion.
  • Cloaking: Traditionally this is when you have two versions of a page, one targeted to users and another to the search engines. A/B testing, although strictly for user testing, can be implemented in a way that appears as if the site is being cloaked. Be careful to avoid this.

For those of us that prefer the simple life, there are a few tools already available that help us create and track A/B tests as well as take out some of the dangers such as cloaking:

The Prerequisites

A/B testing appears to be an awesome tool to play about with and in truth it is, but like anything, it can’t simply be used for the sake of it. For A/B testing to be a worthwhile exercise, there are two prerequisites that a site must have:

A Conversion Point

The way in which A/B testing works is that for each conversion a site receives, a check will be carried out to see which version of the tested element was visible to the user. If this conversion point doesn’t exist then there is no way of telling how a test is performing. Common conversions that are easily tracked are a sale through an ecommerce website or the submission of a form.


A/B testing is all about the numbers, and to be truly accurate A/B testing needs to gather a decent amount of results, which means the site needs a decent level of traffic. Although a result can be gained with low levels of traffic the data gathered is likely to be less reliable. To really benefit from A/B testing you ideally need visitor figures in the hundreds if not thousands per day.

Image credit: th.omas

What to Test?

With A/B testing you pretty much have free range to test just about everything. It might be that you change the color of a button or the headline of a page, nothing is impossible, but it is best to target a few key areas. Primarily, anything you do change should be on the path to conversion. So in the case of an ecommerce website this could include the home page, product page, basket and checkout as well as a few others.

But it’s not just what to change but how to change it. For some clarity I’m going to focus on a Buy button as an example of what can be changed to affect conversion rate.

Micro Copy
Tone of voice plays a big part here, would your users respond better to a more forceful tone by changing the buy button to read ‘Buy Now’ or would they prefer something a little less committal like ‘Add to Cart’? Think also geographically as UK visitors might respond better to ‘Add to Basket’ with cart being a more American term.

For a buy button you’d really need to use a color that contrasts the background of the button to ensure it stands out, but would green be a better color than red?

Yes there is a fold, and most certainly your buy button should be above it, but is it better placed to the left or right, maybe next to the price or quantity input. On such pages you want to guide the user’s eye in a specific way with the buy button the end point.

Maybe your buy button is a little small, would increasing the size lead to more conversions? Maybe your button is huge and you could afford to make it a bit smaller without losing too many conversions. Making it smaller might not seem like a good idea but if it increases the white space around it, you may actually see positive returns.

Is the buy button too simple? Could a border improve conversion, how about a shopping cart icon? The addition of complexity may help a wider range of users quickly identify what the purpose of a given element is.

The opposite of complexity, it’s possible your buy button is simply overcrowded with design elements such as gradients, bevels, shadows and the like. Maybe a user will appreciate a more minimal design.

These just highlight the changes as they could be found on a single element, but can apply to almost every area of a website.

What can Go Wrong with A/B Testing

The benefit of A/B testing is clear, given the right conditions it will provide data to allow you to make a decision as to whether or not an altered element converts better than the baseline. Then this new element becomes the baseline and the tests continue. But there are a few pitfalls to be aware of when undertaking A/B testing.

Stop When The Benefits Are Too Small

You might be lucky enough to already have a high conversion rate, or find that one change raises your conversion rate by a healthy amount. If either of these are true then consider the benefits of undertaking or continuing with any further testing. The time and effort involved in A/B testing may simply not be worth it if the benefits are so small. If you have a good conversion rate then resources are better focused on increasing traffic to make use of these good conversion rates.

External Factors

Whenever possible it will always be more beneficial to undertake A/B testing in as clean an environment as possible. What this means is that the test results are likely to be more reliable if external factors have not affected the behavior of users. External factors that could play a big part include traffic spikes due to offline advertising or seasonal shopping such as Christmas. Testing in these periods can also affect overall revenue if any tests don’t return a positive result.

Start Small

Negative results from testing is a clear possibility, you might simply have the best conversion rate possible already. If this is true then any tests you carry out can only negatively affect the website’s revenue. So to reduce the impact, it’s best to start small, testing on a lower number of users, maybe 10%. If this test goes well then crank up the numbers to ensure accurate results are obtained, if not then kill the test and try something else.

Start Small
Image credit: BWJones

Newly Launched Websites

If you have a brand new website then A/B testing is something you should be wary of undertaking. With a new site it’s likely that the changes already made to the website are interesting users, leading to month by month increases in traffic and conversion rates. It may be a few months until the site normalizes and you can get a clear picture of how the site converts in normal circumstances and whether or not there is potential for improvements through A/B testing.

Going in Blind

To get a good understanding of what it is you should be testing then it’s best to have a good amount of data to help inform your decisions. Analytics and heat maps of user activity are invaluable here in helping to indicate the actions of users and where they might be dropping out. Having such information allows you to better target the pages of concern where the biggest increases might be found.

Going in Blind
Image credit: lupinehorror

Testing for Every Decision

In design it’s sometimes difficult to justify the smaller aspects of a design, why a particular color is used or why a border is 2 pixels instead of 3. Such small elements won’t be noticed by the average user but within a company A/B testing might seem like a solution to solving such discussions over a design.

Famously, Google tested 41 different shades of blue to ensure they got the best conversion rate, though the results weren’t released. Would the average Google user have noticed such a difference, and if so would it make them more likely to click on a link or visit Google more often? Sometimes decisions are best made in-house rather than opened out to the public.

It Can’t Give You An Awesome Website

Sure, A/B testing can help increase conversions for a website, but for the most part you need a well designed and usable website from which to start. If your website looks like it’s from the 1990’s then A/B testing simply isn’t worth it. Instead, any website owner’s money would be better spent hiring a good web designer to get their site modernised and then approach testing afterward if deemed necessary. Simply modernizing an old website should lead to an increased conversion rate.

More Reading

To learn more about A/B testing there is nothing better than to actually read the experiences of others and how they have approached A/B testing, and how their tests have performed. A great site exists called simply where users can upload their own case studies for everyone to read.

Also check out the Conversion Rate Experts Blog for more in-depth analysis.

More Reading
Image credit: moriza


Improving conversion rates is an expert field just like design, SEO and copywriting. Yet some simply can’t afford to employ the experts in every micro field of the web. So frequently it’s left to the web designer or developer to learn the skills for every field in order to please a client. A/B testing gives us the ability to be conversion rate-people and help both clients and ourselves obtain higher conversions.

A/B testing affords us a safe environment where the changes we make can be reversed if they don’t perform as we expected. Over time A/B testing will give you the ability to increase a website’s conversion rate, and thus the revenue generated by the site, if you can achieve this your clients will be most appreciative.


Iconfinder Coupon Code and Review

Iconfinder offers over 1.5 million beautiful icons for creative professionals to use in websites, apps, and printed publications. Whatever your project, you’re sure to find an icon or icon…

WP Engine Coupon

Considered by many to be the best managed hosting for WordPress out there, WP Engine offers superior technology and customer support in order to keep your WordPress sites secure…

InMotion Hosting Coupon Code

InMotion Hosting has been a top rated CNET hosting company for over 14 years so you know you’ll be getting good service and won’t be risking your hosting company…

SiteGround Coupon: 60% OFF

SiteGround offers a number of hosting solutions and services for including shared hosting, cloud hosting, dedicated servers, reseller hosting, enterprise hosting, and WordPress and Joomla specific hosting.