What is A/B Testing? How to Conduct an A/B Test?

ab testing

What is A/B Testing? How to Conduct an A/B Test?

A/B testing (as known as split testing, bucket testing) is a process of showing different versions of the same web page to users in order to determine which one performs better. In web design and development it is used to compare the effectiveness of different elements on a website. Impact of the changes are tested for reaching a certain goal. A/B test is great to determine what changes you need to make on your website to provide a better customer journey and subsequently increase conversion rate.

A/B testing can be used to optimise user experience from start to finish. Whether its the homepage or a landing page, A/B testing allows you to understand which version of a web page, design, ad or email copy performs better. Typography, size, color, CTAs, images, positioning, are examples of the variable that can be tested.

During an A/B test, effect of a single element is measured at a time such as the title, CTA image, content, etc. (Another testing process which is called multivariate testing is used to compare higher number of elements.)

Control results are needed in A /B testing. Therefore, “A” contains the control results of the current situation, while “B” is the variation where you will compare the results with “A”.

For example, let’s say a product page on your e-commerce website gets several hundred visitors a day. You have decided to do an A/B test where you add an extra information about the delivery next to the “Add to Cart” button. So, using your A/B testing software, you create a similar page and divide the traffic evenly between both pages and measure the results.

How to Conduct an A/B Test?

A/B testing is mostly conducted on websites. Tests on the website can be done on pages such as product pages, landing pages, and checkout pages. Off-site tests are tests on ads, e-mails, social media posts, push notifications.

It is very important to run the tests simultaneously with the same traffic sample. Traffic and timing make up the two biggest variables that can skew results. For example, comparing the results of the two variables is absolutely useless if one is tested on New Year’s and the other on Mother’s Day.

We can measure whether the A/B test is successful or not with a statistical significance level. What is statistical significance?
Statistical significance serves to ensure that the results of a particular test are not caused by sampling error. Statistical significance is used in many different industry and test environments, including academic research, medical tests and more.
Statistical significance in A/B tests is calculated using the number of users and conversion count metrics for each variation. Using statistical significance proves whether the A / B test passes or fails. Ideally, all A/B tests needs to reach 95% statistical significance, or at least 90%. Going above 90% allows the change to affect the performance of a site negatively or positively. The best way to have a statistical significance of over 90% is to test pages with high traffic or high conversion rates.

There is a wide range of A/B testing tools used for different purposes. Such as VWO, Optimizely, Google Optimize, AB Tasty and Clickflow

A successful A/B testing in web design process can bring great benefits. More user engagement, better conversion rate, ease of analysis, and eventually higher sales numbers.

Share this post