A/B Testing (Split Testing)

What is A/B Testing or Split Testing?

A/B testing (sometimes known as Split Testing) is comparing two products or ideas in market research studies to see which one performs better. By showing respondents individually or side by side –  the one that gives a better conversion rate wins!

The roots of this tried and trusted testing technique was from the classic direct mail tests where two versions of the same mailing are sent out to households in order to see which one gets the best returns. The approach is based on randomly assigning respondents to test one of the two products/ideas – Product A and Product B.

A/B testing is a very reliable approach in the development and evaluation of new ideas and in recent years has become an important tool in website design and web marketing. Vision One incorporate A/B testing into a number of their innovative research products, including; PackProbe (packaging design testing tool) and IdeaProbe our concept testing and new product development (npd) tool.

Typically A/B testing is used towards the end of the development and optimisation process, where ideas have been reduced to a small number (manageable) of products or concepts or communication components which need final confirmation one way or the other. Whilst you can test virtually anything in your concert or marketing materials: headlines, call to action, body copy, images, colours, layout etc – focus on the things that are most likely to have a big impact otherwise you could end up testing ad infinitum.

The disadvantages of A/B testing can be that it is not always obvious why a particular design/concept/idea is preferred. Clearly it is only designed for testing s small number of ideas – but by incorporating diagnostic questions into quantitative research studies (or supporting with qualitative research) can help fill the information gap and highlight any necessary modifications to ensure the best possible decision. For further reading check out Wikipedia.