In digital marketing, the common technique to compare two variants of web pages is to utilize A/B testing for the purpose of determining which is more effective (“A/B Testing”, 2019). A/B testing can also be used to increase ROI, reduce bounce rates for sites, introduce low-risk changes and implement re-designs in an effective manner (Rawat, 2019).
The following are the steps to create an A/B testing scenario, and compare the steps to the example asked by this post (“A/B Testing”, 2019):
1st: We need to identify the conversion to improve, in this case, it is the best promotion
2nd: Create a hypothesis, in this case, it is that promotion A generates more revenue than promotion B
3rd: Identify the variables and conversations – promotion A and B, and the sales generated by each promotion
4th: Run the experiment – release the test websites to 1,000 users for A and 1,000 users for B
5th: Measure results – more information below.
Once you have collected all the data, the statistical model that is used to analyze the results can vary depending on the type of assumed distribution (Truong, 2018). As this A/B testing scenario has an assumed distribution of a normal or Gaussian distribution, it would be best to utilize a Welch’s t-test or a Student’s t-test. The results show a t-distribution value which can test the null hypothesis identified in steps 2 and 3.
Resources
A/B Testing. (2019). Retrieved from https://www.optimizely.com/optimization-glossary/ab-testing/
Rawat, S. (2019, October 24). A/B Testing: Introduction & A Complete Guide [2019]: VWO. Retrieved from https://vwo.com/ab-testing/.
Truong, S. (2018, October 11). The art of A/B testing. Retrieved from https://towardsdatascience.com/the-art-of-a-b-testing-5a10c9bb70a4
Comments