Talk abstract: The Madness of A/B Testing. A/B testing is a useful tool, which allows us to determine the effects a particular change will have in practice. It controls for nuisance factors influencing player behaviour by randomising allocation between test and control. However, relatively small treatment effects can be difficult to detect against relatively large background noise. Furthermore, engagement may rise initially due to temporary curiosity or may not rise initially due to users needing time to familiarize themselves with a new feature. In this presentation, we will address these issues and discuss the A/B testing methodology that is used in Product Madness.
Bio: As a Data Scientist at Product Madness, Susan uses data to help gain insights, drive decisions and deliver business value to the company. Susan has a special interest in applying statistical methods to real life data. She has worked across multiple industries, working with interdisciplinary teams. She holds a PhD degree in Biostatistics from Erasmus University, Rotterdam.