Benjamin Pettit


Talk abstract: A/B testing: Setting up for success & choosing the right Statistical Framework. We are all familiar with the benefit that A/B testing can have on businesses’ success, whether that’s improving customer experience or increasing revenue on websites and apps. In this session we’ll talk through tips we’ve learned from running a multitude of A/B tests at Cleo, from design through to implementation and analysis. The concepts of ‘holdouts’ in testing will be introduced, with a focus on how they can help teams track long term impacts to KPIs. Lastly we’ll look at Bayesian vs Frequentist frameworks, the benefits and pitfalls of each, and how we’re exploiting Bayesian statistics to get the most out of our A/B tests at Cleo.

Bio: Benj did his PhD at Oxford modelling the flocking behaviour of birds, before turning his attention to how digital products can learn from masses of data on human behaviour. His career as a data scientist has involved developing recommender systems, classification models, and network analytics tools for several London startups. He has been a senior data scientist at Cleo for the past two years, working on machine learning models used in the Cleo app.

Saturday November 27th , 2021
9:00 am-
5:00 pm
We are BACK! Welcome back to the DSF annual live event. Back for its fifth season, after a 2020 break, we are thrilled to be doing a live-in-person Fest! This free day will feature four-stream rooms with over 600 people attending. A day of lectures with over 40 top speakers.…