A/B testing is like sex at high school
A few days ago I went on record saying that A/B testing is like sex at high school. Everyone talks about it, not very many do it in earnest. I want to follow up on the topic with some additional thoughts (don't worry, I won't stretch the high school analogy any further). When talking to people about A/B testing I've noticed that there are four (stereo) types of mindsets which prevent companies from successfully using split tests as a tool to improve their conversion funnel. 1) Procrastinative The favorite answer to suggestions for website or product improvements from people from this camp is "we'll have to A/B test that" – as in "we should A/B test that, some time, when we've added A/B testing capability". It is often used as an excuse for brushing off ideas for improvement, and the fallacy here is that just because the best way to test assumptions is an A/B test doesn't mean that all assumptions are equally good or likely to be true . Yes, ...