What has surprised me recently is how unscientific these claims can be sometimes. Some bad statisticians behind the numbers. The sort of people that would flip a coin 100 times, get heads 57 times and tails 43 times, and come to the conclusion that heads outperforms tails by about 30 percent.

I assume KISSMetrics used a large enough sample size in their experiment for their results to be statistically meaningful, but some recent sites I’ve come across online that purely focus on A/B testing have draw conclusions from tests with tiny sample sizes and tiny percentage differences.

Written by

Founder: Skilled.co.il & Draft Fantasy. Full stack JavaScript Engineer. https://elie.tech

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store