Meet Mister Binomial Confidence Interval

Dodgy math and a lot of numbers are a bad mix and paid search provides enough of the former to create ample opportunity for the latter. Like many other forms of online advertising, paid search generates masses of data and often leads some apparent professionals to confuse information with insights. Like many other forms of online advertising, data is easily available through platforms like Adwords which it leads some apparent professionals to confuse information with insights.

Recommendations for account expansion based on historic placement or search query reports filtered for cost per acquisition (CPA) or conversion rate (CVR) isn’t as useful as it may seem. In practice these lists are mostly populated by junk. Placements and queries that just happened to get their one conversion a year within the reporting period on so little traffic it manages to match the criteria used. This isn’t data, and it certainly isn’t an insight. But it can cost you money.

Math to the Rescue!

Fortunately there are simple, robust tools you can used to deal with this. Tools able to find what is likely to work and identify the riskier options. It’s called math. Numeracy and a familiarity with statistics and probability can be useful in marketing. In the scenario outlined above, the biggest issue is determining how close the data collected reflects reality.

Discovering how frequently a conversion occured is easy. Strictly speaking, a conversion rate is a measure of what has happened. Just because Keyword F got a conversion rate of 50% from the sample below does not mean it would do this again over 10 or 100 clicks. This holds true for any of the examples below. While this can be an indication of future performance, assumuing a large enough sample and that nothing changes, as the number of clicks go down, so does the reliability of this number.

How Wrong are You?

While this data won’t necessarily tell you what to expect for the next 10, 20 or 100 clicks, you can determine within what range the actual conversion rate would be. Confidence Intervals can be used to determine the range within which the actual success rate is likely to be 95% (or 50%, or 99%, or whatever) of the time.

Calculating these ranges was done using Wolfram Alpha’s Confidence interval for binomial tool. Unsurprisingly, the smaller the sample used, the greater the range.

Use Numbers Better

There is no excuse for getting this wrong. Not with the easy availability of these kinds of tools and the information to use them. Probability and Statistics is very much relevant to analysing and managing paid search campaigns, much in the same way that Economics can be useful too. Be it constructing a formula in a spreadsheet or using an online calculator, looking a little bit harder at the data you seek to base decisions on isn’t all that hard.