When can you start to assess and optimise online advertising campaigns in a meaningful way? Adwords, Yahoo! Search Marketing, LinkedIn and Facebook ads allow for an amazing level of market segmentation. Small, highly specific populations can be targeted by a large number of different variables. By their very nature these highly specific campaigns sometimes only generate low levels of traffic and sales, and consequently have a high level of apparent volatility over short time periods.
This volatility is an interesting challenge for account management, and can create a risk in responding to rapid changes. Low traffic and conversion numbers make it difficult to collect meaningful data over short time periods, making it hard to tell the difference between an emerging negative trend and an outlier. This creates problems in both managing low activity campaigns, campaigns in highly competitive and volatile markets, and new campaign testing.
Averages, Standard deviations and Confidence intervals
Averages, standard deviation and confidence intervals are a few statistical tools available for analysing the data. The actual figures used to determine response will vary from campaign to campaign even for the same product, due to other factors such as the size of the audience and the means used to reach them. The tools used to explore the information and create heuristics for guiding analysis often will not change.
The sample data tracks a gradual upwards trend in the average cost per conversion in a focused Adwords campaign. There is an outlier at double the mean that skews the mean and standard deviation in the third reporting period. Normally this would be discarded, as its apparent effect on both the reporting period and ongoing mean and standard deviation is significant.
It is only on the fourth reporting period that the data starts to become consistent. While there is still some volatility in each reporting period, the reporting period mean remains within one standard deviation of the ongoing mean. The hypothesis that the ongoing campaign mean at four reporting periods can be used as a guide for this campaign is supported by the confidence interval of the whole campaign data set. In the example campaign, it can be assumed that after four reporting periods, there will be enough information to make decisions regarding optimising this campaign.
The figures based on the whole campaign can be used as a guide for assessing the effectiveness of specific ads, placements and keywords while minimising the risk of removing a creative that can still be productive. These metrics provide a guide for deciding when a campaign, keyword or creative needs direct intervention, or may just be having a bad week.
The model that you create using data from previous and current campaigns is ultimately only a guide. It can be used as a framework for assessing creatives and traffic, but these heuristics will only ever be as good as the data they are derived from. There is need to review of any model used to guide the decision making process periodically. The market is constantly changing, be it SEM, social or display advertising. Factors such as seasonality of the product, external environmental factors and competitor activity can have a significant impact.
