The real question is how much and for what. Friends and acquaintances need personal information to maintain relationships; Facebook requires it for access to their platform. An email address is standard currency for admittance into most social websites and Google uses location, search history and much more to provide a more meaningful search tool and more relevant ads.

Google’s newest product, Google Instant, also relies on this user data. The live blog of the launch event posted on Readwriteweb.com hinted at past user searches being used to generate SERPs as the user enters their query for very real performance benefits.

How does Google scale this? Optimizations: prioritizes searches that are the most likely. Checks if users is doing searches on another server. Results cached.

Where this deal breaks down is if the information is used in a way the user was not expecting. Entering an email address to post a comment is no big deal, but if it is then used to distribute spam, there is a problem. This surprise and betrayal of expectations is a large part of why Facebook’s users reacted badly to most of the site’s changes, and the same again with Google Buzz. The other issue is a lack of universal agreed value.

One degree from evil

It is the fluid nature of these transactions that has created campaigns and conversations like these. One of the more interesting exchanges was between this defense of Google and this subsequent rebuttal.

Users will always exchange their private information for benefits. How much they are willing to give up will vary from person to person. One thing is consistent though: expectations for how the information will be used need to be set, managed and met, or they might find that what you have is just not worth the price.

Slash the web

What makes Blekko unique in an industry dominated by a giant and littered with the corpses of both startups and mature brands is user driven search segmentation, through something called slashtags.

Slashtags limit a search to a preselected group of sites or APIs, determine how the results are sorted, the information displayed ( /rank is the prime example ) and much more. Slashtags can be private or public, and are transparent in what they do to the search results. There are a number of broad catagories that slashtags fall into: User, Built-in and Topic. Built-in slashtags include tools such as /date (listing by date), /noporn (remove porn), and /weather (five day weather forecasts for a location) and a number of other tools, filters and third party APIs like /twitter.

User generated slashtags can be made to search within a specific group of sites and other slashtags. User created slashtags can be ether private or public. Public slashtags can be seen and used by other users.

Why things rank

If you want to know what influences the rankings of a particular query, Blekko will tell you. There is a built-in slashtag, /rank, which lists ranking criteria and weighting available in their search. Known in and outbound links are also available on any domain within the search engine.

When you consider their founding principles, this is not too extraordinary.

1. Search shall be open
2. Search results shall involve people
3. Ranking data shall not be kept secret
4. Web data shall be readily available
5. There is no one-size-fits-all for search
6. Advanced search shall be accessible
7. Search engine tools shall be open to all
8. Search & community go hand-in-hand
9. Spam does not belong in search results
10. Privacy of searchers shall not be violated

I suspect that the Blekko SEO and link tools will become as popular as Yahoo! Site Explorer used to be for SEO analysis. The fact that Blekko is no more likely to replicate how Google crawls the web and what content and links it finds than Yahoo! was won’t dull their enthusiasm.

Why Blekko?

Blekko is far more transparent in what influences which sites appear in the Search Engine Result Page (SERP). Google is providing a more personalised search experience, and most users are not in a position to judge why something ranks and how much their own behaviour has influenced its position.

Slashtags will probably be the killer app for Blekko. Creating specific sharable groupings of sites for searching has a lot of applications for managing information and removing content farms from a SERP when they are not wanted. Adding an element of human curation to search is very cool and deserves at least one more follow up post.

Winning at PVP in EVE Online

Wining at Spreadsheets in Space

PVP in EVE Online is not fair. In fact the challenge in PVP in EVE Online is in setting up these unfair encounters. In most MMORPGs, the actual act of combat consists of a few mouse clicks and some waiting. EVE Online is no different. It is the risk of losing stuff that makes players focus on everything before the actual combat a lot more. It is taking the right mix of ships, avoiding being out-numbered and cornered by a superior foe and acting before the opponent even knows they are in a fight where player skill starts to make a real difference.

Why SEM is like EVE Online PVP

Search Engine Marketing (SEM) in very similar to PVP. It is a zero-sum environment where operators compete for a resource through actions governed by a set of rules and environmental factors generated through user behaviour. There are a few principles that carry over from EVE Online PVP to SEM.

• Situational awareness is king
  • Know how the advertising network works
  • Understand competitive activity
  • Understand how the market behaves
• Observe, act and assess
  • Analysis without an accompanying action is useless
  • Assess the effectiveness of activity & reassess decision making model
  • And repeat…
• Know where you can compete and where you can’t
  • Don’t waste time & resources competing directly with advertisers intent on outspending you
  • Find alternative ways of reaching potential customers.

Information is the key. Understanding how the query space works, having good situational awareness, and knowing where in the sales funnel certain terms are is valuable. It won’t save you from the SEM equivalent of a gate camp (high margin and ‘branding’ campaigns with large budgets), but it is essential for remaining competitive without burning through your budget.

Google has just announced that Ad Sitelinks are now available for any campaign and the introduction of seller rating extensions. Getting Ad Sitelinks to display is still dependent on the ad meeting certain criteria, most probably relating to the spot on the page the ad displays in as determined by bid, competition and click through.

Ad Sitelinks provide alternative points of entry to the site aside from the main landing page, allowing the advertiser to offer alternative offers. Links to areas such as “Store Location”, “Quote Calculator” or “Product Reviews” can appeal to viewers who might not have responded to the main offer.

Search can be a powerful indication of intent, but with more generic terms, the action or information sought by the market can vary between individuals. With branded or descriptive terms the visitor might be seeking further information beyond the noun and a price point. Ad Sitelinks makes it possible to provide this at first glance, and generate traffic where the user may otherwise have continued on to the organic listings.

Using the same visual language of the organic results also makes the ad appear more authoritative and trustworthy. The addition of seller ratings can create the same effect. Creating richer content in a format closer to what the user is trained to look for on a SERP gives the advertiser the opportunity to disrupt their normal search and engage with the ad.

Bing has announced some new features in search this week. Explained in the Bing Gets a Fresh Look post, content themed on Auto, Finance and Entertainment information was used to demonstrate “great new decision-making tools in [these] areas”. Similar to the enhanced Search Engine Result Pages (SERPs) served by Google and Bing, these new tools take search closer to portals in their content consumption model.

In many ways SERPs are becoming more like portal content pages. Search goes beyond an ordered directory of URLs and descriptions. Content including maps, scanned books, images, video, news, blogs, product listings, and social media material is indexed and increasingly being presented in the main SERP as a part of an enhanced search result.

Indexing Entertainment Media

The Internet is an important source of entertainment, either supporting content available through other channels or material native to the web. As said by Bing’s Yusuf Mehdi in his “A New Entertainment Experience For Bing” post:

“In the field of entertainment, 76 percent of people use search to help find and navigate their entertainment options online, but only 10 percent say they have a trusted place to go.”

Entertainment information, from song lyrics to game trailers to film reviews, matters to the user, and can take many different forms. The challenge is in organising it in a human friendly way, similar to what has already started to happen with geographic information and maps. The search engine can programmatically add more content in locations under their control in a theoretically infinitely scalable fashion.

Search in the Music Business

Google is also in the entertainment business. Beyond their search properties and YouTube, Google’s acquisition of Simplifymedia during May hints strongly at Google directly entering the music business. It is likely that Google would add their product to the search experience in the same way as Google Commerce feeds (formerly Google Base) are added to the index or through applications in Chrome or Android.

Lady Gaga on Bing

Lady Gaga on Yahoo! Music

Search is starting to provide a portal-like experience. As the search experience becomes richer, more personalised and more aware of the location of the user, it gets closer to providing the experience sites like Yahoo! do, but with finer levels of customisation through queries. Pages like Yahoo! Music may soon be eclipsed by pages like Bing Entertainment or the current Bing Lady Gaga SERP. In the end it will be how the consumer prefers to consume and seek out content that will determine this. It is passive consumption versus searching with intent.

The data generated by a campaign is not a true representation of the population. The data is a snapshot limited by the markets targeted and the visibility available for the budget spent. Any single campaign can be exposed to direct competition over the whole market or specific subgroups. For example, just because “Campaign A” does not get traffic from Victoria does not mean that no-one in that state is searching for “Keyword B”.

A competitor could simply be focused on that market and value the traffic more. Other factors to consider are the effectiveness of the competition’s creatives and offers, the appeal of their product, efficiency of their site in turning clicks into sales and how much they return per conversion. All of these factors will influence their budget, and how much they are willing to spend per click or impression. Tools provided by the advertising networks that increase the efficiency of campaigns like Remarketing are also worth considering.

According to Wikipedia, a confidence interval is defined as:

…a particular kind of interval estimate of a population parameter. Instead of estimating the parameter by a single value, an interval likely to include the parameter is given. Thus, confidence intervals are used to indicate the reliability of an estimate. How likely the interval is to contain the parameter is determined by the confidence level or confidence coefficient. Increasing the desired confidence level will widen the confidence interval.

In use here, it is assumed that between similar competitors, the average Cost Per Acquisition (CPA) within the group is likely to be within a 95% confidence interval of the known CPA.

Confidence interval and estimated CPA

Confidence interval can give you an estimate of what other bidders may be paying for a conversion, assuming they are operating as efficiently as you are. In the graph included above, confidence interval of the CPA is used to estimate the most likely highest possible CPA a campaign can still compete on. In conjunction with Cost Per Click data, it is fair to assume that the competitors in the query space are willing to spend over the highest likely observed CPA. Reasons for their bidding strategy can vary from shutting out competitors by absorbing a short term loss, to a higher sustainable CPA. In a query space where a number of different verticals are competing for the same traffic, this metric is considerably less useful and your mileage may vary. For comparing CPA campaigns, creating a model for understanding the market, or simply to assess which ads are potentially performing a lot better or worse than your target in the face of direct competition, it is a useful tool.

Confidence interval can be a guide to how much your competitors expect to spend per conversion, assuming a lot of similarity in product and business practices. Arbitrage and industries with heavily commodified products are prime candidates for this, as well as campaigns with a very aggressive high cost bidding strategy, such as those competing directly with another member of your industry.

Even through the web, the methods used to locate and consume information are diverse. From search to portals to socially generated recommendations, there is a huge range of navigational nodes online that shape the user’s experience. Focusing on what information is consumed and where, rather than on what the user is trying to do, can be very myopic.

Find music with only the Internet, Click for full size.

For example, what if the user wants to listen to music? With access to a computer and a browser they may go to YouTube, or the site of a band recommended in an email from a friend. With P2P software like a BitTorrent client they may use inbuilt search tools and download it via a peer network. They could also use iTunes to find, choose and purchase a song without touching a browser. Ultimately all these methods use the Internet, but only one is dependent on the World Wide Web.  This does not even start to consider devices other than a traditional computer.

Nodes and Friction

Each mode of content location and acquisition uses a different set of nodes and they can range from invisible to obstructive. Each one is another opportunity for the gatekeeper of the node to create friction and shape experience. Search and social sites have contextual advertising, Internet Explorer treats incorrect URLs as search queries, DNS services can redirect mistyped or incorrect URLs and the iPad does not support Flash.

Nodes such as portal sites, search engines, social networks and applications such as Steam control and direct attention in different ways. Each gives the user different tools to discover content, with different levels of friction placed between the user and what they wish to do.

Some sites use a disruptive model and place ads in front of the user, using available data to tailor their ads. Applications like iTunes and Steam operate as shopfronts and work to minimalise frictions between the user and the buy button. They help the user find, acquire and consume the media with the least effort, and generate sales in the process.

Why attention matters more

The internet is an environment where there is almost limitless content and space to display it in. The scarcity is with attention. The limits on the size of the audience and the amount of time they spend online are far more immediate than potential advertising inventory. Unlike traditional media, the Internet does not have a page limit, nor is it restricted by spectrum or the number of hours in a day. The low cost of storing and moving data, the asynchronous nature of most content and the ability to generate more content automatically or through user activity changes its value. There is no value in just existing; there is no ‘only two papers in town’ or ‘only three TV networks’ online. Online, the value of a node is in how much attention it affects. Each one is an opportunity to distribute attention among the next group of nodes in the chain.

Why the Task Model

How most people use and access the Internet has changed over the last few years. The ubiquity of Internet capable devices is as significant a factor as the prevalence of fast and wireless home connections. While the Internet on a phone in some form is not new, large numbers of people with fast and easy access is. A proliferation of applications designed to give access to content independent of the World Wide Web is significant too.

Social networks, both formalised like forums and Facebook, and ad hoc such as email, will remain a factor, as well as portal sites like Yahoo! and search engines. These tools for content distribution and discovery are not being replaced, they are just being supplemented.

The user’s aims and knowledge determine their actions online. The channels they use do have an impact on the kinds of information they access, how they access it and how they locate new material. As the Internet becomes richer in content and tools, it will also continue to fragment and change. We have gone a long way from the Internet being tied to a desktop computer through just a browser or email client, but the user will always have a want or desire that they wish to meet, and they will use whatever tools are available to do it.

When you look at the phrases that actually trigger your ads, such as in an Adwords Search Query Performance report, you will start to see patterns emerge, the most interesting being terms that are common across phrases that triggered conversions. Taking a closer look at Clusters of Queries like these can reveal a lot. Most SEM campaigns will have a subset of keywords that might not even be set up in the account itself that correlate strongly with sales. There will also be phrases or words that obviously do not fit or inflate the campaign’s CPC by crossing over into another industry with an aggressive SEM space.

Keyword performance will always be influenced by a number of factors. The effect will vary depending on other descriptive and qualifying terms. Some queries will naturally return more on investment, yield more traffic or be genuinely cheaper. Not all of this will be the product of consumer behaviour. Competition within the query space is incredibly important, especially once you start to examine the performance of specific clusters of search terms.

For example, suppose a pet shop called ‘I Like Turtles’ has set up a campaign to sell turtles. The landing page, with a strong call to action and a robust cart has been built, and a campaign built around phrase match on species terms like ‘Box Turtles’ has been launched. The traffic is not performing as well as expected. The CPC is unsustainable and the traffic is not engaging on the landing page at all. A quick look at the searches triggering the ads suggest a long list of additional negatives and reveals that some descriptive terms are performing better than others, while some are not even appearing at all. The same is true for locations. Reported first page bid levels can provide a guide to what is happening in the market, but it won’t reveal the reason.

What ads are triggered for a long search phrase?

There are a number of different ways one campaign can compete with others in the same query space by either bidding directly on the same keywords or bidding on other keywords that appear in ad triggering search phrases. There are a few that can create a skewed impression of the query space for any single account. Campaigns targeting specific areas within a larger query space for another coupled with specific geographical regions, devices and time periods can aggressively reduce the larger campaign’s visibility in otherwise profitable spaces.

Either through less wasted coverage leading to a better return or a more aggressive bidding strategy one can lock a more general campaign out of lucrative query groups. These competitors do not even have to be selling the same product to the same people; there can be a lot of overlap in terms used for the SEM campaigns of different businesses.

For ‘I Like Turtles’, their targeted query space may also have a toy store using “Buy Turtles”, travel and accomodation companies targeting the name of their city, a conservation organisation raising awareness on river conservation and a DVD store selling Ninja Turtles movies. These companies do not need to be selling the same product to have an impact on the ‘I Like Turtles’ SEM campaign. Travel companies may bid higher than the pet shop and be less precise with their negatives, and increase the CPC. The DVD store might time their ads to run for the weekend, and periodically bury the pet shop’s ads, and the conservation company and toy store may increase competition on terms that often form a part of a search that would trigger an ad for ‘I Like Turtles’. Each of these actions can increase the effective cost of bidding for the campaign and make it harder to appear for one of the higher converting search strings.

Increasing bids around peak buying times can place ads in front of people more likely to buy for certain products. Any increase in return on traffic can increase the amount spent per click. The same applies for more efficient landing pages and sites, and effective targeting geographically and by device. The more likely a sale, the more that can be spent per click.

This activity within a more general campaign can have an effect on how effective certain query clusters may seem. It can also reduce a campaign’s visibility for certain terms. A Search Query Performance report does not tell the full story of what is happening with your campaign and the targeted terms, but it can hint at competitive activity.

Organic factors such as user interest and active social behaviour has always influenced what a person might see and experience online. Someone with no interest or no friends who are interested in esoteric information like Babylon 5 are not likely to hear much about it if they do not actively seek it. There is less chance of being exposed to information they have no interest in on the Internet than in most mass media. Of course the larger the social network of the individual and the greater their direct involvement, the more information outside of their immediate sphere of interest they will be exposed to.

None of these factors are unique to the Internet. The tools available online make it possible to interact with more people in some way then was even possible before. The speed and diversity of content that can be shared online has no parallel in history, but ultimately, it’s just people being people online. What has become increasingly important is the influence of location, device, software and sites or platforms that actively use user generated data to shape your online experience.

The technology to change what is shown by IP, cookies, logged in accounts, OS and browsers are not a new innovation. Their implementation online is becoming more apparent with more obvious use and a proliferation of Internet capable devices in the population. This trend covers commercial sites, social media, news and search engines. It affects content from advertising, articles through to search listings.

Personalised Search

Currently, one of the most interesting things about Personalised Search is the averages users complete ignorance of it. Personalisation of content thought to be consistent for all who access it will have interesting social ramifications. Most users are not actually aware that their own behaviour, and at some point the behaviour of people they are connected to through their Google products, will have an effect on what appears where in their search results.

Google has for a lot of people become a portal, with users retrieving information through the search box with keywords they have learnt, or told to use. This change in user experience of information retrieval for sites other than brand and generic terms may discourage users from being so totally dependant on Google Search acting as a replacement for sharing and directing accessing URLs.

Cross Platform Content Consumption

Not all content works on all platforms. Mobile browsers are far more sophisticated than they were when WAP was the standard, and most web content is now easily accessible on mobile devices, with a few notable exceptions such as flash. Due to differences in screen size and interface some sites will serve a different site to different devices.

Location Aware

With IP addresses, the ability to serve different content to users from different locations has been available for ages. No where is this more apparent then in search. What is new is how location aware applications are now using device APIs to access information from the GPS chip. This location data makes it possible to serve information based on a far more granular level than is possible through IP addresses.

User Experience and Advertising

Delivering the right message to the right person in the right place at the right time is as important for advertising as it can be for sharing information. Delivering relevant information from trusted sources in the right place and time to a user who has demonstrated an interest does go a long way towards managing the huge volume of information available. There is a cost associated with this, including privacy and an increasingly myopic view on the Internet, especially with content that is currently assumed by the average user to be consistent for all.

By default, Content Network bids will be on all advertising space on the site, both above and below the fold. To display above the fold, below the fold placements need to be excluded and bids made for placement on the whole site need to be beaten.

On the Adwords blog post explaining this change it was stated that:

Our goal with this release is to give brand advertisers greater control over where their ads appear, and make the Google Content Network an even more powerful, controlled environment for running high performing brand campaigns.

In practice this will increase the perceived value of one form of placement over the other. A direct result of this will be a concentration of market participants, and allocated budget competing for one of the two kinds of placements. The above the fold placements will be seen as the more valuable of the two, and as a result, the average cost per click will rise. Many advertisers will diversify their campaigns and bid at different levels on both above the fold only placements and whole site placements for as long as they see value in doing so.

Content Network Above and Below the Fold

There is also a shift towards using online advertising in branding campaigns. With a greater perceived value in search and display advertising for promoting brand building content, the value of certain traffic sources has been inflated. Google Adwords has talked about branding and search marketing a few times already. By selling advertising on branding value and separating the value of an ad from an incremental per sale return increases the amount of money that most organisations can justify internally on paying more for impressions and clicks.

By leveraging different perceptions of value created by these changes to the Content Network, Google Adwords is increasing competition and consequently their margin per click. Separating cost per click from the profit margin on conversion for some markets in the minds of advertisers will also raise the perceived value of impressions and clicks on both search and websites.This trend will increase the actual value of traffic in a market where there is very little competition among suppliers.

Ironically it was Google with their entry into the market that created that first shift towards linking cost of traffic to profit from sale. The introduction of Analytics and Adwords along with using Adsense to grow their inventory were the main drivers in this shift for most marketers new to advertising online.