Not-Really-SSL Search and the Disappearing Data

The number of organic keyword referrals marked as ‘(not provided)’ last week was very low. It did not even account for a single percent of what Google sent through organic search. In the aftermath of Google’s SSL Search announcement last week the small proportion of traffic directly affected has been cited in response to the vitriolic reaction the original announcement provoked. As frustrating as losing valuable data is in theory, in practice very little organic traffic is affected.

And for now, this is correct. Currently the only keyword referral data being replaced with ‘(not provided)’ is that generated by users who:

• Search through Google.com,
• While signed into Google accounts, and
• Click on an organic listing (AdWords keyword referrals are still passed).

Internationally the volume of organic search traffic that meets these criteria is very small, and does not account for much USA traffic either. There is very little available data at this point, and most of these assumptions are based on what people are seeing on their own sites (now that it is it is possible to start to measure the impact) and Matt Cutts’ informal estimate of the percentage of searches performed by signed-in users.

Important, not Urgent

The issue is not how much organic search traffic is ‘(not provided)’ now, but what this number might be in the future. SSL Search isn’t an urgent issue, just an important one. Google builds systems for scale, and has stated their intention to take this further:

As part of our commitment to provide a more secure online experience, today we announced that SSL Search on https://www.google.com will become the default experience for signed in users on google.com.

Making SSL Search the default for signed in users won’t be confined to just Google.com forever, and the number of users who remain logged in while using the internet won’t remain the same either. Most of Google’s popular services like Gmail, Google Docs and, optimistically, Google+ work better when you are signed in. Over time there will be more people searching while logged in, not fewer, and SSL Search is certain to be rolled out across more Google TLDs.

Won’t Someone Think of the Users?

It should not be a surprise that Google is working on making a more secure search experience more widely available. Google pays attention to Electronic Frontier Foundation (EFF) and takes the user experience of search seriously. This culture is often reflected by the employees; for example, when SSL search was first introduced by Google in May, 2010, before being moved to https://encrypted.google.com/ in June, Matt Cutts posted A few thoughts on SSL Search on his personal blog on its introduction:

I believe encrypted search is an important option for Google searchers. The Electronic Frontier Foundation (EFF) has asked for secure search in the past (see this post from 2009), and I credit them for helping to put this on Google’s radar. Another inspiration that helped to spark this project was Cory Doctorow’s book “Little Brother.” It was one of my favorite books of 2008 and while I won’t go into the book’s plot here, it’s a quick, fun read. “Little Brother” also makes a compelling case for encrypting HTTP traffic on the web.

Recently, a few ISPs in the USA demonstrated the value of encrypting search referrals for users by redirecting their customers’ search traffic based on the keywords they used to content the ISP selected. In fact, it is the value that SSL Search has for the user that means that Google is providing organisations such as schools a NoSSLSearch option.

Google’s commitment to their users’ privacy and HTTPS Everywhere means that their latest rollout of SSL Search for logged in users is a logical progression, securing their users’ search queries and protecting their privacy. Unless they click on an AdWords listing. If you are paying for clicks, those keyword referrals are still passed through to the advertiser.

Organic Versus Paid Traffic and Data

Despite the Internet’s tendency towards paranoia when reading the motives of large companies, there is actually a good reason why referrals don’t get passed from HTTPS to HTTP sites. As Thomas P from the Webmaster Central Help Forums pointed out:

Clients SHOULD NOT include a Referrer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol. (Source: Hypertext Transfer Protocol — HTTP/1.1)

Following these standards means that sites behind HTTPS would still see keyword referrals. But this is not the case with SSL Search; what Google is doing is a little different. In a comment by Eric Wu on Google+, he suggested that Google is not using HTTPS as you would expect (follow the link, his explanation is well worth reading). Searching via https://google.com/ differs from the standards (unlike https://encrypted.google.com/, which still follows them) in two important ways:

• Organic referrals won’t be passed to sites entirely behind HTTPS
• AdWords referrals are passed to sites on HTTP

Both of these are odd behaviours, and seem at odds with the aim of protecting users’ privacy. Passing AdWords keyword referrer information to advertisers is not all that surprising once you think of Google as a business. Danny Sullivan pointed out on Search Engine Land in his Google Puts A Price On Privacy post that provoking a negative response from the people who made Google nearly $10 billion last quarter would not be good for business.

Information is valuable

It is the loss of data due to SSL Search that concerns most online marketing professionals. Search referrals often indicate changes in how queries are structured, shifts in brand awareness and use, the effectiveness in offline campaigns based on certain phrases or words and give important insights into how language is used to describe products or ideas. Keyword referrer information is important for conversion optimisation and can inform ongoing IA development. As limited as the scope of Google’s initial implementation of SSL Search is, the fear that it will account for a greater share of search as time goes by is legitimate, and has real business implications.

In a number of blog posts Google has suggested that Webmaster Tools data can replace the information lost in Google Analytics, with Webmaster tools query data currently available in Google Analytics. However the information provided by Webmaster Tools is substantially different to the keyword data in Google Analytics. It is restricted to only 1,000 queries, and is reported in general terms. It is useless for assessing search queries of more than two words for any site that receives a decent amount of search traffic. Google Webmaster Tools gives an overview of what is happening within Google, but in its current form, it can not replace Keyword referral information.

This might not actually be a bad thing for Google. Information, especially information on user behaviour, is valuable. Google will still be collecting search data from those using SSL Search; the change simply means that they won’t be sharing anymore. At least not without being paid first.

Balancing Privacy and Business

Google has taken a compromise position on user privacy with this implementation of SSL Search. Even as they recognise the value of HTTPS Everywhere for users, Google has acknowledged that referral data has value for their customers (their advertisers, that is – those that don’t pay money are not customers). The result is a feature that does not really protect users’ privacy or encourage more websites receiving both free and paid Google traffic to implement HTTPS.

It is interesting that Google’s other SSL Search URL, https://encrypted.google.com/, handles referrers differently to https://google.com/. With clicks on HTTP URLs, https://google.com passes a HTTP referrer, stripped of keywords, but https://encrypted.google.com does not. When the destination is a HTTPS URL, https://encrypted.google.com passes referrer information as you would expect, but https://google.com does not, unless it is an AdWords link. There is a more complete post, with a chart, at Google Modifies SSL Behavior — and the Results Are Troubling.

To a cynical outsider, it would seem that the only objective Google has actually achieved with SSL Search is protecting their advertisers’ keyword referrals while earning favourable press for defending the privacy of innocent Google users from those nasty, evil, SEO scammers.