An upcoming change to Google.com search referrals; Google Analytics unaffected
Tuesday, April 14, 2009 | 2:50 PM
Labels: Announcements
First, just a heads-up that if you don't analyze your own traffic logs, use Urchin web analytics software, or develop web analytics software, you probably don't need to read this post. We're writing this for the most geeky among us, because Google Analytics will not be affected by this information. On the other hand, we do want to let you know about some changes to Google search that are coming down the pike, before you start seeing (potentially) alarming headlines.
Starting this week, you may start seeing a new referring URL format for visitors coming from Google search result pages. Up to now, the usual referrer for clicks on search results for the term "flowers", for example, would be something like this:
http://www.google.com/search?hl=en&q=flowers&btnG=Google+Search
Now you will start seeing some referrer strings that look like this:
http://www.google.com/url?sa=t&source=web&ct=res&cd=7&url=http%3A%2F%2Fwww.example.com%2Fmypage.htm&ei=0SjdSa-1N5O8M_qW8dQN&rct=j&q=flowers&usg=AFQjCNHJXSUh7Vw7oubPaO3tZOzz-F-u_w&sig2=X8uCFh6IoPtnwmvGMULQfw
The key difference between these two urls is that instead of "/search?" the URL contains a "/url?". If you run your own analyses, be sure that you do not depend on the "/search?" portion of the URL to determine if a visit started with an organic search click. Google Analytics does not depend on the "/search?" string in the referrer, so users of Google Analytics will not notice a difference in their reports, but other analytics packages may need to adapt to this change in our referrer string to maintain accurate reports.
The new referrer URLs will initially only occur in a small percentage of searches. You should expect to see old and new forms of the URLs as this change gradually rolls out.
If you are using UTM-based tracking with Urchin Software, you'll want to stay tuned for a software update that we'll be making available soon. If you are using IP-Useragent based tracking with Urchin, you won't be affected since this form of tracking can successfully process both current and new referral strings.
33 comments:
Happy Bear said...
The problem with this change is that people like to click the referrer links in their (non-GA) analytics reports to see the actual search results page. With the new referrer URL, it just shows an info page that you are about to be redirected. I think a lot of people won't be happy about that. I realize you are the analytics team and not the search team but there is no place to give the search team feedback so here I am.
On getclicky.com, I guess we'll just manually change the referrer URL so it links back to /search instead, so that web masters can actually get the useful information they desire.
4:27 PM
Alan Bleiweiss said...
so for any clients I have that may be using the older GA code, will there be a bump in the statistics? Should we go through our client sites and update the code to use the newer version?
Thanks
Alan Bleiweiss
4:31 PM
Phil said...
The "source=web" is going to cause a headache.
I recently encountered a problem using "source=google" on our affiliate campaign where the website we were sending traffic to was using Atlas tags (clk.atdmt.com). It resulted in conversions being attributed to the referring source not the landing page source.
To resolve we avoiding the source= parameter. This change wll mean that sales that we are driving could be attributed to "web" rather than us!
Why did google decided to use source= ? Can the parameter be renames while it is still in beta, to something like “sporce”, “sorce” or “ecruos” (i.e name it anything other than source)
Thanksfully it has not been rolled out to google.co.uk yet so Atlas will hopefully patch :-)
This change could also impact any websites using ppc arbitration, the where the source= is present on both the referring page and landing page.
I am at the GooglePlex tomorrow so will double check this with a GAAC.
Cheers
Phil.
Here is the above in a slightly easier to read format:
------- old
http://www.google.com/search
hl=en
q=flowers
btnG=Google+Search
------- new
http://www.google.com/url
sa=t
source=web
ct=res
cd=7
url=http%3A%2F%2Fwww.example.com%2Fmypage.htm
ei=0SjdSa-1N5O8M_qW8dQN
rct=j
q=flowers
usg=AFQjCNHJXSUh7Vw7oubPaO3tZOzz-F-u_w
sig2=X8uCFh6IoPtnwmvGMULQfw
4:33 PM
Daniel said...
Thanks for the warning.
Why are you moving to a crazy URI format? Doesn't this go against most of the advice Google have given to webmasters recently?
4:43 PM
thelostagency said...
Is this something to make things harder to 3rd party analytics solutions, is it over complicating referrer url?
Will it break?
4:57 PM
Adrian P. said...
Any information on why the switch is taking place?
5:09 PM
Virante Marketing said...
HUZZAH! Thank you very much for this addition, although I imagine a ton of GreaseMonkey scripts that pattern match *.google.com/search* will need to be updated.
7:12 PM
Chris said...
Is there any particular reason for this change? It seems that it only adds bloat to requests, and increases the processing time for the URL on the visited page (if that's your thing).
It's almost as bad as the ViewState variables that the Microsoft AJAX stack passes with every request (I've seen Viewstate variables containing up to 100,000 characters).
Yes the difference will be small - but if you are analysing millions of clicks through, the difference will be significant.
12:13 AM
adrian33 said...
There is an idea I to improve search results here:
http://news.ycombinator.com/item?id=563124
I'd love to help.
12:45 AM
jaamit said...
Thanks for posting about this. Can you please clarify if these new search referral URLs include more data that can will be able to be used by Analytics packages such as:
- Search Result Position #
- Normal web result or Universal search result (local, product, news, blog, video etc)
- Any other new data?
Would be great to hear if any of this is part of the string!
1:01 AM
bsjut said...
Will it be possible to identify organic search clicks that come from the unified search box at the top of a results page (i. e. the are that includes pictures, news, videos, etc. depending on the query)?
Is the new url structure documented anywhere? e. g. what values will be passed along through the "source"-parameter?
Thanks, Bjoern
2:29 AM
bsjut said...
7:41 AM
David Spira said...
What is the benefit for Google?
What is the benefit for the webmaster?
To echo the question that has been raised by others, is they a strategic move to make other analytics tools less usable than GA?
12:57 PM
bukv.net said...
Being a geeky type I would really appreciate the explanation of not-so-obvious parameters in the string.
Thanks!
12:59 PM
Michael Martinez said...
I have long wondered why Google misses so many search referrals in its Analytics data. Now I'm beginning to understand why.
It would be nice to learn more about how Analytics works. We could probably give you feedback on what you're doing wrong to help you improve the product considerably.
Thanks for the heads up nonetheless.
4:15 PM
Andrew said...
The motive here seems clear. Now, Google will be able to redirect all click through to their server, tracking with great precision the click through rate for individual results. This will add to their voluminous amount of data, helping their refine their search results. It's a play to increase relevance, or so it would seem. In the past, I've noticed that only a portion of the results were redirected.
Maybe I'm missing something, but that's my two cents.
5:47 PM
Phil said...
I have previously heard about (from various sources) the intent by Google to roll-out a hash '#' in the search results page URL that would strip the paramaters altogether - is this blog entry implying that this will not happen now?? or is this something different?
8:09 PM
Open Seo said...
The new referral URL, when being clicked shows a specific page, with 2 links. Not a Google search result page.
It means it becomes impossible to replay the search performed by the visitor in the same conditions.
Sounds like we are loosing an opportunity to understand our visitors and and how they use google search.
Additionally, the second link being displayed on this page doesn't work, and the French localization of the message contains a grammar mistake.
Regards
2:56 AM
Andre said...
A statement from google would be nice. Lot's of questions asked, but no answer. Why allow comments in the first place, if you don't respond to them?
4:55 AM
Conference Coordinator Spain said...
Yes, rest assured it is for a better way of Google analysing click data themselves.
5:57 AM
Brett Crosby, Sr. Manager, Google Analytics said...
Sorry for the delay in responding (especially Andre), I was seeking clarification from the search team. A lot of the comments were directed to them and I think it is best to let that team handle those questions. I do want to answer some of the Analytics questions though. Phil's comment above is correct. I posted this because a couple months ago Google tested some search results that added a # into the URL. This created a big problem for people interested in seeing which keywords were driving traffic to their site (anything in the URL after a # doesn't get passed in the referrer... this is particularly a problem for web analytics products), so we worked with the search team to stop that test until they could find a better solution. The announcement above is the answer to that. It allows them to test new search results without the negative side effects (if it introduced others, that was not the intent). Some of the other comments had interesting theories about why we did this (grassy knolls not included), but the goal was to allow new tests of search without making it difficult for analytics products to report on query data.
4:08 PM
antezeta said...
Google has been experimenting with Ajax (JavaScript) based search results. The problem with this initially was that no referrer information was passed, "breaking" the ability of Web Analytics tools to track search referrals.
Google has since found a work-around, which results in the new referrer Url.
One nice enhancement is to pass the search results position, the cd= parameter.
Google's Matt Cutts says the move from basic html to JavaScript is to improve the speed of search results rendering.
I can imagine that another goal is to impede the current crawling/scraping of SERPs for SEO purposes.
This would be fine if Google allowed API access to search results for standalone programs as they use to with the old Soap API. The propriety of useage of the current AJAX Web Search API by standalone programs is not so clear - ironically, you need to send a referring URL... which in the case of a standalone program doesn't really exist.
- Sean Carlos
1:20 AM
Bernt Johansson said...
Will we see all referrals from Google in this new format, and if so will that happen any time soon?
4:21 PM
Locust said...
It seems that the cd var only shows up when a user is logged to his google account...
12:00 PM
Ben Pate said...
I am seeing a querystring variable named "start" that I have confirmed is the rank :) Setting up my filters now!
8:59 AM
Phil said...
In addition to the new google.com/url format there are three others....
google.com/search (old format)
google.com/url (new format)
google.com/ie (mobile search)
google.com/webhp (contains #hash query)
#hash query example: http://www.google.com/webhp#hl=en&q=flowers&btnG=Google+Search&aq=f&oq=flowers&fp=g9hIhDHDw6s
Phil.
4:07 AM
snowstorm said...
Does this have anything to do with
Google Analytics URL Builder - I have setup tracking on many advertising banners thanks to http://www.google.com/support/analytics/bin/answer.py?answer=55578&ctx=sibling however the data analytics produces is not matching the suppliers details by a long way!?
6:35 AM
carol taylor said...
Dear Google,
My spirit is dashed against the rocks. You changed the search engine today and you know exactly what you did. I apologize to all other bloggers but I hate you google, I hate you google, I hate you google, I hate you google, I hate you google, I hate you google.
Carol Taylor
Texas
7:49 PM
tSV75Kl6yNPb.Z1H7UOPJg7v7oq8Xw7q_aZAliBXeA-- said...
Very clever, Google, for using a request scheme that explicitly undermines one of the supporting pillars of the web, the 10 years old specification of URI syntax (http://www.ietf.org/rfc/rfc2396.txt), namely the part that states:
" When a URI reference is used to perform a retrieval action on the identified resource, the optional fragment identifier, separated from the URI by a crosshatch ("#") character, consists of additional reference information to be interpreted by the user agent after the retrieval action has been successfully completed. As such, it is not part of a URI, but is often used in conjunction with a URI.
[...]
A fragment identifier is only meaningful when a URI reference is intended for retrieval and the result of that retrieval is a document for which the identified fragment is consistently defined."
I guess we are back to Altavista to get a search engine that doesn't want to screw you any occasion it gets...
1:10 PM
afewtips.com said...
Since javascript maybe used -
maybe this will help Google allow the user to review results from the back of the pack as easily as the front.
cd=5
cd=609
Display the results in tabbed ranges.
1:03 PM
Justin said...
Does anyone know exactly how much of an impact this change will make to individual user sites statistics.
9:06 PM
SilverMaT Ltd. said...
thanks mate
12:50 PM
renu said...
If google adds API Access, how can it be benficial to us webmasters?
- Renu
12:12 PM
Post a Comment