Was Bing Cheating or Following Policy?
While trying to get my head around the whole “Google caught Bing cheating” I found myself asking more and more questions.
The first question was when Google ran the experiment they say around 20 Google engineers went home and ran the test queries using Internet Explorer with a Bing toolbar installed and the Suggested Sites enabled. They also had to click on the top results in Google ( or the Fake results )
What no one is reporting is how these test queries where actually run ?? I guess one of two ways.
A) They opened IE8, typed in www.google.com then entered the search query ( How many times did they search in Google, 1, 10, 100, 10,000 times) then Clicked on the fake result but we don’t know how many clicks each engineer did on the fake listing?
B) They set the default search engine in IE8 to be Google and started searching for the Fake query :
The first question is really important to me because it actually makes me lean more to Bing doing nothing wrong if it’s actually turns out to be option B and here is why.
The set up that Google actually created, Microsoft states “statistics about your usage of Suggested Sites will also be sent to Microsoft such as the time that websites were visited, which website referred you, and how you got there”
In fact read the whole Privacy policy here http://www.microsoft.com/windows/internet-explorer/privacy.aspx Suggested Sites is quite near the bottom of the page.
so how does all this work ( NB I haven’t a clue ;)) but if I had to guess :
an IE8 user which has agreed to send data to Bing to improve the search quality enters a query, Bing then check the database to see what they have for that term ( I guess if nothing returned then Bing works hard to improve it’s quality ) so I search for Dave Naylor and Bing monitors which sites I visit :
My search moves to the default search engine I have set in IE
http://www.google.com/search?q=dave+naylor&sourceid=ie7&rls=com.microsoft:en-gb:IE-Address&ie=&oe=
I click on #1 result http://www.davidnaylor.co.uk/ I hit the back button within seconds
returning me to : http://www.google.com/search?q=dave+naylor&sourceid=ie7&rls=com.microsoft:en-gb:IE-Address&ie=&oe=
I scan the results and find the twitter feed for dave naylor and click
http://twitter.com/davenaylor, I read a few tweets and click back, not what I was looking for, click back again
http://www.google.com/search?q=dave+naylor&sourceid=ie7&rls=com.microsoft:en-gb:IE-Address&ie=&oe= I see the Globe and Mail and click that
http://www.theglobeandmail.com/blogs/from-the-sidelines/ If I was looking for the sportswriter David Naylor I would stay here longer and read the articles etc..
In Bing’s eyes I searched in IE and visited 3 sites excluding Google
http://www.davidnaylor.co.uk/ – time on site 1 second
http://twitter.com/davenaylor – time on site 3 seconds
http://www.theglobeandmail.com/blogs/from-the-sidelines/ time on site 180 seconds
If people searching for “Dave Naylor” actually wanted the Globe and not me then a quality signal would have been the time on site which Bing actually says they track. Now I agree that it would look like Bing is using Google to get signals to improve their results but if they are doing it on any search engine which is set to default I would have to say they aren’t copying Google, they are using User click stream. A better test would have been to setup a fake search engine and fake the results see if they appeared in Bing as well 
But What if the answer was A, Bing could have just set a script to monitor anything that gets sent with the Url http://www.google.com/search?q=dave+naylor and then follow the data stream. This is a little more naughty but still not copying Google. The fact that Google created fake data that no one had seen before, then created fake clickstream data and sent it to Bing (that says that the data will be used to personalize your experience, as well as improve the quality of our products and services.)
Either way one thing Google has done is given us a way to game Bing and how much monitoring of Bing did Google do and how did they monitor them ?
7 Comments
Bob - http://example.com
Google proved that Bing Bar worked. Big deal…
Mike Glover - http://www.click-finders.com/blog
I have to agree here. Bing posted all vital information in their privacy policy and to be honest, it’s a KNOWN fact that all of the search engines gather and use information from each of their respective tool bars.
PR Stunts…UGH !
I posted more thoughts here in my SEO Blog if anyone wants to read more of my RANT.
LordManley - http://twitter.com/?status=@lordmanley
Clearly not cheating – frankly I found the Google statement a bit embarrassing. If it was any other company then we would be ridiculing them for being childish, but everyone loves to hate M$
Sam Crocker - http://samuelcrocker.com/blog
Hey Dave- nice points here. Like you, the part that I found most interesting/confusing/annoying was the fact that G never publicly said how these tests were run and seemed as if they were trying to create confusion and insinuate things were much shadier than possible. I also think some of the earlier stories that pulled just headlines from the more in depth articles were a bit irresponsible in the way they presented the information.
I think it is important that Bing are/feel the need to look at user behaviour within Google’s pages rather than their own (reasonable I guess given market share) but it seems like it can only be a good thing to be trying to provide the best/most relevant results and calculating this based on a number of factors.
It certainly looks like we can add CTR to the list of acknowledged ranking factors rather than suspected ones
David Fairhurst - http://www.intelligentretail.co.uk
I’d have to agree Dave, remember that stunt a few years ago where one company set out T&C’s that they owned your soul in perpetuity? Did anyone bother to read / say ‘no I don’t agree’ to these? Did they hell!
Big ‘G’ have dropped a major booboo with this one and I think they would do well to remember all that lovely data they are collecting from their OWN users on a by-the-millisecond basis… is Google really now saying some of this data doesn’t sneek into their own algo’s to improve results? Franky that’s like saying the Pope is Jewish!
Beth Wood - http://www.bethsbyte.com
This reminds me of the movie “Enemy of the State”. Who is going to “monitor the monitors”?
Dan - http://www.danieldeceuster.com
Forget everything you have just mentioned and think from a purely scientific and statistical standpoint- what is the one thing required to make any test or study valid? Replication! Has anyone replicated these results Google found? Has any one been able to replicate the results with non-gibberish phrases? No one ever points out that Google did this “test” with success only on complete gibberish phrases. This doesn’t prove anything.
Remember a couple years ago when Yahoo said they no longer supported the keywords meta tag? Then someone put gibberish in their keywords meta tag and ranked for that gibberish in Yahoo? Then Yahoo had to revisit their statement and said they don’t use the keywords tag except in the absence of any other ranking factors. Obviously no ranking factors for gibberish.
This is the same way. So Bing relies on data collected from users at Google on some random gibberish phrases. Who cares? No one has been able to get the same results that I’ve heard of yet and no one, even Google, has gotten these results using real phrases. The whole thing is pretty stupid if you ask me.