OK to start with, these are is just my thoughts..

Go back 2 years, we where all talking about eval.google.com how a select team of Googlers (external) got to rate google serps. search bistro even had a flash video of the whole secret lab.

well remember recently when I Joked about google having :

or even more if you had you own little GUI running down the side of the serps and with a press of a button that site was gone for ever

in my american airlines post

then in this article with Marissa Mayer we get :

Q: How do they tell if they have bad results?
A: They have a bunch of watchdog services that track uptime for various servers to make sure a bad one isn’t causing problems. In addition, they have 10,000 human evaluators who are always manually checking the relevance of various results.

ok DaveN’s theory.. what if you where approached by Google, you sign an NDA and your Google account, switches you to the NEW EVAL, where you get the buttons, where you get the ability to add notes, share findings with your best 10,000 friends. what if you could change the face of Google, what if you could add sites and promote .. would that make a playing field a little different

DaveN

added the old eval system