Time and time again, rumors of a Google “rater’s manual” take the web by storm. This manual is, in fact, a book of guidelines for a team of people assigned by Google to rate the quality and relevancy of webpages that are indexed in its search engine results. Now, The Register claim they have seen a copy of the book.
In October 2011, Miranda Miller, at Search Engine Watch, wrote an extensive piece about a 120+ page training manual for new URL raters, initially discovered by PotPieGirl. That book was called the 2011 Google Quality Raters Handbook, and it magically vanished from the Internet a few days after PotPieGirl (Jennifer Ledbetter) made the link public.
Today, The Register revealed the existence of a second manual, perhaps an updated version of the first. At 160+ pages, this is also supposed to give detailed advice for raters on how to label search results.
A Google April Fool’s Day 2002 joke might be truer than the search engine likes to admit.But The Register reveals even more, they tell us who the raters are:
“Google’s outsources the ratings to contractors Leapforce and Lionbridge, who employ home workers,” the article reveals. “According to one Leapforce job ad there are 1,500 raters. The work is flexible but demanding – raters must pass an examination and are consistently evaluated by Google. For example, a rater is given a “TTR” score – “Time to Rate” measures how quickly they make their decisions.”
Even this is not new information. Lionbridge has been mentioned by several publications, including by Search Engine Land, earlier this year.
So if the existence of the handbook is nothing new, and it is already known who these raters are, why is The Register reigniting the conversation? It probably has a lot to do with the fact that the US Federal Trade Commission (FTC) could be dropping their antitrust case against Google. Andrew Orlowski, who wrote the exclusive piece for The Register, does not mention the FTC deal, but ends his piece with a valid observation:
It’s amazing how the image Google likes to promote – and politicians believe – one of high tech boffinry and magical algorithms, contrasts with the reality. Outsourced home workers are keeping the machine running. Glamorous, it isn’t.
How do you feel about having human raters in the equation?
