Thursday, August 25, 2011 2:00 PMPosted by David Smydra, Google News Product Specialist(Cross-posted on the Webmaster Central Blog)
Google News recently updated our infrastructure to crawl with Google’s primary user-agent, Googlebot
. What does this mean? Very little to most publishers. Any news organizations that wish to opt out of Google News can continue to do so: Google News will still respect the robots.txt entry for Googlebot-News
, our former user-agent, if it is more restrictive than the robots.txt entry for Googlebot.
Our Help Center provides detailed guidance
on using the robots exclusion protocol for Google News, and publishers can contact the Google News Support Team if they have any questions, but we wanted to first clarify the following:
As with any website, from time to time we need to make updates to our infrastructure. At the same time, we want to continue to provide as much control as possible to news web sites. We hope we have answered any questions you might have about this update. If you have additional questions, please check out our Help Center