32 captures
12 Mar 2009 - 08 Apr 2019
About this capture
Sign in

News (publishers) Help
Google HelpGoogle News (publishers) HelpUpdates to your SiteAdditional Tips › Robots  
Additional Tips: Robots
You can prevent parts of your site from being indexed by web crawlers by creating a Robots.txt file, or by using a META tag. Please keep in mind that the robot we use for Google News, called Googlebot, is the same robot that we use for Google Web Search. This means that any settings you modify for Google News will also apply to Google Web Search. Our other robots, such as Googlebot-Mobile and Googlebot-Image, follow rules you set up for Googlebot, but you can set up additional rules for these specific bots as well.
Creating a Robots.txt file
Using a Robots.txt file gives you a high level of control over what parts of your site are indexed by Google. You'll find a comprehensive guide to creating and maintaining Robots.txt files at our Webmaster Help Center.
Creating a META tag
Rather than use a robots.txt file to block crawler access to pages, you can add a <META> tag to an HTML page to tell robots not to index the page. This standard is described at http://www.robotstxt.org/wc/exclusion.html#meta​.
Was this article:
The information you were looking for?
Yes No
You may also be interested in...
Other helpful articles:
Additional Tips: Snippets
Additional Tips: Article titles
Additional Tips: Links to articles
Recommended articles
  1. Google News
  2. Article URLs
  3. Cost of inclusion
  4. Creating a News Sitemap
  5. Press releases
Learn from other Google users
Find answers, ask questions, and share your expertise with others in the Google News Help Forum.
Increase your earning potential
Google AdSense can help you maximize your site's revenue potential. AdSense matches ads to your site's content, and you earn money whenever your visitors click on them. Sign up today.
Suggest my news site for Google News
If your site hasn't been added to the Google News index, please don't hesitate to suggest it for inclusion.
Can't find an answer to your question? Contact us
©2009 Google - Google News - Help for Publishers - Google News Help - About Google News - Terms of Use - Privacy Policy -
Change Language:
Google News (publishers) HomeHelp topicsGetting StartedYour site and Google NewsUpdates to your SiteHelp forumHelp for PublishersGoogle News HelpAbout Google NewsNews BlogContacting Support