|
Additional Tips: RobotsYou can prevent parts of your site from being indexed by web crawlers by creating a Robots.txt file, or by using a META tag. Please keep in mind that the robot we use for Google News, called Googlebot, is the same robot that we use for Google Web Search. This means that any settings you modify for Google News will also apply to Google Web Search. Our other robots, such as Googlebot-Mobile and Googlebot-Image, follow rules you set up for Googlebot, but you can set up additional rules for these specific bots as well.
Creating a Robots.txt file Using a Robots.txt file gives you a high level of control over what parts of your site are indexed by Google. You'll find a comprehensive guide to creating and maintaining Robots.txt files at our Webmaster Help Center. Creating a META tagRather than use a robots.txt file to block crawler access to pages, you can add a <META> tag to an HTML page to tell robots not to index the page. This standard is described at http://www.robotstxt.org/wc/exclusion.html#meta.
You may also be interested in... |
Recommended articlesLearn from other Google usersFind answers, ask questions, and share your expertise with others in the Google News Help Forum. Increase your earning potentialGoogle AdSense can help you maximize your site's revenue potential. AdSense matches ads to your site's content, and you earn money whenever your visitors click on them. Sign up today.Suggest my news site for Google NewsIf your site hasn't been added to the Google News index, please don't hesitate to suggest it for inclusion. |