You can prevent parts of your site from being indexed by web crawlers by creating a Robots.txt file, or by using a META tag. Please keep in mind that the robot we use for Google News, called Googlebot, is the same robot that we use for Google Web Search. This means that any settings you modify for Google News will also apply to Google Web Search. Our other robots, such as Googlebot-Mobile and Googlebot-Image, follow rules you set up for Googlebot, but you can set up additional rules for these specific bots as well.
Creating a Robots.txt file
Using a Robots.txt file gives you a high level of control over what parts of your site are indexed by Google. You'll find a comprehensive guide to creating and maintaining Robots.txt files at our Webmaster Help Center.
The date and time must be specified in the RFC 850 format. This information is treated as a removal request: it will take about a day after the removal date passes for the page to disappear from the search results. However, in order for the tag to function properly, it must be included with your article at the time that it is first crawled.