Two more weeks to update your Google News Sitemap

Tuesday, April 20, 2010 3:05 PM

Posted by Andy Golding, Google News Software Engineer

Almost five months ago, we announced a redesign for our Google News Sitemap. We wanted to give publishers plenty of advance notice so they could make a smooth transition to the new format.

This transition period will end on May 4, when we'll discontinue supporting all old-style Google News Sitemaps. If you are a publisher and haven't already shifted your Google News sitemap to the new format, we urge you to do so by following these instructions as soon as possible. This update is key to avoid interruption in crawling your content and allow our crawler to capture all necessary information about your articles.

If you've already made the changes to your sitemap, we thank you. There is no need to worry about this update. If you're interested in learning more about Google News sitemaps, we encourage you to read this section of our Help Center and submit a sitemap for your articles in Google News. You may also be interested in checking out our Help Forum where other publishers have shared useful tips and discussed other Sitemaps-related questions.

Don't forget to update your Google News Sitemaps. There are only two more weeks to go!

Eric Schmidt discusses innovation with news editors

Monday, April 12, 2010 6:48 PM

Posted by Chris Gaither, Senior Manager, Global Communications & Public Affairs

This week, hundreds of newsroom leaders from across the country are gathered in Washington DC for the American Society of News Editors’ annual conference. The theme of this year's event is "ideas," and Eric Schmidt shared many of his as the opening speaker. He spoke about some of the big trends that are shaping the Internet, such as the rise of mobile and cloud computing, and some lessons he has learned about how to navigate the Web’s constant pace of change.

He also talked about the importance of journalism to functioning democracies and encouraged the group to work together -- as well as with technology partners like Google -- to find new ways to reach and engage audiences, tell important stories and build thriving businesses online.

You can watch Eric's speech, which we've posted on YouTube, below. And if you’re interested in reading or contributing to the discussion about the future of news, check out the unofficial conference blog and #asne10 on Twitter.

Open-sourcing the Living Stories format

Wednesday, February 17, 2010 8:04 AM

Posted by Neha Singh, Software Engineer, and Josh Cohen, Senior Business Product Manager

For the past two months, small teams of reporters and editors from the New York Times and Washington Post have been experimenting with Living Stories, a new format for covering news on the web. Using this technology platform, we can capture hundreds of developments as events unfold on a single dynamic page so that readers have many ways to easily digest the information. Living Stories has helped the Times enlighten readers on such subjects as global warming, the Afghanistan war, the N.F.L. playoffs and executive compensation. The Post has used it to report on health care reform, the Redskins' season and the overhaul of the D.C. school system.

Since we launched this proof-of-concept test on Google Labs in December, 75% of people who sent us feedback said they preferred the Living Stories format to the traditional online news article. Users also spent a significant amount of time exploring stories. This tells us there's a strong appetite for great journalism displayed in a compelling way.

In addition to the positive input from visitors, we've also heard from publishers interested in telling their own stories through the format. So we think it's time for the next stage of this experiment: releasing Living Stories more broadly to see what you can do with it. Today we're open-sourcing the code so all developers can build their own Living Story pages. (Here's the open-source documentation for technical details; read our Google News Help Forum to ask and answer general support questions.) Now that we're shifting into this public phase of the experiment, the Times and the Post are going to wind down their work on the version hosted on Google Labs. We'd like to thank them for embarking on this stage of the project with us. We're looking forward to continuing to work with them, and many other publishers, on Living Stories as well as other projects that help to advance how news is presented online.

In coming months, we're going to look into creating software tools that make Living Stories even easier to use for news organizations. Until then, we can't wait to see what fascinating works of journalism developers, reporters and editors, working together, create using the open-sourced Living Stories code.

Starring stories in Google News

Monday, February 1, 2010 2:00 PM

Posted by Jude Britto, Software Engineer, Google News

A couple of months back, we launched the Custom Sections Directory feature in Google News to allow users to set up sections on topics of their interest, and to share them with other users.

Today, we are giving users even more options for following stories. Users can mark a story cluster by clicking on the star next to it, like they can with messages in Gmail and items in Google Reader. When you star a story in Google News, it's one way to let us know that you're interested in that subject. When there are significant updates, we will alert you by putting the headline in bold so you can get more information. You can also follow your 20 most recent starred stories in the "Starred" section of Google News.


We hope you enjoy the new feature and welcome your feedback.

Google News Comes Back For More

Monday, January 25, 2010 11:53 AM

Posted by Andy Golding and Kiran Gunda, Software Engineers

If you read news online, you've probably noticed that articles aren't static. They often change over time, to reflect things like typo fixes, shifts in emphasis, new information or corrections of previous mistakes. Sometimes they even switch URLs, or become unavailable after a certain period of time. As a human being, reading at most a few dozen articles a day, this is no big deal.

But if you happen to be, say, a news search engine that crawls hundreds of articles at thousands of sites every minute, this presents a unique set of challenges. How do you balance looking for new content against the need to update older content? How can you make sure the content is fresh, doesn't link to dead pages or display headlines that have been changed by the publisher?

To deal with these issues, Google News has implemented a recrawl feature that allows us to focus on getting the newest articles around while still ensuring that we're displaying the most up-to-date information. From the moment we discover a new article, we'll keep revisiting it looking for changes. Since we've noticed that most changes to articles occur just after they're published, we revisit articles most frequently in the first day after we've found them. In some cases, we'll even revisit articles we had trouble crawling the first time around. After that, we visit them less often. Either way, we try hard to present users with the freshest news. (We bet whoever wrote "Dewey Defeats Truman" wishes they had recrawl!)

For readers, this feature is intended to reduce the number of outdated headlines and dead links you might find. And for publishers, rest assured that we'll be back to find your latest stories and updates as soon as we can.

Fast Flip now available on Google News homepage

Friday, January 8, 2010 4:23 PM

Posted by Jack Hebert, Matthew Watson and Corrie Scalisi, Software Engineers

Today you may notice a change to the Google News home page: Near the bottom, we're now displaying stories from Google Fast Flip, the article-reading service we launched in September. Fast Flip is still in Google Labs, so we'll continue to experiment with the format. But so far we've found that the speed and visual nature of the service encourages readers to look at many articles and, for the ones that catch their interest, click through to the story publishers' websites.

In December we added more than 50 newspapers, magazines, web outlets news wires and TV and radio broadcasters, bringing the total number of news sources discoverable in Fast Flip to more than 90. Encouraged by the positive feedback we've received from users and partners, we decided to expose the service to more potential readers by integrating it with the U.S. English version of Google News. As always, we welcome your thoughts.

More great news sources to discover in Fast Flip

Wednesday, December 16, 2009 1:11 PM

Posted by Jack Hebert, Software Engineer
[cross-posted from the Official Google Blog]

Three months ago, we launched Google Fast Flip, a service that seeks to make reading articles online as fast and simple as flipping through a magazine or newspaper. It's still early in this experiment, which is why Fast Flip remains in Google Labs. But so far our initial thesis has held up: If you make it easier to read news online, people will read more of it. Users have told us they like being able to browse content so quickly, and we've been pleased with the amount of time they have spent reading articles in Fast Flip.

We've also received good feedback from the three dozen publishers who joined us for the launch, as well as a lot of interest from others. Today, we're excited to be adding articles from another two dozen publishers representing more than 50 newspapers, magazines, web outlets, news wires and TV and radio broadcasters. Some of the new sources include Tribune Co. newspapers such as the Los Angeles Times and the Chicago Tribune, McClatchy Company newspapers such as the Miami Herald and the Kansas City Star, the Huffington Post, Popular Science, Reuters, Public Radio International, POLITICO and U.S. News & World Report. Now you can use Fast Flip to engage with content from even more of your favorite news outlets in an innovative way, and continue to explore topics covered by a diverse group of sources. And, through the mobile version, you can flip through all these new articles on your Android-powered device or iPhone.

While we're encouraged by the positive feedback about Fast Flip, it's just one of many experiments you'll see us try in partnership with news publishers. Our goal is to work with the industry to help it continue to innovate and build bigger audiences, better engage those audiences and generate more revenue. We're looking forward to innovating and iterating with all these new partners in Fast Flip. And if you have more suggestions for ways we can improve Fast Flip, please let us know.

The holiday season and Google News Archive Search

Tuesday, December 15, 2009 10:42 AM

Posted by David Smydra, Google News Online Team

Crowded shopping malls, radio stations pumping songs about sleigh bells and chestnuts, inclement weather from coast to coast -- all signs point to one explanation. We're smack in the middle of the holiday season.

Whichever holidays you observe, you might be surprised to learn of another one to add to the roster. Two hundred eighteen years ago today, the founders ratified the Bill of Rights, which the United States officially celebrates every December 15 as U.S. Bill of Rights Day. We have President Franklin D. Roosevelt to thank for officially creating the holiday, which he inaugurated in 1941, on the 150th anniversary of the document's ratification.

I took a spin through Google News Archive Search to learn more. Searching for "Bill of Rights Day 1941," I was able to drill down to autumn of that year, where I hoped to find articles explaining how the holiday took shape. Sure enough, the St. Petersburg Times ran an Associated Press story on November 29, 1941, quoting a proclamation from President Roosevelt. In what appears to be a clear reference to the events of World War II, Roosevelt mentions the "privileges lost in other continents and countries," and how Americans "can now appreciate their meaning to those people who enjoyed them once and now no longer can." And so December 15 would become "a day of mobilization for freedom and for human rights, a day of remembrance of the democratic and peaceful action by which these rights were gained, a day of reassessment of their present meaning and their living worth."

Little more than a week later, of course, arrived "a date which will live in infamy," the attack on Pearl Harbor, which brought the United States into the war.

So when Bill of Rights Day arrived eight days later, the holiday had renewed meaning. Once again, in the St. Petersburg Times, you could find a full page featuring another proclamation from Roosevelt, the full Bill of Rights reprinted for readers, and a picture of Roosevelt with New York Mayor Fiorello Henry La Guardia. And the day after the holiday, this write-up summarized various ways that the country marked the occasion, including Chicago school girls reading the Bill of Rights "publicly at State and Madison streets."

It's interesting to see how the meaning and observance of different holidays continue to change, sometimes even just days after they have been created.

Exploring a new, more dynamic way of reading news with Living Stories

Tuesday, December 8, 2009 9:54 AM

Posted by Neha Singh, Software Engineer, and Josh Cohen, Senior Business Product Manager
[cross-posted from the Official Google Blog]

There's been no shortage of talk recently about the "future of news." Should publishers charge for news online? How do they replace lost sources of revenue such as classified ads? How will accountability journalism endure? And, even more fundamentally, will news survive in the digital era? These are questions we're deeply interested in, and we've been exploring potential solutions. But what's often overlooked in these debates is the nature of the news story itself and the experience of how it's read online. We believe it's just as important to experiment with how news organizations can take advantage of the web to tell stories in new ways — ways that simply aren't possible offline.

While we have strong ideas about how information is experienced on the web, we're not journalists and we don't create content. So over the last few months we've been talking to a number of people to help develop the concept of something that we and some others in the industry call the "living story." Today, on Google Labs, we're unveiling some of the work we've done in partnership with two world-class news organizations: The News York Times and The Washington Post. The result of that experiment is the Living Stories prototype, which features new ways to interact with news and the quality of reporting you've come to expect from the reporters and editors at The Post and The Times. We're excited to learn from this experiment, and hope to eventually make these tools available to any publisher that wants to use them.

The idea behind Living Stories is to experiment with a different format for presenting news coverage online. News organizations produce a wealth of information that we all value; access to this information should be as great as the online medium allows. A typical newspaper article leads with the most important and interesting news, and follows with additional information of decreasing importance. Information from prior coverage is often repeated with each new online article, and the same article is presented to everyone regardless of whether they already read it. Living Stories try a different approach that plays to certain unique advantages of online publishing. They unify coverage on a single, dynamic page with a consistent URL. They organize information by developments in the story. They call your attention to changes in the story since you last viewed it so you can easily find the new material. Through a succinct summary of the whole story and regular updates, they offer a different online approach to balancing the overview with depth and context.

This project sprang from conversations among senior executives at the three companies. We shared thoughts about how the web can work for storytelling, and the Times and Post shared their core journalistic principles. The Living Stories started taking shape over the summer after our engineering and user interface teams spent time in the newsrooms of both papers. We're providing the technology platform, the Times and Post's journalists are writing and editing the stories, and we're continuously collaborating to make the user interface fit with their editorial vision.

Over the coming months, we'll refine Living Stories based on your feedback. We're also looking to develop openly available tools that could aid news organizations in the creation of these pages or at least in some of the features. If you're a news reader, we'd love to hear your thoughts. If you're a news organization, we want to hear your comments on the Living Story format. If you decide to implement this on your site, we would love to hear about that too. At the very least, we hope this collaboration will kick off debate and encourage innovation in how people interact with news online. To see how Living Stories works, check out the video below.

Same Protocol, More Options for News Publishers

Wednesday, December 2, 2009 8:10 AM

Posted by Josh Cohen, Senior Business Product Manager

There are more than 25,000 publishers from around the world in Google News today. That's because Google News is a great source of readers, sending publishers about 1 billion clicks every month. Each of those clicks is an opportunity for publishers, allowing them to show ads, sell subscriptions and introduce readers to the great content they produce every day. While we think this offers a tremendous opportunity for any publisher who wants new readers, publishers are the ones who create the content and they're in control of it. If they decide they don't want to be in Google, it's easy to do. Today, we're making it even easier with a web crawler specifically for Google News.

Publishers have always had the ability to block Google from including their content in Google's index. How? With something called Robots Exclusion Protocol (or REP) - a web-wide standard supported by all major search engines and any reputable company that crawls the web. When our crawler arrives at any site, it checks to see if there's a robots.txt file to make sure we have permission to crawl the site. With this file, or similar REP directives on specific pages, publishers can block their entire site, certain sections or individual pages. They can also give instructions on how they want us to index their content, such as telling us to exclude images or snippets of text. Furthermore, they can apply different instructions to different crawlers, giving access to some while blocking others.

The new Google News web crawler extends these controls to Google News. If they wanted to, it's always been easy for publishers to keep their content out of Google News and still remain in Google Search. They just had to fill out a simple contact form in our Help Center. Now, with the news-specific crawler, if a publisher wants to opt out of Google News, they don't even have to contact us - they can put instructions just for user-agent Googlebot-News in the same robots.txt file they have today. In addition, once this change is fully in place, it will allow publishers to do more than just allow/disallow access to Google News. They'll also be able to apply the full range of REP directives just to Google News. Want to block images from Google News, but not from Web Search? Go ahead. Want to include snippets in Google News, but not in Web Search? Feel free. All this will soon be possible with the same standard protocol that is REP.

Our users shouldn't notice any difference. Google News will keep helping people discover the news they're looking for, different perspectives from across the world and new sources of information they might not otherwise have found.

While this means even more control for publishers, the effect of opting out of News is the same as it's always been. It means that content won't be in Google News or in the parts of Google that are powered by the News index. For example, if a publisher opts out of Google News, but stays in Web Search, their content will still show up as natural web search results, but they won't appear in the block of news results that sometimes shows up in Web Search, called Universal search, since those come from the Google News index.

Most people put their content on the web because they want it to be found, so very few choose to exclude their material from Google. But we respect publishers' wishes. If publishers don't want their websites to appear in web search results or in Google News, we want to give them easy ways to remove it. We're excited about this change and will start rolling it out today. You can learn more about the details of this change on our Webmaster Central blog. If you see any problems or have any questions, please let us know.