Google News Blog - The Official Blog from the team at Google news

Open-sourcing the Living Stories format

Wednesday, February 17, 2010 8:04 AM



For the past two months, small teams of reporters and editors from the New York Times and Washington Post have been experimenting with Living Stories, a new format for covering news on the web. Using this technology platform, we can capture hundreds of developments as events unfold on a single dynamic page so that readers have many ways to easily digest the information. Living Stories has helped the Times enlighten readers on such subjects as global warming, the Afghanistan war, the N.F.L. playoffs and executive compensation. The Post has used it to report on health care reform, the Redskins' season and the overhaul of the D.C. school system.

Since we launched this proof-of-concept test on Google Labs in December, 75% of people who sent us feedback said they preferred the Living Stories format to the traditional online news article. Users also spent a significant amount of time exploring stories. This tells us there's a strong appetite for great journalism displayed in a compelling way.

In addition to the positive input from visitors, we've also heard from publishers interested in telling their own stories through the format. So we think it's time for the next stage of this experiment: releasing Living Stories more broadly to see what you can do with it. Today we're open-sourcing the code so all developers can build their own Living Story pages. (Here's the open-source documentation for technical details; read our Google News Help Forum to ask and answer general support questions.) Now that we're shifting into this public phase of the experiment, the Times and the Post are going to wind down their work on the version hosted on Google Labs. We'd like to thank them for embarking on this stage of the project with us. We're looking forward to continuing to work with them, and many other publishers, on Living Stories as well as other projects that help to advance how news is presented online.

In coming months, we're going to look into creating software tools that make Living Stories even easier to use for news organizations. Until then, we can't wait to see what fascinating works of journalism developers, reporters and editors, working together, create using the open-sourced Living Stories code.

Starring stories in Google News

Monday, February 1, 2010 2:00 PM



A couple of months back, we launched the Custom Sections Directory feature in Google News to allow users to set up sections on topics of their interest, and to share them with other users.

Today, we are giving users even more options for following stories. Users can mark a story cluster by clicking on the star next to it, like they can with messages in Gmail and items in Google Reader. When you star a story in Google News, it's one way to let us know that you're interested in that subject. When there are significant updates, we will alert you by putting the headline in bold so you can get more information. You can also follow your 20 most recent starred stories in the "Starred" section of Google News.


We hope you enjoy the new feature and welcome your feedback.

Google News Comes Back For More

Monday, January 25, 2010 11:53 AM



If you read news online, you've probably noticed that articles aren't static. They often change over time, to reflect things like typo fixes, shifts in emphasis, new information or corrections of previous mistakes. Sometimes they even switch URLs, or become unavailable after a certain period of time. As a human being, reading at most a few dozen articles a day, this is no big deal.

But if you happen to be, say, a news search engine that crawls hundreds of articles at thousands of sites every minute, this presents a unique set of challenges. How do you balance looking for new content against the need to update older content? How can you make sure the content is fresh, doesn't link to dead pages or display headlines that have been changed by the publisher?

To deal with these issues, Google News has implemented a recrawl feature that allows us to focus on getting the newest articles around while still ensuring that we're displaying the most up-to-date information. From the moment we discover a new article, we'll keep revisiting it looking for changes. Since we've noticed that most changes to articles occur just after they're published, we revisit articles most frequently in the first day after we've found them. In some cases, we'll even revisit articles we had trouble crawling the first time around. After that, we visit them less often. Either way, we try hard to present users with the freshest news. (We bet whoever wrote "Dewey Defeats Truman" wishes they had recrawl!)

For readers, this feature is intended to reduce the number of outdated headlines and dead links you might find. And for publishers, rest assured that we'll be back to find your latest stories and updates as soon as we can.

Fast Flip now available on Google News homepage

Friday, January 8, 2010 4:23 PM



Today you may notice a change to the Google News home page: Near the bottom, we're now displaying stories from Google Fast Flip, the article-reading service we launched in September. Fast Flip is still in Google Labs, so we'll continue to experiment with the format. But so far we've found that the speed and visual nature of the service encourages readers to look at many articles and, for the ones that catch their interest, click through to the story publishers' websites.

In December we added more than 50 newspapers, magazines, web outlets news wires and TV and radio broadcasters, bringing the total number of news sources discoverable in Fast Flip to more than 90. Encouraged by the positive feedback we've received from users and partners, we decided to expose the service to more potential readers by integrating it with the U.S. English version of Google News. As always, we welcome your thoughts.

More great news sources to discover in Fast Flip

Wednesday, December 16, 2009 1:11 PM


[cross-posted from the Official Google Blog]

Three months ago, we launched Google Fast Flip, a service that seeks to make reading articles online as fast and simple as flipping through a magazine or newspaper. It's still early in this experiment, which is why Fast Flip remains in Google Labs. But so far our initial thesis has held up: If you make it easier to read news online, people will read more of it. Users have told us they like being able to browse content so quickly, and we've been pleased with the amount of time they have spent reading articles in Fast Flip.

We've also received good feedback from the three dozen publishers who joined us for the launch, as well as a lot of interest from others. Today, we're excited to be adding articles from another two dozen publishers representing more than 50 newspapers, magazines, web outlets, news wires and TV and radio broadcasters. Some of the new sources include Tribune Co. newspapers such as the Los Angeles Times and the Chicago Tribune, McClatchy Company newspapers such as the Miami Herald and the Kansas City Star, the Huffington Post, Popular Science, Reuters, Public Radio International, POLITICO and U.S. News & World Report. Now you can use Fast Flip to engage with content from even more of your favorite news outlets in an innovative way, and continue to explore topics covered by a diverse group of sources. And, through the mobile version, you can flip through all these new articles on your Android-powered device or iPhone.

While we're encouraged by the positive feedback about Fast Flip, it's just one of many experiments you'll see us try in partnership with news publishers. Our goal is to work with the industry to help it continue to innovate and build bigger audiences, better engage those audiences and generate more revenue. We're looking forward to innovating and iterating with all these new partners in Fast Flip. And if you have more suggestions for ways we can improve Fast Flip, please let us know.

The holiday season and Google News Archive Search

Tuesday, December 15, 2009 10:42 AM



Crowded shopping malls, radio stations pumping songs about sleigh bells and chestnuts, inclement weather from coast to coast -- all signs point to one explanation. We're smack in the middle of the holiday season.

Whichever holidays you observe, you might be surprised to learn of another one to add to the roster. Two hundred eighteen years ago today, the founders ratified the Bill of Rights, which the United States officially celebrates every December 15 as U.S. Bill of Rights Day. We have President Franklin D. Roosevelt to thank for officially creating the holiday, which he inaugurated in 1941, on the 150th anniversary of the document's ratification.

I took a spin through Google News Archive Search to learn more. Searching for "Bill of Rights Day 1941," I was able to drill down to autumn of that year, where I hoped to find articles explaining how the holiday took shape. Sure enough, the St. Petersburg Times ran an Associated Press story on November 29, 1941, quoting a proclamation from President Roosevelt. In what appears to be a clear reference to the events of World War II, Roosevelt mentions the "privileges lost in other continents and countries," and how Americans "can now appreciate their meaning to those people who enjoyed them once and now no longer can." And so December 15 would become "a day of mobilization for freedom and for human rights, a day of remembrance of the democratic and peaceful action by which these rights were gained, a day of reassessment of their present meaning and their living worth."

Little more than a week later, of course, arrived "a date which will live in infamy," the attack on Pearl Harbor, which brought the United States into the war.

So when Bill of Rights Day arrived eight days later, the holiday had renewed meaning. Once again, in the St. Petersburg Times, you could find a full page featuring another proclamation from Roosevelt, the full Bill of Rights reprinted for readers, and a picture of Roosevelt with New York Mayor Fiorello Henry La Guardia. And the day after the holiday, this write-up summarized various ways that the country marked the occasion, including Chicago school girls reading the Bill of Rights "publicly at State and Madison streets."

It's interesting to see how the meaning and observance of different holidays continue to change, sometimes even just days after they have been created.

Exploring a new, more dynamic way of reading news with Living Stories

Tuesday, December 8, 2009 9:54 AM


[cross-posted from the Official Google Blog]

There's been no shortage of talk recently about the "future of news." Should publishers charge for news online? How do they replace lost sources of revenue such as classified ads? How will accountability journalism endure? And, even more fundamentally, will news survive in the digital era? These are questions we're deeply interested in, and we've been exploring potential solutions. But what's often overlooked in these debates is the nature of the news story itself and the experience of how it's read online. We believe it's just as important to experiment with how news organizations can take advantage of the web to tell stories in new ways — ways that simply aren't possible offline.

While we have strong ideas about how information is experienced on the web, we're not journalists and we don't create content. So over the last few months we've been talking to a number of people to help develop the concept of something that we and some others in the industry call the "living story." Today, on Google Labs, we're unveiling some of the work we've done in partnership with two world-class news organizations: The News York Times and The Washington Post. The result of that experiment is the Living Stories prototype, which features new ways to interact with news and the quality of reporting you've come to expect from the reporters and editors at The Post and The Times. We're excited to learn from this experiment, and hope to eventually make these tools available to any publisher that wants to use them.

The idea behind Living Stories is to experiment with a different format for presenting news coverage online. News organizations produce a wealth of information that we all value; access to this information should be as great as the online medium allows. A typical newspaper article leads with the most important and interesting news, and follows with additional information of decreasing importance. Information from prior coverage is often repeated with each new online article, and the same article is presented to everyone regardless of whether they already read it. Living Stories try a different approach that plays to certain unique advantages of online publishing. They unify coverage on a single, dynamic page with a consistent URL. They organize information by developments in the story. They call your attention to changes in the story since you last viewed it so you can easily find the new material. Through a succinct summary of the whole story and regular updates, they offer a different online approach to balancing the overview with depth and context.

This project sprang from conversations among senior executives at the three companies. We shared thoughts about how the web can work for storytelling, and the Times and Post shared their core journalistic principles. The Living Stories started taking shape over the summer after our engineering and user interface teams spent time in the newsrooms of both papers. We're providing the technology platform, the Times and Post's journalists are writing and editing the stories, and we're continuously collaborating to make the user interface fit with their editorial vision.

Over the coming months, we'll refine Living Stories based on your feedback. We're also looking to develop openly available tools that could aid news organizations in the creation of these pages or at least in some of the features. If you're a news reader, we'd love to hear your thoughts. If you're a news organization, we want to hear your comments on the Living Story format. If you decide to implement this on your site, we would love to hear about that too. At the very least, we hope this collaboration will kick off debate and encourage innovation in how people interact with news online. To see how Living Stories works, check out the video below.

Same Protocol, More Options for News Publishers

Wednesday, December 2, 2009 8:10 AM



There are more than 25,000 publishers from around the world in Google News today. That's because Google News is a great source of readers, sending publishers about 1 billion clicks every month. Each of those clicks is an opportunity for publishers, allowing them to show ads, sell subscriptions and introduce readers to the great content they produce every day. While we think this offers a tremendous opportunity for any publisher who wants new readers, publishers are the ones who create the content and they're in control of it. If they decide they don't want to be in Google, it's easy to do. Today, we're making it even easier with a web crawler specifically for Google News.

Publishers have always had the ability to block Google from including their content in Google's index. How? With something called Robots Exclusion Protocol (or REP) - a web-wide standard supported by all major search engines and any reputable company that crawls the web. When our crawler arrives at any site, it checks to see if there's a robots.txt file to make sure we have permission to crawl the site. With this file, or similar REP directives on specific pages, publishers can block their entire site, certain sections or individual pages. They can also give instructions on how they want us to index their content, such as telling us to exclude images or snippets of text. Furthermore, they can apply different instructions to different crawlers, giving access to some while blocking others.

The new Google News web crawler extends these controls to Google News. If they wanted to, it's always been easy for publishers to keep their content out of Google News and still remain in Google Search. They just had to fill out a simple contact form in our Help Center. Now, with the news-specific crawler, if a publisher wants to opt out of Google News, they don't even have to contact us - they can put instructions just for user-agent Googlebot-News in the same robots.txt file they have today. In addition, once this change is fully in place, it will allow publishers to do more than just allow/disallow access to Google News. They'll also be able to apply the full range of REP directives just to Google News. Want to block images from Google News, but not from Web Search? Go ahead. Want to include snippets in Google News, but not in Web Search? Feel free. All this will soon be possible with the same standard protocol that is REP.

Our users shouldn't notice any difference. Google News will keep helping people discover the news they're looking for, different perspectives from across the world and new sources of information they might not otherwise have found.

While this means even more control for publishers, the effect of opting out of News is the same as it's always been. It means that content won't be in Google News or in the parts of Google that are powered by the News index. For example, if a publisher opts out of Google News, but stays in Web Search, their content will still show up as natural web search results, but they won't appear in the block of news results that sometimes shows up in Web Search, called Universal search, since those come from the Google News index.

Most people put their content on the web because they want it to be found, so very few choose to exclude their material from Google. But we respect publishers' wishes. If publishers don't want their websites to appear in web search results or in Google News, we want to give them easy ways to remove it. We're excited about this change and will start rolling it out today. You can learn more about the details of this change on our Webmaster Central blog. If you see any problems or have any questions, please let us know.

Google and paid content

Tuesday, December 1, 2009 9:35 AM



As newspapers consider charging for access to their online content, some publishers have asked: Should we put up pay walls or keep our articles in Google News and Google Search? In fact, they can do both - the two aren't mutually exclusive. There are a few ways we work with publishers to make their subscription content discoverable. Today we're updating one of them, so we thought it would be a good time to remind publishers about some of their options.

Google has strict policies against what's known as cloaking: showing one web page to the crawler that indexes it but then a different page to a user. We do this so that users aren't deceived into clicking through to a site that's not what they were expecting. While the anti-cloaking policies are important for users, they do create some challenges for publishers who charge for content. Our crawlers can't fill out a registration or payment form to see what's behind a site's paywall, but they need access to the information in order to index it.

One way we overcome this is through a program called First Click Free. Participating publishers allow the crawler to index their subscription content, then allow users who find one of those articles through Google News or Google Search to see the full page without requiring them to register or subscribe. The user's first click to the content is free, but when a user clicks on additional links on the site, the publisher can show a payment or registration request. First Click Free is a great way for publishers to promote their content and for users to check out a news source before deciding whether to pay. Previously, each click from a user would be treated as free. Now, we've updated the program so that publishers can limit users to no more than five pages per day without registering or subscribing. If you're a Google user, this means that you may start to see a registration page after you've clicked through to more than five articles on the website of a publisher using First Click Free in a day. We think this approach still protects the typical user from cloaking, while allowing publishers to focus on potential subscribers who are accessing a lot of their content on a regular basis.

In addition to First Click Free, we offer another solution: We will crawl, index and treat as "free" any preview pages - generally the headline and first few paragraphs of a story - that they make available to us. This means that our crawlers see the exact same content that will be shown for free to a user. Because the preview page is identical for both users and the crawlers, it's not cloaking. We will then label such stories as "subscription" in Google News. The ranking of these articles will be subject to the same criteria as all sites in Google, whether paid or free. Paid content may not do as well as free options, but that is not a decision we make based on whether or not it's free. It's simply based on the popularity of the content with users and other sites that link to it.

These are two of the ways we allow publishers to make their subscription content discoverable, and we're going to keep talking with publishers to refine these methods. After all, whether you're offering your content for free or selling it, it's crucial that people find it. Google can help with that.

New Google News for mobile

Thursday, November 19, 2009 2:04 PM


[cross-posted from the Official Google Mobile Blog]

At Google, we are committed to giving you a consistent user experience across products and devices, and we really value the feedback you've given us about Google News for mobile. Today we're excited to announce a completely new Google News offering for iPhone, Android, and Palm Pre users. (We already offer a mobile-optimized version of Google News for other phones, such as Blackberry, Windows Mobile, and S60, and more improvements will be coming to those in the near future)

This new version provides the same richness and personalization on your phone as Google News provides on desktop. Our new homepage displays more stories, sources, and images while keeping a familiar look and feel. Also, you can now reach your favorite sections, discover new ones, find articles and play videos in fewer clicks. If you are an existing Google News reader on desktop, you will find that all of your personalizations are honored in this mobile version too.

Google News for mobile is now available in 29 languages and 70 editions.

So pick up your mobile phone and point your browser to http://news.google.com to catch up on news anytime and anywhere. Feel free to check out more information or leave feedback in our Help Center.