Google News, one of Google's most popular, older and controversial services, is the article aggregation and search site that media companies love to hate. It has become a major source of web traffic - and frustration - for many of them.
Some, like Agence France Presse and Copiepresse, have sued for copyright infringement over Google News' modus operandi of indexing media content without permission. Meanwhile, every day, many editors cringe at the way Google News ranks articles.
All along, Google has defended News saying that it is protected by the fair use principle - because it only reproduces headlines, text snippets and thumbnail images - and that it provides great benefit to media web sites by sending them readers. IDG recently had a chance to chat about these and other issues, such as video news and social news sites, with Nathan Stoll, a Google product manager involved with Google News. This is an edited version of the interview:
IDG: Although many web publishers like having Google News drive traffic to their advertising-supported sites, it's probably no coincidence that you had to deal one-on-one to accommodate the concerns of Associated Press and Agence France Presse. As wire services, their revenue mostly comes from subscription fees, not web advertising. Have you rethought how you deal with wire services in Google News?
Nathan Stoll: We've always had a consistent approach to content producers and publishers: to give control to the original publisher of a piece of work. Google News should reward original journalism, whether it's from the AP or a local newspaper. That's the ultimate goal. When we think about who's the publisher we want to be talking with when they have issues or concerns, if it's the AP's articles, whether they are syndicated out there or not, AP is the one we need to work with.
IDG: Before you and AFP settled the lawsuit, it seemed technically difficult for you to keep AFP's content out of Google News. Although you significantly cut down, you never totally eliminated AFP content, apparently because, as wire service material, it appears on many web sites.
Stoll: It's a good observation. It's certainly more challenging than when a typical webmaster can put a robots.txt file telling us not to crawl their site. However, even with a newspaper that doesn't want us crawling their site, it could sell its articles to another site. Then we would need to come up with a way of detecting that. Or the newspaper could tell the site to meta-tag the article so that we wouldn't crawl it. It's very easy to do.
With the AFP lawsuit, we acted as if that had been an opt-out request and we tried to make sure, to the best of our ability, that their content wasn't in Google News.
IDG: Any plans to add a video component to Google News?
Stoll: We don't want to pre-announce any features but our Google News philosophy is to give users access to all the perspectives on a news story. To the extent that a lot of those are in video and becoming available online, we'd certainly love to make those perspectives available and easily discoverable.
With the YouTube team, working hard, it's certainly an area we'd like to make progress in.
It's true that lots of publishers have asked us about our plans, whether or not we're planning to include video because they'd like to get added traffic to their video content. There's certainly a need out there.
IDG: What are you doing on the social news front, along the lines of sites such as Digg and Slashdot?
Stoll: Obviously Google has a number of products and services that touch on those types of areas. In News today we offer a number of customisation and personalisation features. If I was to give you themes about areas that we're working on, that would be one area in which we're very interested.
IDG: What can people do in terms of tagging and categorisation in Google News?
Stoll: Not much. Today, the tools we offer allow them to customise the content they get on their pages but we're not actually using many explicit feedback loops from users directly. We offer features built on top of the Web History product which allow users to give us permission to keep track of the articles they've read and we can use that information to offer them some better news articles and better rankings.
IDG: If you had the massive user feedback that these social news sites get, could you use that feedback to improve the Google News ranking calculation?
Stoll: It would be an additional type of signal. We offer a "most popular" section on the front page of many of our editions. That popularity ranking signal is different from how the front page is ranked, which tries to reflect what editors are publishing on their sites. If we introduced a Digg-style feature, it would be more similar to that popularity metric. Today, we're still trying to reflect what editors are doing because that's an important purpose Google News has served: to reflect aggregate editorial interests in a story.
IDG: What are the advantages of using Google News versus Digg or Slashdot?
Stoll: Comprehensiveness is a huge piece. If you want to know if someone in the news space is writing about a subject, you do a search on Google News. If you want to monitor that on an ongoing basis, you set up a Google News alert.
IDG: Some critics say that the ranking algorithms of Google News don't take into consideration story quality or who broke it.
Stoll: This is why we're out there talking with editors, because we hear these things from them, the ways in which we need to improve the algorithms. In an ideal world, Google News would show you who broke the story and the other articles that built on that. There are places where we're not doing that perfectly today.
IDG: How much outreach do you do to talk to editors in the publications you index?
Stoll: It's something we're looking to always do more of. Our ability to reach out in various regions of the world has dramatically improved in recent years. We go to [media] conferences and do tours of regions. But it's hard to reach out to everyone individually, which is why we do a lot of online support so that any editor with a concern, suggestion, question or comment can share it with us.
IDG: Do you pay any of the news outlets you index in Google News?
Stoll: We very much believe that what we do with Google News is fair use. When someone disagrees with that, we're happy to talk to them and work with them to make sure their content is opted-out of the product.
We believe there is value in Google's search services in the traffic that we give back and that most publishers want to participate. When we want to do things with content producers that go beyond that [fair use] of linking back and giving traffic back, whether it's a maps producer or a satellite imagery provider, or even some of the video publishers, then we enter into contractual relationships that may involve money for content.
IDG: Any plans to run ads in Google News?
Stoll: We're focused on improving the quality of the product. Some of the issues that you threw out earlier are higher on our list of things to tackle, so that's where our focus is.