Newspapers, like bloggers, write headlines for search engines

April 9th, 2006

The New York Times has an article This Boring Headline Is Written for Google that talks about how journalists and their editors are now writing headlines with search engines in mind.

“Journalists over the years have assumed they were writing their headlines and articles for two audiences — fickle readers and nitpicking editors. Today, there is a third important arbiter of their work: the software programs that scour the Web, analyzing and ranking online news articles on behalf of Internet search engines like Google, Yahoo and MSN.

The search-engine “bots” that crawl the Web are increasingly influential, delivering 30 percent or more of the traffic on some newspaper, magazine or television news Web sites. And traffic means readers and advertisers, at a time when the mainstream media is desperately trying to make a living on the Web.”

We’ve been trying to do this with good effect on our own ebiquity research site for the past year or so by carefully choosing the titles of web pages and blog posts. We also apply the same principle when choosing titles for research papers to have them match desired search terms. Of course, this has to be done in a principled way — we don’t want to title a paper on Swoogle like “Swoogle: it’s more fun than Britney Spears”. While that might attract more visitors, they aren’t the ones we want. A few might click on our adsense ads which support Mr. Capresso, they are not likely to read and cite our papers. But a title like “Swoogle: searching for knowledge on the Semantic Web” is likely to be better and easier for the right people to find than “Evaluating Swoogle’s performance on finding RDF data”.

These changes are hard to make. Here’s a bit more from the Times article.

“Journalists, they say, would be wise to do a little keyword research to determine the two or three most-searched words that relate to their subject — and then include them in the first few sentences.”That’s not something they teach in journalism schools,” said Danny Sullivan, editor of SearchEngineWatch, an online newsletter. “But in the future, they should.”

Such suggestions stir mixed sentiments. “My first thought is that reporters and editors have a job to do and they shouldn’t worry about what Google’s or Yahoo’s software thinks of their work,” said Michael Schudson, a professor at the University of California, San Diego, who is a visiting faculty member at the Columbia University Graduate School of Journalism.

“But my second thought is that newspaper headlines and the presentation of stories in print are in a sense marketing devices to bring readers to your story,” Mr. Schudson added. ‘Why not use a new marketing device appropriate to the age of the Internet and the search engine?’”

The same applies to research authors. We have to give up, somewhat reluctantly, some old habits — like trying to write clever and memorable titles. By the way, I think my favorite all time AI paper title as Janet Kolodner’s “Car 54 where are you”. But you have to date back to the early 60’s to find that funny. (Sadly, I can’t find a reference to that paper anywhere on the Web. If anyone has the citation information, please send it to me. I think it was from a paper in the late 70s).

Conventional wisdom is that one should also choose file names to produce URLs with the right terms. So we, like many others, configure our software to automatically produce file names using words selected from titles of papers, talks, events, posts, etc. Most MSM organizations are not yet doing that. For example, the Times article that triggered this post has the filename 09lohr. That’s not likely to draw traffic. Compare that to the URL generated by WordPress for this post, newspapers-like-bloggers-write-headlines-for-search-engines.

What’s next? Well, I hope its the appearance of machine understandable metadata in online content — microformats, RDF/A, linkes RDF, etc. Once this becomes a disciminator in getting thigs noticed by your intended audience, the Semantic Web will gain a lot more traction.