UMBC ebiquity
Google, structured data and the deep web

Google, structured data and the deep web

Tim Finin, 4:54pm 4 February 2009

IDG news service has a story sketching how Google Researcher Targets Web’s Structured Data. This is not directed at data published in machine understandable form (e.g., in RDF), but on other kinds of structured data accessible on the web.

“Internet search engines have focused largely on crawling text on Web pages, but Google is knee-deep in research about how to analyze and organize structured data, a company scientist said Friday. “There’s a lot of structured data out on the Web and we’re not doing a good job of presenting it to our users,” said Alon Halevy during a talk at the New England Database Day conference at the Massachusetts Institute of Technology,

Halevy was referring in part to so-called “deep Web” sources, such as the databases that sit behind form-driven Web sites like Cars.com or Realtor.com. Google has been submitting queries to various forms for some time, retrieving the resulting Web pages and including them in its search index if the information looks useful.

But the company also wants to analyze the data found in structured tables on many Web sites, Halevy said, offering as an example a table on a Web page that lists the U.S. presidents. And there are reams of those tables — Google’s index turned up 14 billion of them, according to Halevy. He “realized very quickly that over 98 percent of these are not that interesting,” but even after significant filtering there remain about 154 million tables worth indexing, he said.

ReadWriteWeb also has a story (Google: “We’re Not Doing a Good Job with Structured Data”)on that Google is or isn’t doing with structured data, including an interesting admission by Google researcher Halevy.

“During a talk at the New England Database Day conference at the Massachusetts Institute of Technology, Google’s Alon Halevy admitted that the search giant has “not been doing a good job” presenting the structured data found on the web to its users. By “structured data,” Halevy was referring to the databases of the “deep web” – those internet resources that sit behind forms and site-specific search boxes, unable to be indexed through passive means.”

For some technical details on the issues and current work, see the paper Google’s DeepWeb Crawl by researchers from Google (including Halevy), UCSD and Cornell published in the Proceedings of VLDB 2009.


Comments are closed.