Semantic Discovery: Discovering Complex Relationships in Semantic Web

October 1, 2003 - October 1, 2008

Research in search techniques was a critical component of the first generation of the Web, and has gone from academe to mainstream. A second generation Semantic Web will be built by adding semantic annotations that software can understand and from which humans can benefit. Modeling, discovering and reasoning about complex relationships on the Semantic Web will enable this vision and transform the hunt for documents into a more automated analysis enabled by semantic technology. The beginnings of this shift from search to analysis can be observed in research and industry as users look beyond finding relevant documents based on keywords to finding actionable information leading to decision making and insights. Large scale semantic annotation of data (domain-independent and domain-specific) is now possible because of an accumulation of advances in entity identification, automatic classification, taxonomy and ontology development, and metadata extraction. The next frontier, which fundamentally changes the way we acquire and use knowledge, is to automatically identify complex relationships between entities in this semantically annotated data. Instead of a search engine that returns documents containing terms of interest, we envision a system that returns actionable information (with the associated sources and supporting evidence) to a user or application. The user interacts with information universe through a hypothesis driven approach that combines search and inferencing, enabling more complex analysis and deeper insight. The examples in our narrative show that such a capability also greatly enhances the capacity of intelligence analysts to obtain (in time) information leading to a more secure homeland and world.

Our research will focus on the design, prototyping and evaluation of a system, called SemDIS (Semantic Discovery) that supports indexing and querying of complex semantic relationships and is driven by notions of information trust and provenance and models of hypotheses and arguments under investigation.

From scientific perspective, we face the challenges of formally defining and representing meaningful and interesting relationships (which we call semantic associations), and defining the notion of quality of results similar to the familiar metrics of precision, recall and document ranking. Another challenge is the (semi) automatic construction of argument structures built on these relationships to validate or deny a given hypothesis. Additional scientific and engineering challenges include those related to the scale of storing and complex query processing of large metadata sets, with corresponding more complex data structures to represent entities and relationships, the need to utilize context to select relevant subsets of metadata to process, and new techniques that use information provenance and trust to improve ranking of relationships. These challenges call for a fresh look at indexing, query processing, ranking, as well as tractable and scalable graph algorithms that exploit heuristics. Our work proposes to address these challenges building on our preliminary results in semantic metadata extraction, practical domain-specific ontology creations, defining semantic associations, main-memory query processing, using distributed trust to enforce security policies, and knowledge representation and reasoning on the semantic web. Scientific results from SemDIS will involve detailed scenarios and an evaluation testbed, and will be measured in terms of novel techniques as well as performance metrics and measures of quality, scalability and performance for computing complex semantic relationships. Corresponding to the breadth and depth of the topics involved in the challenge undertaken, ours is a collaborative proposal involving researchers at UGA and UMBC, covering the areas of information modeling and knowledge representation, storage and database management, information retrieval and artificial intelligence.

Our effort will have broader impacts beyond the education and training of graduate students, and the publication of research findings. Results from our research will be integrated with courses we teach, both existing and new. We will use institutional mechanisms in place to seek participation of students from underrepresented groups. Datasets used for testbed evaluations and some of the targeted tools will be made public or open source, and new measures for relevance and ranking of semantic associations will provide input to future work on comparing various approaches and techniques. Our work will also gain from several academic-industry collaborations of the investigators. We will have the opportunity to leverage commercial infrastructure and raw metadata provided by Semagix and IBM and, when appropriate, technology licensing will be encouraged. The researchers will collaborate with industry, and the students will be encouraged to intern at collaborating industrial labs. Within a broader social context, emerging knowledge-centric technologies raise legitimate privacy and civil liberties concerns. Building upon past policy making experience, we will comment on potential implications of our scientific progress. This research is supported in part by an NSF award ITR 0325172, and is a collaborative effort with colleagues at U. Georgia and Wright State University . Some demos associated with our efforts to find trust using DBLP and FOAF data can be found at the project's web page. We've also developed related software systems, supported by this and other awards, such as a semantic web search engine Swoogle, Community/sentiment/trust detection systems such as Feeds that matter and PolVox, amongst others.

Tags: blog, blogosphere, foaf, information integration, semantic web, splog, trust

OWL Tweet