MS defense: Building a crawler based indexing and retrieval system for the semantic web

The Semantic Web is a collection of documents which is much more structured as compared to the other documents on the World Wide Web. So to get more relevant results from the Semantic Web we need a search engine which takes advantage of the structured nature of the Semantic Web.

Swoogle is a crawler-based indexing and retrieval system for the Semantic Web documents (SWDs). We have built a repository containing the metadata of the Semantic Web Documents available online and we are trying to make it as comprehensive as possible. The system analyzes the documents it has discovered to compute useful metadata properties and relationships between them.

We have created a classification of all the possible relations which can exist between any two given Semantic Web Documents. We have also implemented a Semantic Web Document (SWD) crawler which crawls the web and finds new potential Semantic Web Documents, which are then in turn stored and further analyzed. Thus, we have built a prototype crawler-based indexing and retrieval system for the semantic web.