UMBC ebiquity

Clustering short status messages: A topic model based approach

Authors: Anand Karandikar

Date: July 17, 2010

Abstract: Recently, there has been an exponential rise in the use of online social media systems like Twitter and Facebook. Even more usage has been observed during events related to natural disasters, political turmoil or other such crises. Tweets or status messages are short and may not carry enough contextual clues. Hence, applying traditional natural language processing algorithms on such data is challenging. Topic model is a popular method for modeling term frequency occurrences for documents in a given corpus. A topic basically consists of set of words that co-occur frequently. Unsupervised nature allows topic models to be trained easily on datasets meant for specific domains. We use the topic modeling feature of MALLET - a machine learning tool kit, to generate topic models from unlabelled data. We propose a way to cluster tweets by using the topic distributions in each tweet. We address the problem of determining which topic model is optimal for clustering tweets based on its clustering performances. We also demonstrate a use case wherein we cluster Twitter users based on the content they tweet. We back our research with experimental results and evaluations.

Type: MastersThesis

Organization: Computer Science and Electrical Engineering

Publisher: University of Maryland, Baltimore County

Tags: twitter, clustering, topic model

Google Scholar: search

Number of downloads: 8168

 

Available for download as


size: 839149 bytes

size: 1096696 bytes