Improving Binary Classification on Text Problems using Differential Word Features

Authors: Justin Martineau, Tim Finin, Anupam Joshi, and Shamit Patel

Book Title: Proceedings of the 18th ACM Conference on Information and Knowledge Management

Date: November 02, 2009

Abstract: We describe an efficient technique to weigh word-based features in binary classification tasks and show that it significantly improves classification accuracy on a range of problems. The most common text classification approach uses a document's ngrams (words and short phrases) as its features and assigns feature values equal to their frequency or TFIDF score relative to the training corpus. Our approach uses values computed as the product of an ngram's document frequency and the difference of its inverse document frequencies in the positive and negative training sets. While this technique is remarkably easy to implement, it gives a statistically significant improvement over the standard bag-of-words approaches using support vector machines on a range of classification tasks. Our results show that our technique is robust and broadly applicable. We provide an analysis of why the approach works and how it can generalize to other domains and problems.

This is a preprint of a short (poster) paper to appear in the Proceedings of the 18th ACM Conference on Information and Knowledge Management, Hong Kong, 2-6 November 2009.

Type: InProceedings

Publisher: ACM Press

Tags: text classification, learning, sentiment, natural language processing, natural language processing, learning

Google Scholar: q7_Ee-QysQIJ

Number of Google Scholar citations: 4 [show citations]

Number of downloads: 1412

Available for download as

size: 103482 bytes

size: 1876480 bytes

Bookmark at: Digg | Del.icio.us | Connotea | CiteULike