Cost-Sensitive Classifier Evaluation Using Cost Curves

The evaluation of classifier performance in a cost-sensitive setting is straightforward if the operating conditions (misclassification costs and class distributions) are fixed and known. When this is not the case, evaluation requires a method of visualizing classifier performance across the full range of possible operating conditions. This talk argues that the classic technique for classifier performance visualization -- the ROC curve – is inadequate for the needs of researchers and practitioners in several important respects. It then describes a different way of visualizing classifier performance -- the cost curve -- that overcomes these deficiencies. No familiarity with ROC curves or cost curves is necessary, they will be fully explained.