CMU AI talk: Cost-sensitive Classifier Evaluation Using Cost Curves

Speaker: Robert C. Holte
When: Monday, September 25, 2006 at 3:30p
Where: Newell Simon Hall 1305

The evaluation of classifier performance in a cost-sensitive setting is straightforward if the operating conditions (misclassification costs and class distributions) are fixed and known. When this is not the case, evaluation requires a method of visualizing classifier performance across the full range of possible operating conditions. This talk argues that the classic technique for classifier performance visualization -- the ROC curve -- is inadequate for the needs of researchers and practitioners in several important respects. It then describes a different way of visualizing classifier performance -- the cost curve -- that overcomes these deficiencies. No familiarity with ROC curves or cost curves is necessary, they will be fully explained. Joint work with Chris Drummond (National Research Council, Ottawa)