Publications des scientifiques de l'IRD

Hanczar B., Zucker Jean-Daniel. (2018). An approach to optimizing abstaining area for small sample data classification. Expert Systems with Applications, 95, p. 153-161. ISSN 0957-4174.

Titre du document
An approach to optimizing abstaining area for small sample data classification
Année de publication
2018
Type de document
Article référencé dans le Web of Science WOS:000423635700012
Auteurs
Hanczar B., Zucker Jean-Daniel
Source
Expert Systems with Applications, 2018, 95, p. 153-161 ISSN 0957-4174
Given a classification task, an approach to improve accuracy relies on the use of abstaining classifiers. These classifiers are trained to reject observations for which predicted values are not reliable enough: these rejected observations belong to an abstaining area in the feature space. Two equivalent methods exist to theoretically compute the optimal abstaining area for a given classification problem. The first one is based on the posterior probability computed by the model and the other is based on the derivative of the ROC function of the model. Although the second method has proved to give the best results, in small-sample settings such as the one found in omits data, the estimation of posterior probabilities and derivative of ROC curve are both lacking of precision leading to far from optimal abstaining areas. As a consequence none of the two methods bring the expected improvements in accuracy. We propose five alternative algorithms to compute the abstaining area adapted to small-sample problems. The idea of these algorithms is to compute an accurate and robust estimation of the ROC curve and its derivatives. These estimation are mainly based on the assumption that the distribution of the output of the classifier for each class is normal or mixture of normal distributions. These distributions are estimated by a kernel density estimator or Bayesian semiparametric estimator. Another method works on the approximation of the convex hull of the ROC curve. Once the derivative of the ROC curve are estimated, the optimal abstaining area can be directly computed. The performance of our algorithms are directly related to their capacity to compute an accurate estimation of the ROC curve. A sensitivity analysis of our methods to the dataset size and rejection cost has been done on a set of experiments. We show that our methods improve the performances of the abstaining classifiers on several real datasets and for different learning algorithms.
Plan de classement
Sciences fondamentales / Techniques d'analyse et de recherche [020]
Localisation
Fonds IRD [F B010072026]
Identifiant IRD
fdi:010072026
Contact