Horizon / Plein textes La base de ressources documentaires de l'IRD

IRD

Publications des scientifiques de l'IRD

Leroy B., Delsol R., Hugueny Bernard, Meynard C. N., Barhoumi C., Barbet-Massin M., Bellard C. (2018). Without quality presence-absence data, discrimination metrics such as TSS can be misleading measures of model performance. Journal of Biogeography, 45 (9), 1994-2002. ISSN 0305-0270

Accès réservé (Intranet IRD) Document en accès réservé (Intranet IRD)

Lien direct chez l'éditeur doi:10.1111/jbi.13402

Titre
Without quality presence-absence data, discrimination metrics such as TSS can be misleading measures of model performance
Année de publication2018
Type de documentArticle référencé dans le Web of Science WOS:000443394200002
AuteursLeroy B., Delsol R., Hugueny Bernard, Meynard C. N., Barhoumi C., Barbet-Massin M., Bellard C.
SourceJournal of Biogeography, 2018, 45 (9), p. 1994-2002. ISSN 0305-0270
RésuméThe discriminating capacity (i.e. ability to correctly classify presences and absences) of species distribution models (SDMs) is commonly evaluated with metrics such as the area under the receiving operating characteristic curve (AUC), the Kappa statistic and the true skill statistic (TSS). AUC and Kappa have been repeatedly criticized, but TSS has fared relatively well since its introduction, mainly because it has been considered as independent of prevalence. In addition, discrimination metrics have been contested because they should be calculated on presence-absence data, but are often used on presence-only or presence-background data. Here, we investigate TSS and an alternative set of metricssimilarity indices, also known as F-measures. We first show that even in ideal conditions (i.e. perfectly random presence-absence sampling), TSS can be misleading because of its dependence on prevalence, whereas similarity/F-measures provide adequate estimations of model discrimination capacity. Second, we show that in real-world situations where sample prevalence is different from true species prevalence (i.e. biased sampling or presence-pseudoabsence), no discrimination capacity metric provides adequate estimation of model discrimination capacity, including metrics specifically designed for modelling with presence-pseudoabsence data. Our conclusions are twofold. First, they unequivocally impel SDM users to understand the potential shortcomings of discrimination metrics when quality presence-absence data are lacking, and we recommend obtaining such data. Second, in the specific case of virtual species, which are increasingly used to develop and test SDM methodologies, we strongly recommend the use of similarity/F-measures, which were not biased by prevalence, contrary to TSS.
Plan de classementEtudes, transformation, conservation du milieu naturel [082] ; Sciences fondamentales / Techniques d'analyse et de recherche [020]
LocalisationFonds IRD [F B010073978]
Identifiant IRDfdi:010073978
Lien permanenthttp://www.documentation.ird.fr/hor/fdi:010073978

Export des données

Disponibilité des documents

Télechargment fichier PDF téléchargeable

Lien sur le Web lien chez l'éditeur

Accès réservé en accès réservé

HAL en libre accès sur HAL


Accès aux documents originaux :

Le FDI est labellisé CollEx

Accès direct

Bureau du chercheur

Site de la documentation

Espace intranet IST (accès réservé)

Suivi des publications IRD (accès réservé)

Mentions légales

Services Horizon

Poser une question

Consulter l'aide en ligne

Déposer une publication (accès réservé)

S'abonner au flux RSS

Voir les tableaux chronologiques et thématiques

Centres de documentation

Bondy

Montpellier (centre IRD)

Montpellier (MSE)

Cayenne

Nouméa

Papeete

Abidjan

Dakar

Niamey

Ouagadougou

Tunis

La Paz

Quito