On the impact of Citizen Science-derived data quality on deep learning based classification in marine images

Deniz, Cem M.; Langenkämper, Daniel; Simon-Lledó, Erik ORCID: https://orcid.org/0000-0001-9667-2917; Hosking, Brett; Jones, Daniel O. B. ORCID: https://orcid.org/0000-0001-5218-1649; Nattkemper, Tim W.. 2019 On the impact of Citizen Science-derived data quality on deep learning based classification in marine images. PLOS ONE, 14 (6). e0218086. 10.1371/journal.pone.0218086

Before downloading, please read NORA policies.

Preview

Text
journal.pone.0218086.pdf
Available under License Creative Commons Attribution 4.0.
Download (3MB) | Preview

Official URL: http://dx.doi.org/10.1371/journal.pone.0218086

Abstract/Summary

The evaluation of large amounts of digital image data is of growing importance for biology, including for the exploration and monitoring of marine habitats. However, only a tiny percentage of the image data collected is evaluated by marine biologists who manually interpret and annotate the image contents, which can be slow and laborious. In order to overcome the bottleneck in image annotation, two strategies are increasingly proposed: “citizen science” and “machine learning”. In this study, we investigated how the combination of citizen science, to detect objects, and machine learning, to classify megafauna, could be used to automate annotation of underwater images. For this purpose, multiple large data sets of citizen science annotations with different degrees of common errors and inaccuracies observed in citizen science data were simulated by modifying “gold standard” annotations done by an experienced marine biologist. The parameters of the simulation were determined on the basis of two citizen science experiments. It allowed us to analyze the relationship between the outcome of a citizen science study and the quality of the classifications of a deep learning megafauna classifier. The results show great potential for combining citizen science with machine learning, provided that the participants are informed precisely about the annotation protocol. Inaccuracies in the position of the annotation had the most substantial influence on the classification accuracy, whereas the size of the marking and false positive detections had a smaller influence.

Item Type:	Publication - Article
Digital Object Identifier (DOI):	10.1371/journal.pone.0218086
ISSN:	1932-6203
Date made live:	06 Aug 2019 14:54 +0 (UTC)
URI:	https://nora.nerc.ac.uk/id/eprint/524644

Item Type:

Publication - Article

Digital Object Identifier (DOI):

10.1371/journal.pone.0218086

ISSN:

1932-6203

Date made live:

06 Aug 2019 14:54 +0 (UTC)

URI:

https://nora.nerc.ac.uk/id/eprint/524644