Assisting human annotation of marine images with foundation models

Orenstein, Eric C.; Woodward, Benjamin; Lundsten, Lonny; Barnard, Kevin; Schlining, Brian; Katjia, Kakani

Assisting human annotation of marine images with foundation models

Orenstein, Eric C.; Woodward, Benjamin; Lundsten, Lonny; Barnard, Kevin; Schlining, Brian; Katjia, Kakani. 2025 Assisting human annotation of marine images with foundation models. Frontiers in Marine Science, 12. 10.3389/fmars.2025.1469396

Before downloading, please read NORA policies.

Preview

Text
© 2025 Orenstein, Woodward, Lundsten, Barnard, Schlining and Katjia. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
fmars-2-1469396.pdf - Published Version
Available under License Creative Commons Attribution 4.0.
Download (3MB) | Preview

Official URL: https://doi.org/10.3389/fmars.2025.1469396

Abstract/Summary

Marine scientists have been leveraging supervised machine learning algorithms to analyze image and video data for nearly two decades. There have been many advances, but the cost of generating expert human annotations to train new models remains extremely high. There is broad recognition both in computer and domain sciences that generating training data remains the major bottleneck when developing ML models for targeted tasks. Increasingly, computer scientists are not attempting to produce highly-optimized models from general annotation frameworks, instead focusing on adaptation strategies to tackle new data challenges. Taking inspiration from large language models, computer vision researchers are now thinking in terms of “foundation models” that can yield reasonable zero- and few-shot detection and segmentation performance with human prompting. Here we consider the utility of this approach for ocean imagery, leveraging Meta’s Segment Anything Model to enrich ocean image annotations based on existing labels. This workflow yields promising results, especially for modernizing existing data repositories. Moreover, it suggests that future human annotation efforts could use foundation models to speed progress toward a sufficient training set to address domain specific problems.

Item Type:

Publication - Article

Digital Object Identifier (DOI):

10.3389/fmars.2025.1469396

ISSN:

2296-7745

Additional Keywords:

foundation model, marine imagery, segmentation, object detection, human-in-the-loop

NORA Subject Terms:

Marine Sciences

Date made live:

01 Sep 2025 14:45 +0 (UTC)