Predicting seafloor visual classes from multimodal remote sensed priors using location-guided self-supervised Learning

Remote sensed mapping data and seafloor in-situ imagery are often gathered to infer benthic habitat distributions. However, leveraging multimodal data is challenging because of inherent inconsistencies between measurement modes (e.g., resolution, positional offsets, shape discrepancies). We investigate the impact of using location metadata in multimodal, self-supervised feature learning on habitat classification. Experiments were carried out on a multimodal dataset gathered using and Autonomous Underwater Vehicle (AUV) at the Darwin Mounds Marine Protected Area (MPA). Introducing location metadata improved F1 classification performance of a Bayesian classifier by an average of 27.7% over all conditions tested in this work, with a larger improvement of 32.9% achieved when multiple remote sensing data modes are combined for the analysis.