Acoustic data is collected from echosounders that measure how sound waves are reflected off the seabed and marine organisms below. The output is annotated by experts and will be utilized to train our deep learning models, but there are many related challenges that needs to be addressed.

The volume of annotations is large, but the quality is variable, and the annotations are not suited for direct use for training deep learning models. Furthermore, characteristics of surveys and equipment may change over time. Finally, there will also be auxiliary information available from the surveys that can provide useful information.

For future use in stock assessment important issues are also challenges related to providing confidence and uncertainty measures and providing explainability and reliability. Solutions will also need to enable prototyping at IMR, where predictions will be tested for use in abundance assessment.

For acoustic data early work has investigated the use of deep learning for semantic segmentation of sand eel schools. Annotated surveys for more species will be made available during 2021.

An activity on acoustic classification of more species with different characteristics will be started. We will also start working on suitable models for exploitation of contextual information, such as how to include trawl samples and other auxiliary information to improve the performance of the network. Another activity related to analysis of acoustic data will focus on semi-supervised approaches. This to reduce the dependency on large amounts of annotated to better deal with an increasing volume of datasets that may present both new characteristics and new species. The aim is to investigate different semi-supervised models and approaches that can provide good prediction performance with only a few annotated data.

Acoustic target classification in multi-frequency echosounder data is a major interest for the marine ecosystem and fishery management since it can potentially estimate the abundance or biomass of the species. A key problem of current methods is the heavy dependence on the manual categorization of data samples. As a solution, we propose a novel semi-supervised deep learning method leveraging a few annotated data samples together with vast amounts of unannotated data samples, all in a single model. Specifically, two inter-connected objectives, namely, a clustering objective and a classification objective, optimize one shared convolutional neural network in an alternating manner. The clustering objective exploits the underlying structure of all data, both annotated and unannotated; the classification objective enforces a certain consistency to given classes using the few annotated data samples. We evaluate our classification method using echosounder data from the sandeel case study in the North Sea. In the semi-supervised setting with only a tenth of the training data annotated, our method achieves 67.6% accuracy, outperforming a conventional semi-supervised method by 7.0 percentage points. When applying the proposed method in a fully supervised setup, we achieve 74.7% accuracy, surpassing the standard supervised deep learning method by 4.7 percentage points.