Deep learning has made a substantial impact in the seismic community, with self-supervised learning (SSL) being a novel technique of interest. SSL utilizes large amounts of unlabelled data, learning to predict labels generated from the data itself and making it suitable for developing foundation models capable of performing multiple tasks without extensive fine-tuning. While efforts have been made within the seismic community in this direction, the impact of large-scale pre-training and data curation is yet to be explored. This study aims to investigate these aspects using an SSL approach based on masked auto-encoders to robustly learn features. The first study contribution shows that the model pre-trained on seismic data outperforms one pre-trained on natural images. Secondly, we demonstrate the positive effect of data curation and the use of larger datasets for pre-training. Lastly, we showcase the potential of the developed foundation model through two post-stack applications with minimal labelling: reservoir mapping and salt body segmentation.

‍