Earth observation

Earth observation

We develop deep learning models for monitoring and prediction of objects, hazard risks and streamlining aerial surveys.

Motivation

Earth observation sensors provide vast amounts of data. Our planet is covered by observations from satellites, airplanes, helicopters, and unmanned aircrafts like drones, providing details down to the centimetre scale.

Exploiting this data has great potential for critical applications like environmental monitoring, situational awareness, and mapping illegal vessel activities, which are of key importance to our user partners.

Our innovations

Our deep learning research within the earth observation domain contributes to new innovative solutions for the mentioned purposes, including approaches for:

• producing super-resolution multi-spectral Sentinel 2 satellite images, in collaboration with user partner Kongsberg Satellite Services (KSAT).

This method enables more photo-realistic imagery when analyzing such imagery, paving the way for more precise analysis of satellite imagery for downstream applications.

• multi-stage detection of vessels, together with KSAT.

This approach, dubbed YOLOF, represents an efficient and promising pipeline for identifying vessels at sea.

• detecting oil spills and characterizing their thickness, together with KSAT.

This method automatically segments oil spills in real-life scenarios, which could lead to faster detection of environmental hazards in the ocean.

• reducing missed oil spills with deep learning, in collaboration with KSAT.

Closely related to our work on oil spills, this method relies on computing distances to oil spills, representing novel methodology that can help reduce the number of missed spills.

• improving the efficiency of deep learning-based sea ice forecasting models, together with the Alan Turing Institute and The British Antarctic Survey

This method uses explainability (XAI) techniques like gradient saliency maps to reduce the IceNet’s – the state-of-the-art sea ice forecasting model – input features. The work highlights XAI’s potential in refining deep learning models and offers insights for broader applications in climate research and beyond. Read more.

Addressing research challenges

Limited and incomplete training data is a general problem in remote sensing. Combinations of multi-sensor data, such as from optical and radar sensors, and time dependencies is another key challenges. The mentioned methods address these challenges in different ways.

For instance, our oil spill detection method is a step towards quantifying deep learning models’ uncertainty when analyzing remote sensing data. Our method for improving the efficiacy of the IceNet model incorporates explainability techniques that offer invaluable information about the model’s performance and predictions.

‍

Synergies within the innovation area and across innovation areas

As for all image analysis applications, the development of deep learning methodology to solve certain tasks in earth observation often benefits from solutions developed to solve other problems. For instance, when developing methods for building segmentation, ideas from segmentation of oil spills can potentially be transferred.  

It can also be valuable to reveal any different behaviour of segmentation algorithms due to different properties in data sources. Aerial imagery and satellite imagery come with different resolution, contrast, noise properties, and by contrasting seemingly similar deep learning methods, the influence of different data properties may be revealed and better understood.

We have developed new methods and obtained insights into how one can use self-supervised learning when there are images acquired at two or more times available, both as a pre-training step and as an integral part of change detection. The insights and experience gained here on the concept of self-supervised learning in general has a wide range of other relevant applications. We have explored similar methodology within seismic analysis, where it has been utilized to identify and characterize geological regions in vast seismic datasets.