Oil spills are a significant threat to marine ecosystems, wildlife, and public health. Ships sinking, boats crashing, or leaking tankers pollute our rivers and ports, causing ecological damage. In our latest collaboration with Rijkswaterstaat (The Dutch Ministry of Infrastructure and Waterways), we aimed to use Artificial Intelligence to improve the response times of their Oil Clean-up teams. By working with drone-captured RGB images, we aimed to develop an advanced oil-volume estimation tool that would enhance the efficiency and accuracy of oil spill assessments. This tool would also facilitate prompt and effective response measures to minimize the damage caused by oil spills.

The outcome of our challenge holds the potential to revolutionize environmental protection efforts. An efficient and precise tool to assess oil spills can accelerate cleanup operations, reduce ecological harm, and safeguard marine biodiversity. With the help of AI analysis, we can accurately estimate the volume of oil, allocate resources accordingly, deploy effective containment and remediation strategies, and ultimately restore the impacted areas.

Goal for this Challenge

Detecting and cleaning up inland oil spills is crucial. To aid in this process, we aimed to create computer vision models that could analyze drone imagery and estimate the volume of oil spills. By providing accurate and rapid information, these models can assist clean-up teams in making informed decisions and taking appropriate actions.

Hurdles we had to overcome

Incorrect labels and overall difficult task of oil classification: 

Several challenges affected the usability of the provided images with annotation, primarily due to issues with annotation quality and visibility of the oil spills. Regarding annotation quality, we encountered two main problems: incorrect annotations, where labels did not accurately reflect the content, and inconsistent annotations, where similar features were labeled differently across images. These issues are illustrated in Figure 1. Additionally, the visibility of oil spills presented significant difficulties. In many instances, the oil spills were not discernible, and determining the precise boundaries of the spills was challenging in several images, as demonstrated in Figure 2. Due to these limitations, a substantial number of images were deemed unsuitable for our analysis, resulting in a final dataset of 90 adequately annotated images.

Given the initial shortage of usable images, efforts were made to augment the dataset through semi-supervised labeling. This enriched dataset played a crucial role in improving the models' training and overall performance​​. We started with a bare 90 images and were able to iterate the semi-supervised approach until we reached a dataset 3 times the size! 

Finding a suitable segmentation model for our limited dataset: 

Experimenting with Unet, Unet++, SAM, and YOLO models, we tried to differentiate between oil and non-oil elements within images. The best results were achieved with Unet++, after parameter optimization we reached an MIoU of .67

Volume Estimation: 

The variable that is of interest to Rijkswaterstaat is an estimate of the volume of the oil spill. To get there we need to post-process the number of detected oil pixels to a volume.

Assuming that:

A) the drone camera is filming straight down and 

B) the drone operators use no (digital) zoom, 

we can use: 

1) the field of view (FOV) of the drone camera, 

2) the specified image width of the resulting images and 

3) the altitude of the drone with respect to the water surface, provided via GPS, to calculate the conversion from image pixels to square meters.

‍

From the drone provider DJI, we know that the DJI M3T used by RWS has two cameras. The telecamera, when in full resolution, with no zoom, covers 4000x3000 pixels and has a FOV of 15 degrees. By photographing a reference point that is also visible in Google Maps at different drone altitudes, one can calibrate the cm/pixel conversion.

Reflecting on the challenge's outcomes

The potential impact of successfully implementing AI in oil spill assessments is very promising. Looking specifically at the AI models it is possible to train accurate and fast tools to help in the oil cleanup. But this does require the right dataset. As we’ve seen in this Challenge it can be tricky to accurately annotate oil spills. Influences like weather, waves, or trash can make it difficult to classify what is oil and what isn’t, even for the trained human eye. Moving forward we have provided a list of recommendations for RWS in Data gathering and labeling that can help them take the next step! 

Big thanks to everyone who made this Challenge a success! 

Hamzah Al-Qadasi, Laurenz Schindler, Robert Korpershoek, Gerson Foks, Neacail Hilhorst, Nik de Geus, Purva Marfatia, Rafaël Mostert, Ruurd Kuiper, Soogeun Park, Sujatha Sreekumar, Sharadhi A.S, Maurice van Ginneken, Julian Bakker, Steve Vanlanduit, Robbert Heinecke, Svetlana Samsonova, Frederik Winters