AI for Seals Challenge
Develop facial recognition CNN models for non-invasive study of harbor seals and other marine mammals, monitoring their population and movement patterns.
Premium participation: Get 1-on-1 Mentoring & Guidance
Try a free mentoring sessionUsing facial recognition CNNs to monitor the population and movement patterns of harbor seals
The dynamic nature of marine ecosystems requires monitoring populations for conservation efforts. methods for long-term monitoring of coastal species, including marine mammals, are often invasive, costly, and time-consuming. Tagging is expensive and often disturbing or even harmful to the individual. Aerial observations are time consuming and expensive as well. The best methods seem to be photo-based identification techniques that are non-invasive. But manual interpretation of photographs is time-intensive and often limited to small-scale projects.
Harbor seals are key regulators and indicators of marine ecosystem health. Accurate monitoring of their population and movement patterns is essential to understanding marine ecosystems. They are relatively easy to monitor via photographic analysis as large numbers of seals can be observed as they congregate out of water at sites like rocky islets, allowing them to thermoregulate and avoid predation. It makes them easily visible to researchers from afar. These observation methods have to be transformed into systematic data collection and analysis to be scalable. This is where computer vision techniques come in handy.
A team from Colgate University used automated facial recognition technology as a system for the identification of seals. Using convolutional neural networks, they developed SealNet, the first marine mammal facial recognition software, to help researchers. But the model isn’t perfect. That's why Colgate University has teamed up with FruitPunch AI for the AI for Seals Challenge to improve SealNet with new techniques to increase its accuracy, robustness and usability.
Info session
Our challenge partner
GOAL: Increase the accuracy and robustness of SealNet, the automated facial recognition technology for identification of seals
The development of facial recognition techniques for harbor seals allows for a rapid, non-invasive means for detailed study of ecologically important species. Improving SealNet will give researchers the ability to customize the software and the GUI to suit their own needs at each step of data collection - training the face detector for additional species, modifying the alignment procedure, or preprocessing images for face recognition.
In this challenge, we will:
- Use facial recognition for seals to do behavioral genetics
- Improve and automate the SealNet workflow by:
- Augmenting clipped faces
- Generating embedding of each face
- Clustering embeddings
- Comparing clusters with know individuals
- Categorizing the individuals - Improve current generation SealNet by using the latest CNN architecture
- Experiment with open datasets
- Investigate Transfer Learning options of models trained for other species, e.g. primates or humans
- Develop GUI for non-technical people
Technologies we will use
- CNNs
- Face recognition
- Clustering
- Transfer Learning
- Face detection
- Image augmentation
Who are we looking for?
We are looking for data science & AI engineers with experience in computer vision. Previous work with face recognition would be of great benefit.
You will collaborate with a diverse team of up to 50 international collaborators in subteams. You can join as a contributor (~12 hours per week commitment for 10 weeks) or coach (2-4 hours per week, only for experienced ML professionals)
We’ll organize a masterclass on SealNet facial recognition during the challenge to bring you up to speed.
Did you know
🦭 The worldwide harbor seal population is estimated to be 350,000 - 500,000 individuals
🏷 Tagging animals may interfere with behaviors such as swimming speed, oxygen consumption, and metabolic rate, potentially corrupting the data collected or harming or disturbing the individual.
🛰 GPS-monitoring devices like tracking collars used for wildlife monitoring are expensive, ranging from $1000 to $3000 for one device
🫥 Why are harbor seals identified by their faces? The identification based on pelage patterns is difficult due to the density of individuals on haul-out sites (you’d need to get a whole-body photo) and to changing coat patterns as seals mature or during annual molting.
Timeline
Application Deadline: 4 Dec 2022
Challenge Kick-off: 6 Dec 2022
Midterm Presentations: 10 Jan 2023
Final Presentations: 13 Feb 2023