May 14, 2024
The Bear Necessity of AI in Conservation
The AI for Bears Challenge results which aims to improve the monitoring and identification of bears using advanced computer vision techniques.
This challenge focused primarily on two eagle species: the golden eagle Aquila chrysaetos and the white-tailed eagle Haliaeetus albicilla. But why focus on eagles? The answer is pretty simple: they are not only the rulers of the skies but as top predators, they act as keystone species of an ecosystem. A keystone species influences an entire ecosystem’s food chain in profound ways. So, to understand the population dynamics of eagles also means to understand the entire food chain of an ecosystem and the resulting feedback with the environment. There is a famous example of the influence a top predator has on an entire ecosystem: When wolves were re-introduced into Yosemite National Park, they stopped the deer population from growing rapidly, which then caused a recovery of young forests because deers love to eat small tree saplings.
Our primary task was to build a classifier to identify crucial aspects of eagle behavior and demographics. For this, we targeted three sub-tasks: recognition if a bird is flying or sitting, identification of the bird’s species, and age classification for golden eagles (juvenile, sub-adult, or adult). This complex challenge necessitated a multi-faceted approach, from creating a well-balanced dataset all the way to implementing and fine-tuning our models.
We started out with a dataset of about 70’000 images of birds in various behaviors. About 56’000 images were from golden eagles and 14’000 from white-tailed eagles. To begin, we needed to filter through these images rapidly to see whether they actually contained a bird. To do this, we initiated the processing pipeline with a YOLOv8 model. A powerful object detection model that elegantly cropped images, setting the stage for subsequent classifications.
Next in line was to create a classifier that differentiates flying birds from other behaviors. This is a crucial step of the pipeline because the features needed, especially for age classification, are best visible when an eagle is in flight. Luckily, the original dataset yielded plenty of labels for the bird's behaviour, allowing us to build a robust classifier.
Species identification, a critical aspect, involved developing a model capable of distinguishing between golden eagles and white-tailed eagles. For this, we employed The Vision Transformer with a large model size (ViT_l16), a powerhouse in image classification. Its self-attention mechanism enables a holistic understanding of images, capturing global context simultaneously - an excellent prerequisite to detecting individual features of a bird that give away its species.
Again, thanks to the largely labeled dataset, we achieved a robust model to distinguish the two eagle species from one another. Validating the attention mechanism of our model, we could confirm that the model focused correctly on key features. For example, as given away by its name, adult white-tailed eagles have very distinct and bright white tails. Plus, their beacon turns yellower as they age. In contrast, the golden eagle tends to have darker feathers and a black pattern on its tail. As visible in the pictures below, the model paid attention mainly to the head and tail, as was to be expected.
Understanding the age classes of eagles required us to first dig deeper into the complex evolution of a bird’s gown as it ages. Typically, birds shed their feathers in an annual cycle, making their patterns various and complex. On top of this, the color, shading, and patterns can vary enormously across populations, making this classification an even more challenging task. Unfortunately, the original dataset did not have many age labels, meaning we had to start labeling ourselves. Quickly, we realized that our limited knowledge of age classification is a significant bottleneck for the age classifier.
So, we tapped into the great network of the SLU and asked how we could speed up this labeling process. The result of many discussions was the idea of an online platform where seasoned and well-trained ornithologists could quickly annotate pictures. We also added features so that users could play with the brightness and contrast of pictures to give a better age guess. After many manhours of work, we ended with around 500 annotated images. Unfortunately, this dataset was balanced heavily towards adult animals, which limited our model's predictive capabilities. Nonetheless, the entire setup for more annotating is set up and passionate people at SLU can take it from here to improve the model’s performance.
Taking together all our classifiers, we aimed further to create the backbone of a continuous implementation. This means not only that we can now continuously label data, rerun all models, and improve their performances. But also that we created an interface where anyone in the world can upload their pictures of eagles, let them be classified by our models, and then receive a complete report on this classification, plus the option to give feedback on it.
As we worked on this project tirelessly over several months, we heavily relied on and benefited from everyone’s expertise. Passionate data engineers worked away to clean the dataset by finding ways to remove faulty crops, blurred pictures and corrupted files. This improved dataset was then utilized by the skilled model builder team, which applied different data augmentation techniques and AI models to find the model that worked best. Finally, thanks to the web developers, we were able to host and display our work online to further improve our data and models. Last but not least, the professional guidance of the SLU with their passionate ornithologists provided essential insights into how we can best make use of their knowledge, and they always provided majestic motivating pictures of eagles they were monitoring actively in the field.
Overall, this AI for Good challenge was a great success. Naturally, everyone on the team learned so much from each other about ecology and data science. But what is even more important is that we built an excellent basis for an integrated behaviour-species-age classification routine that can be utilized by the SLU to study better the complex relationships between top predators and their ecosystems.
We could not have completed this Challenge without the amazing efforts from: Navinder Singh, Amee Assad, Deepak Doddera, Pascal Schneider, Rizdi Aprilian, Tom Groot, Anh Nguyen, Sonny Burniston, Jeandre de Bruyn, Kush Aswani, Prasad Jettiboina and eBird
