Tech for Wildlife: MegaDetector (Part 2 of 2) – Improving MegaDetector results (Theoretical/Hypothetical) ????

About half a year ago, we had a meeting with Serge Belongie , where I represented Really A Robot and spoke about our Animal Detect plans and ideas (which have changed a lot since then). Serge is one of the leading figures in AI, and we in Denmark got lucky to have "imported” him to be the director of the Pioneer Centre of AI. He has been deeply involved in AI for animals and wildlife for a while, including his work with iNaturalist and Merlin. ????

Anyways, enough fanboying – one of the takeaways from the meeting that stuck with me was: “Use MegaDetector as a benchmark and see how well you compare.”

Now a bit wiser, I realized I wouldn't want to collect millions of images, label, customize, optimize, and train a neural network to create a model worthy of competing against MegaDetector. You can read my first part of MegaDetector (Part 1 of 2) Here to learn a bit about what MegaDetector is, along with some use cases.

The idea of benchmarking with MegaDetector really stuck with me, and instead of making my own MegaDetector model, I thought about seeing if I could get even better results from MegaDetector itself instead. ?????

Understanding Neural Networks ????

To understand a bit about how neural networks work (without diving into backpropagation formulas like the pain I went through during my master’s thesis):

We have an input image. This image goes through “layers” of the neural network, where different operations take place, looking for shapes, lines, and patterns using various image processing steps. At each step, a neuron is activated, leading to the final layers where the result is presented. If you really want to learn more about neural networks, I highly recommend the 3Blue1Brown YouTube channel, which also helped me through my exams. He also has a video on backpropagation! ????

Alright Hugo, get to the point and stop yapping about neural networks! ??

The Core Idea ????

MegaDetector has been trained on millions of images from different resolutions, brands, etc., but maybe there’s an invisible bias towards some types of images. What if we modify our images to match that bias the most to increase our chances of getting a correct detection? ????

Just look at the GIF of images – a slight modification can completely erase the detection or strengthen it! (Please scroll a bit more down to see it)

So, my hypothesis is: even if the neural network performs numerous operations to minimize image variance, would modifying my input have a direct effect? The short answer is yes – looks like it! The long one is #IWishIHadMoreTimeToPlayAround. ???

Testing the Hypothesis ????

Obviously, I had to test this partially out which I did a while ago and, stupid me, didn’t save the folder with code and images. But I found some images from an email and some notes on the test and results:

Test Setup:

• 200 images: 100 for “training” and 100 for validation.
• The images selected were ones where MegaDetector had no or very poor confidence on images with actual animals.
• Baseline detection: With a 0.1 confidence threshold, MegaDetector had only a 64% true positive rate on these images.

Training Process:

1. Used MegaDetector combined with some manual labeling of animals, so all 100 training images had only true positive animals saved to a JSON file.
2. Adjusted three parameters: sharpness, contrast, and gamma. Created 100 different random settings for these parameters (I honestly can’t recall the exact ranges, which isn’t very scientific of me).
3. Applied these settings to all 100 training images, resulting in 10,000 images in total.
4. Ran MegaDetector on all modified images, saved the results, and compared:

Does the detector find an animal?

How close is the bounding box compared to the ground truth?

What confidence does the bounding box have?

Based on these three parameters, I asked ChatGPT (sorry for the laziness ??) to write a code to find the optimal settings of sharpness, contrast, and gamma that could identify the most correct animals while maintaining or improving confidence levels and avoiding false positives. With these settings, I ran the tests on all 200 images (should have maybe just used the validation data – my bad ??) and achieved an 88.5% detection rate (177 out of 200 images) correctly finding animals, with no false positives!

Conclusion and Discussion ????

As mentioned, even though I managed to get improved results, the work is still theoretical/hypothetical. I can’t see myself finding the time anytime soon to continue the idea of "improving MegaDetector results". Therefore, I would love to see others get inspired to explore what can be done. ???

My test limitations:

1. Used a limited number of images from the same camera brand and three locations. Applying the same settings to other images, might not yield as promising results.
2. Focused on images with poor/no detection, so I didn’t include images with strong detections to see if these settings would negatively impact them.
3. Use of Human-in-the-Loop. The approach included manual labeling of the "training" set, which isn’t ideal for scaling. ???

Future Aspirations: I hope to see ways to find the “ideal” settings for each individual image for the best detections and modify them dynamically, not with static numbers, to boost performance. ???

This could also be combined with other AI technologies, such as Generative Adversarial Networks (GANs), which can be used to colorize grayscale images or upscale images to enhance detection results, as I illustrated in my first post Recoloring of grayscale wildlife image.

Please let me know if you enjoy these hypothetical posts or if you’d prefer I stick to existing and developed AI technologies! ????

#TechForWildlife #AI #MachineLearning #WildlifeConservation #MegaDetector #NeuralNetworks #AIInnovation #WildlifeTechnology #DeepLearning #TechForGood #ArtificialIntelligence #WildlifeResearch #ConservationTech #Robotics #GenerativeAI