Diving into the Deep End of Machine Learning

I'm Philip Aguiar, a self-taught web developer deeply passionate about technology. Today, I'm excited to share the journey I'm embarking on with the FastAI course in Deep Learning. This blog will serve as my platform to communicate the challenges and successes I encounter as I weave my web development skills with the transformative potential of AI. Join me as I share the highs, lows, challenges, and breakthroughs in this exciting intersection of web development and AI.

You can follow me Substack to get notified when I post my next update

What is FastAI?

FastAI is a user-friendly, high-level deep learning library built on PyTorch, with a mission to make artificial intelligence accessible to all. This platform simplifies the process of creating and training neural networks, focusing on practicality and hands-on learning through its 100% free, Practical Deep Learning course.

Their Practical Deep Learning course is designed for people with some coding experience, who want to learn how to build and deploy state-of-the-art models for the latest and greatest deep learning and machine learning technologies to practical everyday problems.

Lesson #1 - Getting Started

When I first started the course I was a bit nervous as I was used to learning from the bottom level basics first and slowly building on top of the fundamental knowledge step by step but in this course they flip that learning process by starting on the top level knowledge and get you working on ideas without the full understanding of what’s going on behind the scenes and slowly peel back the curtain each lesson. One of the best features of the course is that it has a lot of interactive explorations without any need for state of the art hardware. For example, in the first lesson, they teach you how to identify pictures of birds from pictures of a forest in a matter of minutes with a 99%+ accuracy.

This process blew my mind at first as I had no previous knowledge of Deep Learning at all and assumed you would need a data set of thousands of images and a dedicated server room to create a program like this but it turns out using my laptop from the couch and about 60 images total I was able to make a near flawless model. Really puts the fast in FastAI.

How does it work?

For basic image recognition we first have to download the images to train from and by naming each file the name of the subject, our model now has a label to differentiate images by. Behind the scenes, the training process goes through each image individually and starts its pattern recognition on what should be labeled a bird vs a forest. In our training dataset, comprising approximately 60 images, we reserve 20% of them for validation purposes.. These withheld images are used for the model to test its categorization abilities as it knows the answer allowing it to fine-tune and improve its accuracy.

My test

When I finished the lesson I wanted to try my own test and remembered a funny idea a friend of mine created in the past called “Broccoli or Weed” that can identify if an image is of broccoli or marijuana as they are often compared in pop culture. As simple as going through the Kaggle document and replacing every mention of the word “bird” with “broccoli” and “forest” with “marijuana” I was able to create a model with ~0% error rate.

You can test this model here.

For fun I wanted to see how good Machine Learning is at differentiating between people. The experiment I wanted to try was comparing an old internet meme that was popular during the heyday of Breaking Bad. When Jesse Plemons character was introduced in Breaking Bad people used to call his character Meth Damon due to his similar look to actor Matt Damon.

You can test this model here.

Key Takeaways

• Creating an AI image detection model was way easier than I ever imagined.
• FastAI is a wonderful free platform that prioritizes teaching the student how to experiment first and slow drip the technical details as needed.
• You don’t need expensive equipment or deep understanding of intricate math to do machine learning. A laptop and the basics of programming is more than enough.
• To train a model we use around 80% of the data set to train the model and 20% is set aside to validate and see if the model can correctly label those images.

Lesson #2 - Deployment

As lesson #1 was mostly a demo to show how simple it is to get started with deep learning, lesson #2 explained more of what was happening behind the scenes as well as the big picture ideas to think about when coming up with a deep learning program. For this lesson, I wanted to try to create a model that didn’t just learn is it X or Y but instead differentiate different types of X. I’ve recently seen a couple youtube videos about the physics of bridges so I decided to experiment if I could use computer vision to identify types of bridges

I used a quick python script to collect photos of each bridge type using DuckDuckGo and used FastAI’s transformer to create the model.

bridge_types = 'arch','suspension','cantilever'
path = Path('bridges')

if not path.exists():
    path.mkdir()
    for o in bridge_types:
        dest = (path/o)
        dest.mkdir(exist_ok=True)
        results = search_images_ddg(f'{o} bridge')
        download_images(dest, urls=results)        

The transformer behind-the-scenes manipulates the images in multiple ways to aid the model in recognizing the essential attributes within each image we are targeting.

This lesson also shows us how to clean an image set so that we use images that are valid to our query. Contrary to my initial assumption, I would have expected to have to manually go through each image one by one before we create a model and delete any images that don’t fit but the best practice is to train the model and look at the outliers and images it had the most challenge labeling.

FastAI features a user-friendly cleaning tool that visually highlights the images where the model struggled the most and an easy interface to delete or reclassify problem images.

Following the lecture’s instructions I was able to create my own python file and deploy it to HuggingFace seen here.

Key Takeaways

• A model can only be as good as the data we give it. For real world products it is important to consider the edge cases of how the model can be wrong~. (Example from the course is a model that detects bears based on google images might have difficulty used in a real life park as factors such as different types of weather/time of day as well as google images always have the subject in clear focus and you won’t likely see a bear from behind.)
• I originally had more types of bridges when trying to create the model but it ended up creating more confusion and the model couldn’t get confidence of what made each type specifically unique. If I am to tackle this problem again in the future I believe I would need to find another way to get a dataset to train from as the random images from a search engine returned many false images.
• Between FastAI’s library making it relatively simple to create a model and Gradio simplicity of launching a quick webpage you can go from a fun idea in a conversation to a showable demo in under an hour with only 15~ lines of code. I’m excited to eventually combine my ability of making full stack websites with these models. The possibilities are endless!

from fastai.vision.all import *
import gradio as gr
learn = load_learner('bridges.pkl')


categories = ('Arch','Cantilever','Suspension')

def classify_image(img):
    pred, idx, probs = learn.predict(img)
    return dict(zip(categories, map(float, probs)))

image = gr.inputs.Image(shape=(192, 192))
label = gr.outputs.Label()

examples = ['arch bridge 1.jpeg', 'arch bridge 2.jpeg','cantilever bridge 1.jpeg','cantilever bridge 2.jpeg','suspension bridge 1.jpeg','suspension bridge 2.webp']

intf = gr.Interface(fn=classify_image, inputs=image,
                    outputs=label, examples=examples)

intf.launch(inline=False)
        

Conclusion

As this was my first ever blog post, I would appreciate any and all feedback. If anyone has any fun ideas they would like to see visualized message below and I would love to tackle it. My aim is to try to complete one or two lessons a week with a blog post to accompany each. I’m looking forward to see where the course goes from here as I had a lot of fun with just the first two lessons. If this is interesting to anyone reading this, I 100% recommend also doing the course yourself. It doesn’t take much at all and the teacher Jeremy Howard is amazing and I definitely see why many top companies recommend their employees to go through the course and the best part it’s all 100% free.

You can follow me Substack to get notified when I post my next update