Laptop Vision ML versus Cloud Vision ML

<abstract> ML system quality depends a lot of the quality of the training dataset. Here is a simple demonstration, Cloud's providers Vision APIs for objects detection tends to return better results than pre-trained object detection models. </abstract>

Using pre-trained object detection models could be a perfect way to upskill or to prototype solutions, but when the ambition is to move in production at scale, it looks like using the Cloud's provided pre-trained models accessible through APIs are definitely a more complete solution. Before using pre-trained model you want to ensure the expected predictions were embraced by the space covered by the dataset the model was trained on.

This means, if you want to detect dogs with a pre-trained detection model, ensure it have been trained with lots of dogs samples.

I was testing a course around Object Detection with tensorflow that used a pre-trained model on a 80 object categories dataset called COCO (for comon object in context).

While it was interesting to load the pre-trained model, host it in a virtual machine hosting a Flaks app, the model performances on my chosen samples were barely satisfying so I thought about comparing it with the Google Cloud Vision API using the Testing page.

You just have to upload an image file and you will get outputs of detected objects among others images features such as sensitives categories if detected, dominant colors, sentiment predicted on faces ect.

Here are how the results compare

Using the pre-trained model on COCO dataset:

it detects the bed

While on Google Cloud Vision API, the bed is also detected, the person is detected , flower, clothing.

Using other samples

the pre-trained model on COCO dataset

3 persons are detected, which is great

And we could see here again the Cloud Vision API is pushing it a little bit further

As we can see the persons are also detected, but also piece of fashion themed objects, such as the necklace.

(Fore the reggaeton lovers, yes this is Bad Bunny, the same singing Dákiti : listen to Dakiti on Youtube )

Continuing using cartooned images

with the pre-trained model on COCO dataset

It detects a persons, crop the head of the character... Poor Petit Ours Brun we understand why he looks angry :(

While the Google Cloud Vision API is providing more details

Animal, Footwear, Shoe, ok, if you're like me you would like to get the information that the image looks like a drawing

Building a model detecting drawing vs photography is something that could be easily done as my university pal Thulfakar Hammodi could testify !

In the " Labels " field of the GCP Vision API "Cartoon" and "Illustration" are specified

So what does that means?

Pre-trained model over 80 categories, using cropped pictures on the object detected, will be useful to detect objects of those categories, in pictures cropped on the same way.

The Machine Learning Object Detection model job is to recognise objects that looks like the objects from the Training datasets, if we take a look at the COCO dataset we could understand why my samples have not been perfectly covered

The Cloud's providers Vision Object Detection models are trained on massively more data and categories, this is why they're providing more detailed answers.

As a conclusion, this is a demonstration that a Machine Learning system is as powerful as its input trained dataset have been well curated. The best quality data you will use to train your model the best quality Machine Learning system you will get.

Have a nice Day!

Tensorflow Object detection API github

Images sources:

• Flower as a skirt : https://www.mediafactory.org.au/gloria-tanuseputra/2016/03/21/photography/
• Bad Bunny : https://www.vanityfair.com/style/2019/08/week-in-fashion-bad-bunny-vmas
• Petit Ours Brun est grognon: de Marie Aubiniais (Auteur), Danièle Bour (Illustrations) , Bayard Jeunesse

Images used as samples, all rights reserved, there is no link between images and the ML models used or any companies mentioned in this post.