Get Started With Your Own Software Sensor

Do you remember the times when we had to use physical, wired hardware sensors to detect if a car park lot was occupied or free? These days may be over now and this might be the beginning of a big paradigm shift. Camera feeds can now be augmented with computer vision skills easily and cheaply in order to detect almost anything that's visually detectable. Such a system is what we call a "Software Sensor" (as opposed to a "Hardware Sensor"). And now you can create your own little software sensor completely yourself - for free. Here's how.

Detecting Dirty Dishes

A very common problem most of us are familiar with is unwashed dishes in the sink. Let's try to tackle that by detecting if there are dishes in the sink, and if there are, send a notification to the household chat members to do the dishes. Sounds easy, doesn't it?

Thanks to the Gravio platform it's not only easy, it's also completely free to build such a system. All you need is a Google Cloud Platform account, and use the free credits you get for singning-up to train your model. Then you can deploy that model to the free version of Gravio that is connected to a camera that points to the sink. In fact, we can just have this camera store pictures in regular intervals to a folder that Gravio monitors, and every time a picture is stored, it will apply the computer vision algorithm to detect dishes.

Broadly spoken steps are very straightforward:

Step 1: Install Camera

Step 2: Collect Images to Train the Camera

Step 3: Upload the Pictures to your Goolge AutoML Account

Step 4: Tag your Pictures

Attention, this is the hardest and most time-consuming part of the process. You have to go through every single picture and tag the objects / classify the image type. But the good news is, there are plenty of people on upwork.com who can do this for you very cost effectively. Just search for "data labellers". In this example we use the Object Detection method. Although you could achieve the same with Image Classification.

Step 5: Train, Test and Export the Model

The Google AutoML platform will give you a rough indication if you have enough labels to start the training. When the counts of the labels are sufficient, you can start the training.

The Google AutoML platform also allows you to test the models. Just deploy it to the server, upload a few sample images and see what the platform detects.

Step 6: Deploy the Models to the Gravio Edge IoT Platform

After you have downloaded the files from your Google Cloud storage, you can use Gravio Studio to deploy and test the model:

Once you have uploaded your new model, you can use the model like a sensor and attach it to triggers and actions.

You can also use pre-trained models from websites such as tfhub.dev, although there is no guarantee that the models work. Just filter by Image, TFLite, TF1. We have hwever tested Gravio successfully using the models for the food, birds and insects.

With the free version, you can now deploy one model to your edge computer. The model will behave like a sensor and can therefore trigger all actions you may wish to program within Gravio. For more information on how to create Gravio Actions, please refer to the Gravio documentation on https://doc.gravio.com or our Gravio Edge IoT Platform YouTube channel.