Fast Light SAM now available in QGIS GeoAI plugin

Fast Light SAM now available in QGIS GeoAI plugin

Read in Spanish

As shown in the cover image, one of the functions of the GeoAI plugin is to facilitate the digitization of elements on geospatial images using the IA SAM model (META), however, so far, this process has required large computing power.

In this new version of the plugin I have incorporated a light version of the SAM model developed by ETH VIS Group, which allows to execute the segmentation consuming less resources and in a very fast way.

With the incorporation of this version of the SAM model, QGIS emerges as the simplest and most intuitive Open Source alternative for segmenting spatial images with AI.

This version of SAM was developed by ETH VIS Group with the intention of incorporating it in mobile devices. Source: Github ETH VIS Group, Paper: Ke, Lei and Ye, et. al, 2023 Segment Anything in High Quality, NeurIPS.

How does the lightweight SAM model differ from the SAM model?

The main difference is that the file with the model parameters (checkpoint) weighs only 48 mb compared to the more than 2 Gb of the original files.

This makes it possible to run the segmentation very quickly and on standard equipment without GPU usage.

Advantages of the lightweight SAM model at GeoAI

  1. The preloading process is very fast, for example: 2 minutes on a computer that previously took 20 minutes.
  2. It can be run using only CPU due to its lower requirements.
  3. Given the above conditions it is possible to use it with larger images.
  4. Its execution does not require additional files since the checkpoint is built into the plugin.

Limitations of the lightweight SAM model in GeoAI

  1. It can only be used with the Interactive Segmentation module, i.e. selecting points or areas. It does not work with the whole image segmentation module.
  2. Generates only one segment per point or area. It does not generate multiple segments. Although selecting this option has given me better results.
  3. In medium to low resolution images, or when trying to segment diffuse elements in the image, its performance is lower than the original SAM version.
  4. It requires installing the Python timm package, which is a Pytorch add-on, however, its installation is simple and fast.

Requirements

As previously mentioned, in order to use the light and fast version of SAM in GeoAI we must install the Python timm package.

What is the timm package?

"timm" is a deep-learning library created by Ross Wightman and is a collection of SOTA computer vision models, layers, utilities, optimizers, schedulers, data-loaders, augmentations and also training/validating scripts with ability to reproduce ImageNet training results. Source: timm documentation.

Installation of the timm package:

We can quickly install using pip in the OSGEO shell. The detailed explanation to open the shell and install with pip can be found in the GeoAI repository. The steps are briefly described:

1.- In the MS DOS window enter the following code to configure the working environment. If the code generates an error, do not worry, continue with the next step.

py3_env        

2.- It is always advisable to update pip. Enter and execute the following code:

python -m pip install -U pip        

3.- We proceed to install timm. Insert and execute the following code:

pip install timm        

4.- When the installation is finished, close the MS DOS window, run QGIS and proceed to activate and use the GeoAI plugin.

GeoAI Installation

If you are installing the GeoAI Plugin for the first time, you must start by installing Torch, the procedure is explained in detail in the Github documentation: GeoAI documentation.

Then you must install timm as explained in the previous item.

Once the necessary libraries are installed, proceed to download the latest version of GeoAI, you have two options:

  1. In the repository, in the main branch, you click on <> Code (green color button) and then on Download zip (highlighted in green in the following image).
  2. In the repository on the right side click on "Releases" and select on v1.1.0_GeoAI (highlighted in blue in the image below).

After downloading the zip file you install it in QGIS and activate it, details of the procedure in GeoAI documentation.

How to use Fast Light SAM in GeoAI?

The GeoAI user interface has changed very little, the three modules are maintained, only one option has been added in the Pre-load model/image module.

By clicking on the Pre-load model/image module, you will notice that the option (selected by default) Utilizar modelo HQ ligero (integrado, solo para segmentación interactiva) has been added.

Using the Fast Light SAM model is very simple, just select the image and then click Run. When the process is finished run the Interactive Segmentation module and start capturing.

The Fast Light SAM model is the default option of the plugin and no additional configuration is required. It will be used as long as the Utilizar modelo HQ ligero (integrado, solo para segmentación interactiva) is enabled.

How to deactivate the Fast Light SAM model and use the original SAM model?

To use the SAM model with its respective checkpoints, uncheck the Utilizar modelo HQ ligero...... option.

Final notes

The GeoAI plugin is just emerging, there are many possibilities for growth and optimization, parallel to every moment new versions of SAM and other IA models useful for the Geospatial field.

Soon I will be publishing an article about alternative improvements that can be incorporated to the plugin.

Remember that you can help support this and other developments. We can also explore different ways of integration and collaboration, feel free to write me.

Faraz Hussain Buriro

?? 23K+ Followers | ?? Linkedin Top Voice | ?? AI Visionary & ?? Digital Marketing Expert | DM & AI Trainer ?? | ?? Founder of PakGPT | Co-Founder of Bint e Ahan ?? | ?? Turning Ideas into Impact | ??DM for Collab??

9 个月

Great news! Excited to try out the new version of the #QGIS #GeoAI #Plugin. ??

要查看或添加评论,请登录

社区洞察

其他会员也浏览了