Fast Light SAM now available in QGIS GeoAI plugin
Luis Eduardo Perez Graterol
Analista y desarrollador de SIG comerciales y Open Source
As shown in the cover image, one of the functions of the GeoAI plugin is to facilitate the digitization of elements on geospatial images using the IA SAM model (META), however, so far, this process has required large computing power.
In this new version of the plugin I have incorporated a light version of the SAM model developed by ETH VIS Group, which allows to execute the segmentation consuming less resources and in a very fast way.
With the incorporation of this version of the SAM model, QGIS emerges as the simplest and most intuitive Open Source alternative for segmenting spatial images with AI.
This version of SAM was developed by ETH VIS Group with the intention of incorporating it in mobile devices. Source: Github ETH VIS Group, Paper: Ke, Lei and Ye, et. al, 2023 Segment Anything in High Quality, NeurIPS.
How does the lightweight SAM model differ from the SAM model?
The main difference is that the file with the model parameters (checkpoint) weighs only 48 mb compared to the more than 2 Gb of the original files.
This makes it possible to run the segmentation very quickly and on standard equipment without GPU usage.
Advantages of the lightweight SAM model at GeoAI
Limitations of the lightweight SAM model in GeoAI
Requirements
As previously mentioned, in order to use the light and fast version of SAM in GeoAI we must install the Python timm package.
What is the timm package?
"timm" is a deep-learning library created by Ross Wightman and is a collection of SOTA computer vision models, layers, utilities, optimizers, schedulers, data-loaders, augmentations and also training/validating scripts with ability to reproduce ImageNet training results. Source: timm documentation.
Installation of the timm package:
We can quickly install using pip in the OSGEO shell. The detailed explanation to open the shell and install with pip can be found in the GeoAI repository. The steps are briefly described:
1.- In the MS DOS window enter the following code to configure the working environment. If the code generates an error, do not worry, continue with the next step.
py3_env
2.- It is always advisable to update pip. Enter and execute the following code:
python -m pip install -U pip
3.- We proceed to install timm. Insert and execute the following code:
领英推荐
pip install timm
4.- When the installation is finished, close the MS DOS window, run QGIS and proceed to activate and use the GeoAI plugin.
GeoAI Installation
If you are installing the GeoAI Plugin for the first time, you must start by installing Torch, the procedure is explained in detail in the Github documentation: GeoAI documentation.
Then you must install timm as explained in the previous item.
Once the necessary libraries are installed, proceed to download the latest version of GeoAI, you have two options:
After downloading the zip file you install it in QGIS and activate it, details of the procedure in GeoAI documentation.
How to use Fast Light SAM in GeoAI?
The GeoAI user interface has changed very little, the three modules are maintained, only one option has been added in the Pre-load model/image module.
By clicking on the Pre-load model/image module, you will notice that the option (selected by default) Utilizar modelo HQ ligero (integrado, solo para segmentación interactiva) has been added.
Using the Fast Light SAM model is very simple, just select the image and then click Run. When the process is finished run the Interactive Segmentation module and start capturing.
The Fast Light SAM model is the default option of the plugin and no additional configuration is required. It will be used as long as the Utilizar modelo HQ ligero (integrado, solo para segmentación interactiva) is enabled.
How to deactivate the Fast Light SAM model and use the original SAM model?
To use the SAM model with its respective checkpoints, uncheck the Utilizar modelo HQ ligero...... option.
Final notes
The GeoAI plugin is just emerging, there are many possibilities for growth and optimization, parallel to every moment new versions of SAM and other IA models useful for the Geospatial field.
Soon I will be publishing an article about alternative improvements that can be incorporated to the plugin.
Remember that you can help support this and other developments. We can also explore different ways of integration and collaboration, feel free to write me.
?? 23K+ Followers | ?? Linkedin Top Voice | ?? AI Visionary & ?? Digital Marketing Expert | DM & AI Trainer ?? | ?? Founder of PakGPT | Co-Founder of Bint e Ahan ?? | ?? Turning Ideas into Impact | ??DM for Collab??
9 个月Great news! Excited to try out the new version of the #QGIS #GeoAI #Plugin. ??