New functionality of the QGIS GeoAI plugin. Impact on urban area imagery

Read in Spanish

As we commented in the article How to obtain better results when capturing elements on geospatial images, there are several ways to influence the result of the segmentation of an image, gradually we have been implementing them in the development of the GeoAI Plugin.

1.- Model adaptation: other versions of the SAM model have been incorporated, such as the light SAM model. In addition, process configuration capabilities have been added. The selection options have been extended, in addition to points and polygons, with the combination of both, exclusion and inclusion zones.

2.- Adjustment of the input image: this aspect is the focus of the new functionality, allowing the user to select a region of interest of the image to be segmented. In addition, an optimization of the dynamic range of the data of that region is applied.

Process region of interest

The new functionality incorporated in the GeoAI plugin will allow the user to select a region of interest on the image to be processed. This option will be available for both Interactive Segmentation and Whole Image Segmentation.

As can be seen in the following image, the Corte a región button has been added in both interfaces, which allows the user to draw a rectangle on the image to define the region of interest.

What impact will this tool have on image segmentation? will it allow to obtain better segmentations or speed up the process? will it affect the SAM and light SAM model in the same way? will there be differences between high, medium and low resolution images?

Questions such as these arise with this new tool, but at the same time they are valid topics for one or more research projects. One of the benefits of GeoAI for the user community is to allow them to explore these types of studies without programming.

As shown in the cover image of this article (see it again) join me to explore this new capability on high resolution images of urban areas using Whole Image Segmentation.

Special thanks to our friend Mario Alberto Luna Pavo for providing us with one of the images obtained from his drone survey projects.

Evaluation of the selection of region of interest in the result of segmentation in urban areas.

We will perform the comparison of the GeoAI function for the Segmentation of the whole image using two scenes, the first one a small image of a residential area in Venezuela and the second one of a residential area in Mexico, making the following comparisons:

• SAM model and SAM light
• With and without region of interest selection.

Ortho segmentation image 1

Here is the result of applying the SAM model and SAM light on the entire image. We can notice how SAM generates more segments than the light version, highlighting the portion of the track in the center of the image, however, in this execution light SAM identifies certain segments that SAM does not detect.

Now let's try selecting a region in the upper left side of the image.

In both cases the processing of the region of interest improves segmentation. However, the impact differs in both models.

The improvement of the result of the original SAM model is notorious, in this case the selection of the region of interest produces a remarkable increase of the number of segments, identifying all the available elements, including the road.

Ortho segmentation image 2

Now we will try with an image of greater magnitude, both in its extension and in its spatial resolution.

As expected, again the SAM model generates a higher number of segments. But in both results very few vehicles are identified, even in some streets no cars are detected.

Let's try generating a region on the second horizontal street and evaluate the result.

In this case, the selection of the region allowed identifying vehicles in a similar way in both models (SAM vs. SAM Light). However, in the case of SAM, the segmentation of the buildings surrounding the road was improved.

Conclusions

From the brief study carried out, we can affirm:

• The SAM model presents a better performance than the light SAM model (expected result). However, this is compensated by a lower processing load of the Light SAM model (few minutes).
• The processing of the region of interest influenced the segmentation result in the images used, improving the process by increasing the number of segments obtained.
• The effect of the region of interest was greater on the SAM model than on the Light SAM model in the two iterations performed.
• Based on the results obtained, the selection of the region of interest is presented as an alternative to improve the image processing tasks in conjunction with the other functions available in the GeoAI plugin.

Final Notes

The new features add to the capabilities of the GeoAI plugin, providing more alternatives to the user.

However, although this preliminary study is very encouraging, many questions remain, for example:

• How does the region selection influence the interactive segmentation, will we get similar results?
• Can the result of region selection be improved by altering the segmentation settings of the whole image?
• How will this functionality impact images of agricultural areas?
• Others............

On the other hand, the GeoAI Plugin can still increase its functionality, some options are:

• Integrate more models such as HQ SAM, SAM2 among others.
• Explore and include other options to improve the input image.
• Include a classifier that allows to categorize segments or even identify them by text promtp using pre-trained models.

If you want to support the development of the plugin and articles like this you can contact me and we will coordinate.

You can also support by providing images that I will use for development and research as presented in this article.