Processing multiple image formats with GEOSAM

Leer en espa?ol

This article shows the progress in the development of the GEOSAM plug-in by testing it on multiple types of imagery and contexts (areas and objects of study) common in the geospatial field.

We will use low, medium and high resolution images. From one to eight bands. Even a scanned and georeferenced map. These are described below:

• Drone image of crop area
• 8-band Sentinel 2 image composite.
• Processed SPOT image.
• Ikonos image.
• Scanned map.
• Orthoimage for urban cadastre purposes.

Image sources: the data used come from web sources, e.g. example images from the Micasense Red Edge, historical panchromatic Ikonos image, old SPOT image, Sentinel2 image used in the tutorial of the Semi-automatic-classification plugin developed by Luca Congedo.

INTRODUCTION

GEOSAM is an plug-in under development for QGIS that aims to exploit with versatility the capabilities of the Segment Anything (SAM) segmentation model developed by META on geospatial data.

Segment Anything is a pre-trained artificial intelligence model that allows to generate masks on recognised objects in images for highlighting or extraction (Source SAM project page, SAM project github repository).

Segment Anything produces high quality object masks from inputs such as points or boxes, and can be used to generate masks for all objects in an image. It has been trained on a dataset of 11 million images and 1.1 billion masks, and offers great performance on a wide variety of segmentation tasks (Source SAM project page, SAM project github repository).

Challenges in processing geospatial data

The SAM model is designed to process RGB images (only three bands), with a range of values from 0 to 255 (8 bits).?

However, the imagery used in geospatial projects is much more diverse, presenting different formats, scales, value ranges, origins, number of bands and coordinate systems.

Application of GEOSAM

In the following video the interactive segmentation module of the? GEOSAM plug-in is used to extract elements from the various images.

What's next?

There is still a lot to develop and test, among which we can highlight:

• Create a model loading module that allows segmentation on base maps (XY Tiles) such as Open Street Map, ESRI Satellite, Google Maps, etc.
• Modify the loading module to allow the selection of the image bands to be used in the segmentation.
• Incorporate advanced configuration options to change the parameters of the SAM model.
• Build the segmentation module for the whole image.
• Implement classification methods on the segments to automate map generation.

Leave your comments about this project and any interest or support you can write to me privately.