It's been quite some time since I last wrote a post, as I've been caught up
with numerous activities related to projects I needed to complete. Honestly, I
miss writing on this blog to share my experiences, as it not only helps me
remember what I've learned but also proves useful to others.
Geographical Segment Anything Model (SAMGeo)
Let's begin this post with SAMGeo. The Geographical Segment Anything Model (SAMGeo) was created by Rafi Ibn Sultan and a team of researchers in late 2023. This innovative adaptation of the Segment Anything Model (SAM) is specifically tailored for geographical imagery, particularly focusing on mobility infrastructure like roads, sidewalks, and crosswalks in satellite and aerial images. SAMGeo addresses the challenges of segmenting such narrow and texture-blending elements by fine-tuning SAM with multi-modal prompts. This includes using both visual cues from pre-trained models for precise location data and text prompts for semantic guidance, enhancing the model's ability to accurately segment complex geographical scenes. With its specialized approach, SAMGeo offers significant improvements over traditional methods, providing a more accurate and efficient way to analyze and segment geographical data for various applications.
Installing SAMGeo
As another module in Python, SAMGeo can be easily installed using either Conda or pip via a shell or command line. However, segmenting an image is a computationally intensive task, requiring significant hardware resources such as ample RAM and a powerful GPU. Recognizing that not everyone has access to such expensive resources, this tutorial will utilize Google Colab, which offers a limited free GPU for use.
Open
Google Colab. To enable GPU select Runtime and then choose
Change runtime type as in figure 1. In the Change runtime window select
T4 GPU and don't forget to click Save button, as shown in the
figure 2.
|
| Figure 1. Change Google Colab Runtime |
|
| Figure 2. Select T4 GPU |
Move to the coding cell. Type !pip install segment-geospatial. Running the cell it will install the SAMGeo module and all it's dependency. Sit back and relax cause it will take some time to finish the installation.
Importing SAMGeo and Other Modules
After installing the module, it's time to import the necessary modules, namely leafmap and, of course, samgeo, as shown in the following code:
import leafmap from samgeo import SamGeo2
If no error appears, it means the installation was successful, and we are ready to proceed to the next step.
The following step we will take a sample image. For this the leafmap module is used to get the image. The code below is the code to view the satellite imagery somewhere in Santa Maria, California, United States. Running the code we'll get the result as shown in the figure 3.
m = leafmap.Map(center=[34.9401058,-120.4252747], zoom=18) m.add_basemap("SATELLITE") m
 |
| Figure 3. Satellite imagery |
The satellite view shown in the previous figure consists of tiles from a web map server. Direct segmentation on these tile images isn't possible; instead, we must download a portion of the image. To do this, we need to define a bounding box (bbox) which includes the minimum and maximum coordinates (longitude, latitude). The map_tiles_to_geotiff method from the leafmap module is used for this purpose. In this method, we specify the output file name, the bbox, the zoom scale, and the source, as seen in line 3 of the following code.
The rest code from line 5 is used to add the downloaded image to the leaflet layer with a name 'Image' and view it as shown in the figure 4.
1 2 3 4 5 6 7 | bbox = [-120.4248951,34.9400119,-120.4226193,34.9412893] image = "santa_maria.tif" leafmap.map_tiles_to_geotiff(output=image, bbox=bbox, zoom=17, source="Satellite", overwrite=True) m.layers[-1].visible = False m.add_raster(image, layer_name="Image") m |
 |
| Figure 4. The downloaded image view over the map |
Image Segmentation with SAMGeo
Now we are ready to segmentation. To do the segmentation process we only need to select segmentation model id. In this case I used sam2-hiera-large. There are some models we can use namely: sam2-hiera-tiny, sam2-hiera-small, sam2-hiera-base-plus and sam2-hiera-large.
The difference between each model as following:
- sam2-hiera-tiny: Smallest model, least resource-intensive. Basic segmentation performance, suitable for resource-limited devices or when speed is prioritized.
- sam2-hiera-small: Slightly larger than tiny, offering a balance between resource use and performance. Better accuracy than tiny but still efficient for resource management.
- sam2-hiera-base-plus: Medium-sized model, providing good accuracy for complex scenes. Versatile for applications needing detailed segmentation without high resource costs.
- sam2-hiera-large: Largest model with the highest accuracy and detail. Best for high-precision tasks, suitable where computational resources are abundant.
1 2 3 | sam = SamGeo2(model_id="sam2-hiera-large", automatic=True) sam.generate(image) sam.show_anns(axis="off", figsize=(8,7), alpha=0.7, output="annotations.tif") |
Running the code above we will get an output as shown in figure 5. Can be seen from the figure there are some buildings were segmented.
 |
| Figure 5. Segmentation Result |
It was not great, I want to segment more buildings and objects. For that we can customize some parameters in the segmentation model to optimize the result.
Segmentation Model Customization
In the segmentation model there are some parameters that can be tuned to optimize the segmentation result. In the following code can be seen the parameters starting from model_id, apply_postprocessing, points_per_side, etc.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | sam = SamGeo2( model_id="sam2-hiera-large", apply_postprocessing=False, points_per_side=32, points_per_batch=64, pred_iou_thresh=0.7, stability_score_thresh=0.92, stability_score_offset=0.7, crop_n_layers=1, box_nms_thresh=0.7, crop_n_points_downscale_factor=2, min_mask_region_area=150.0, use_m2m=True, ) sam.generate(image, output="masks2.tif") sam.show_anns(axis="off", alpha=0.7, output="annotations2.tif") |
Explanation of Parameters:
- apply_postprocessing=False: Setting this to False means no post-processing will be applied to the model's output. Post-processing might include operations like smoothing, refining, or filtering the segmentation mask.
- points_per_side=32: This parameter relates to how the image or area is sampled or divided. For grid-based processing, it specifies 32 points along each side of a grid.
- points_per_batch=64: This controls how many points are processed simultaneously in each batch, impacting performance and memory usage when handling large images or areas.
- pred_iou_thresh=0.7: The Predicted Intersection over Union (IoU) Threshold determines if a predicted segment is valid. An IoU of 0.7 means the prediction needs at least 70% overlap with the true segment to be accepted.
- stability_score_thresh=0.92: This threshold ensures the stability or reliability of segmentation. Segments with a stability score below 0.92 might be discarded or re-evaluated.
- stability_score_offset=0.7: This parameter adjusts the stability score calculation, possibly by adding or subtracting this value for normalization or adjustment.
- crop_n_layers=1: Indicates how many layers of cropping or scaling operations are applied, useful for hierarchical or multiscale processing.
- box_nms_thresh=0.7: The Box Non-Maximum Suppression (NMS) Threshold is used in object detection to suppress overlapping bounding boxes. A threshold of 0.7 means boxes with an IoU above this value are suppressed.
- crop_n_points_downscale_factor=2: This is used to reduce the number of points in cropping operations, enhancing efficiency or accommodating memory constraints.
- min_mask_region_area=150.0: Sets a minimum size for segmented regions. Segments smaller than 150 pixels will be ignored or merged.
- use_m2m=True: Stands for "use model-to-model," suggesting that some form of model interaction or transfer learning is enabled.
The outcome of running the code, as depicted in Figure 6, demonstrates a significant improvement in building segmentation. Compared to earlier results, a greater number of buildings have been successfully identified and delineated.
 |
| Figure 6. Optimized Segmentation Result |
That concludes our tutorial on how to execute automatic segmentation of satellite imagery using Python in Google Colab. This guide has only scratched the surface of the capabilities of SAMGeo. There's a vast array of features and techniques yet to be explored. I highly recommend visiting the official SAMGeo website to delve deeper into advanced functionalities, find additional resources, and discover how you can leverage this tool for more complex geospatial analysis. Whether you're interested in enhancing your skills in remote sensing, urban planning, or environmental monitoring, the potential applications are numerous and exciting.
