SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images

Adapting SAM 3 for remote sensing OVSS

Kaiyu Li¹  Shengqi Zhang¹  Yupeng Deng²  Zhi Wang¹  Deyu Meng¹  Xiangyong Cao^✉1 

¹Xi'an Jiaotong University  ²Chinese Academy of Sciences 

• [Code] • [arXiv] • [Demo] •

Inference results of SegEarth-OV3 on a remote sensing image exceeding 10k×10k resolution. The image originates from OpenMapCD.

The overall inference pipeline of SegEarth-OV3. Given an input image and a list of text prompts, we leverage SAM 3's decoupled outputs. The pipeline involves: (1) instance aggregation to consolidate sparse object predictions; (2) dual-head mask fusion to combine the fine-grained instance details with the global coverage of the semantic head; and (3) presence-guided filtering (using the presence score) to suppress false positives from absent categories. "MAX" denotes the element-wise maximum operation, and "×" denotes multiplication.

Abstract

Most existing methods for training-free Open-Vocabulary Semantic Segmentation (OVSS) are based on CLIP. While these approaches have made progress, they often face challenges in precise localization or require complex pipelines to combine separate modules, especially in remote sensing scenarios where numerous dense and small targets are present. Recently, Segment Anything Model 3 (SAM 3) was proposed, unifying segmentation and recognition in a promptable framework. In this paper, we present a preliminary exploration of applying SAM 3 to the remote sensing OVSS task without any training. First, we implement a mask fusion strategy that combines the outputs from SAM 3's semantic segmentation head and the Transformer decoder (instance head). This allows us to leverage the strengths of both heads for better land coverage. Second, we utilize the presence score from the presence head to filter out categories that do not exist in the scene, reducing false positives caused by the vast vocabulary sizes and patch-level processing in geospatial scenes. We evaluate our method on extensive remote sensing datasets. Experiments show that this simple adaptation achieves promising performance, demonstrating the potential of SAM 3 for remote sensing OVSS.

Dependencies and Installation

You only need to focus on installing mmcv and mmsegmentation correctly; other dependencies are not strict.

Datasets

We include the following dataset configurations in this repo:

1. Semantic Segmentation: OpenEarthMap, LoveDA, iSAID, Potsdam, Vaihingen, UAVid^img, UDD5, VDD
2. Building Extraction: WHU^Aerial, WHU^Sat.Ⅱ, Inria, xBD^pre
3. Road Extraction: CHN6-CUG, DeepGlobe, Massachusetts, SpaceNet
4. Water Extraction: WBS-SI

Please refer to dataset_prepare.md for dataset preparation.

Download checkpoints of SAM 3

Download checkpoints from HF or ModelScope.

Quick Inference

python demo.py

Model evaluation

python eval.py ./configs/cfg_DATASET.py

Results

Citation

@article{li2025segearthov3,
  title={SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images},
  author={Li, Kaiyu and Zhang, Shengqi and Deng, Yupeng and Wang, Zhi and Meng, Deyu and Cao, Xiangyong},
  journal={arXiv preprint arXiv:2512.08730},
  year={2025}
}

Acknowledgement

This implementation is based on SAM 3 and SCLIP. Thanks for the awesome work.