Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set

By Shunping Ji, Shiqing Wei, Meng Lu. IEEE Transactions on Geoscience and Remote Sensing, 2019

1,850 citations recorded by the source corpus.

The application of the convolutional neural network has shown to greatly improve the accuracy of building extraction from remote sensing imagery. In this paper, we created and made open a high-quality multisource data set for building detection, evaluated the accuracy obtained in most recent studies on the data set, demonstrated the use of our data set, and proposed a Siamese fully convolutional network model that obtained better segmentation accuracy. The building data set that we created contains not only aerial images but also satellite images covering 1000 km2 with both raster labels and vector maps. The accuracy of applying the same methodology to our aerial data set outperformed several other open building data sets. On the aerial data set, we gave a thorough evaluation and comparison of most recent deep learning-based methods, and proposed a Siamese U-Net with shared weights in two branches, and original images and their down-sampled counterparts as inputs, which significantly improves the segmentation accuracy, especially for large buildings. For multisource building extraction, the generalization ability is further evaluated and extended by applying a radiometric augmentation strategy to transfer pretrained models on the aerial data set to the satellite data set. The designed experiments indicate our data set is accurate and can serve multiple purposes including building instance segmentation and change detection; our result shows the Siamese U-Net outperforms current building extraction methods and could provide valuable reference.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Ocean Engineering

View Paper: https://www.semanticscholar.org/paper/53edb3776cea0e6ee4de742d7bb906355fd8279b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Ocean Engineering

Citations in the corpus

Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges (2020, methodology): “Although some methods are based on the WHU building dataset [88], unfortunately, their experimental data are different with regard to area, which makes it difficult to compare them directly.”
Building Change Detection for Remote Sensing Images Using a Dual-Task Constrained Deep Siamese Convolutional Network Model (2019, background): “public WHU building data set [11] provides the semantic labels of the bitemporal images and the corresponding change labels.”
Change Detection on Remote Sensing Images Using Dual-Branch Multilevel Intertemporal Network (2023, background): “With the sophistication and innovation of sensor devices, extensive multitemporal high-resolution RS images are available for land management [1], disaster assessment, and urban/rural development investigation [2], [3].”
Remote Sensing Image Change Detection With Transformers (2021, background): “provided in [54], we crop the images into small patches of size 256×256 with no overlap and randomly split it into three parts: 6096/762/762 for training/validation/test, respectively.”
Fast Segment Anything (2023, methodology): “We evaluate FastSAM on the dataset proposed by [14].”
An Empirical Study of Remote Sensing Pretraining (2022, methodology): “Then, [132] clipped the images to a series of 256 × 256 patches and generated a dataset, where the sizes of the training, validation and testing set are 10,000/3,000/3,000, respectively.”
SwinSUNet: Pure Transformer Network for Remote Sensing Image Change Detection (2022, methodology): “The WHU-CD [45] dataset includes two aerial images with 0.075m spatial resolution, both of which were taken in Christchurch, but at different times, one in 2012 and the other in 2016.”
ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Model (2024, methodology): “2) WHU-CD [55]: The WHU-CD dataset, a segment of the larger WHU Building dataset, is tailored for the task of building change detection.”
RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model (2023, methodology): “We utilize three publicly available remote sensing instance segmentation datasets to validate the effectiveness of the proposed method, including the WHU building extraction dataset [84], the NWPU VHR-10 dataset [2], [85], and the SAR Ship Detection Dataset (SSDD) [1], [86].”
Optical Remote Sensing Image Change Detection Based on Attention Mechanism and Image Difference (2021, methodology): “In general, change detection plays an important role in urban planning [1], forest monitoring [2], disaster assessment [3]–[5], and the dynamic detection of land use [6].”
SemiCDNet: A Semisupervised Convolutional Neural Network for Change Detection in High Resolution Remote-Sensing Images (2020, methodology): “To verify the effectiveness of the proposed method, two VHR RS image data sets are employed: WHU building data set [23] and Google data set, both of which include a large amount of VHR image pairs for CD task.”
Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery (2020, methodology): “Semantic Segmentation in Remote Sensing Community There are a lot of applications using semantic segmentation technique in the remote sensing community, such as land use and land cover (LULC) classification [46, 23, 47], building extraction [45, 24, 43, 15], road extraction [28, 9, 2, 34, 3], vehicl”
Transformers in Remote Sensing: A Survey (2022, methodology): “STEB-UNet [73] Building Extraction WHU [60], Massachusetts [102] IoU, F1 score Introduces a transformers framework capturing semantic information from multi-scale features which are further fused to local features.”
Stacked Autoencoders Driven by Semi-Supervised Learning for Building Extraction from near Infrared Remote Sensing Imagery (2021, methodology): “Other adopts a Fully Convolutional Network (FCN) to complete the classification [26,27].”
Towards Geospatial Foundation Models via Continual Pretraining (2023, background): “The WHU Aerial building [22] dataset is sampled over Christchurch, New Zealand at a GSD of 0.”
DASNet: Dual Attentive Fully Convolutional Siamese Networks for Change Detection in High-Resolution Satellite Images (2020, background): “9, the proposed method has realized satisfactory results on both the CDD and BCDD datasets.”
Object Detection and Image Segmentation with Deep Learning on Earth Observation Data: A Review-Part I: Evolution and Recent Trends (2020, background): “[126] IS multiple classes plane multispectral very high U-Net, DeepLabV3+, PSPNet, LANet (patch attention module) [167], MobileNetV2(with atrous conv) + Dual path encoder + SE modules [168] DeepGlobe-Road [169] IS road network WorldView3 multispectral high D-LinkNet (ResNet-34 + U-Net with atrous de”