Lecture on October 30, 2020.
Special Lecture: Classification of land cover and land use based on Deep Learning
Summary. The classification of images and other remote sensing data is a fundamental task to derive semantic information about the objects in the depicted scene automatically. For several years, research on image classification in remote sensing has been dominated by deep learning (Zhu et al., 2017), mainly in the form of variants of convolutional neural networks (CNN) (Krizhevsky et al., 2012). This presentation will discuss two different applications of CNN in the context of remote sensing, namely the pixel-based classification of land cover from remote sensing imagery and derived data and the prediction of land use for the objects of an existing geospatial database. The third part of the presentation will be dedicated to deep domain adaptation (DA) (Wang & Deng, 2018), a strategy for mitigating the requirements of deep learning with respect to the availability of training data.
The prediction of land cover essentially results in a pixel-wise classification of the input data. Land cover is related to the material of the surface depicted in a pixel. It can be the first step towards the generation of a map from scratch. Besides having to provide the required training data, the main challenges are related to the implementation of an appropriate network structure, in particular if different types of input data (e.g. image and derived height data) are to be combined, and the design of task-specific loss functions and training strategies. This part of the presentation will mainly focus on the classification of high-resolution aerial imagery and height data (Yang et al., 2018; 2020a; Wittich & Rottensteiner, 2019; Wittich, 2020), but it will also discuss an application involving Sentinel-2 imagery (Voelsen et al., 2020). The latter is particularly interesting because it uses an existing (though potentially outdated) map to generate the training data without manual intervention.
The prediction of land use as discussed in this presentation is related to the task of the verification of an existing geospatial database as a part of the update process of that database. In this context, a land use label is predicted for every object existing in the database on the basis of current remote sensing data and the prediction is compared to the information contained in the database in order to see whether the latter is correct. A major problem is the varying size of land use objects, which contradicts the requirements of a CNN to have images of identical size as input (Yang et al., 2018). Another problem is the very detailed nature of the object catalogue, which contains object types of very similar appearance that cannot be expected to be differentiate on the basis of image data. Investigations in (Yang et al., 2020b) that will be discussed in this presentation show that it may be useful to exploit the hierarchical nature of many object catalogues to obtain a consistent prediction of land use at multiple semantic levels.
The last part of this presentation will discuss two different strategies for deep domain adaptation for land cover classification. The assumption is that there is a source domain in which there is an abundance of labelled training data and a target domain where we only have the remote sensing data but no label information. If the data in the target domain follow a different distribution than those in the source domain, a classifier trained using the training samples in the source domain will perform poorly in the target domain. Domain adaptation comprises methods to mitigate this performance loss due to this so-called domain gap. The two methods discussed here are based on different strategies. The first one is based on Adversarial Discriminative Domain Adaptation and tries to learn a representation of the data that is independent from the domain from which a sample was drawn (Wittich & Rottensteiner, 2020). The second method is based on instance transfer, i.e. it tries to use samples in the target domain which receive their labels from the current state of the classifier, which is initialized using source domain data only. In order to get samples with high-quality labels, an additional loss that should compensate for the domain gap is introduced (Wittich, 2020). Both the potential and the limitations of these approaches will be discussed.
References
Krizhevsky, A., Sutskever, I., Hinton, G. E., 2012. ImageNet classification with deep convolutional neural networks. International Conference on Neural Information Processing Systems (NIPS’12) 25 Vol. 1, 1097-1105.
Voelsen, M., Bostelmann2, J., Maas, A., Rottensteiner, F., Heipke, C., 2020. Automatically generated training data for land couver classification with CNN using Sentinel-2 images. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLIII-B3, accepted for publication.
Wang, M., Deng, W., 2018. Deep visual domain adaptation: a survey. Neurocomputing 312, 135-153.
Wittich, D., Rottensteiner, F., 2019. Adversarial domain adaptation for the classification of aerial images and height data using convolutional neural networks. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-2/W7, 197–204.
Wittich, D., 2020. Deep domain adaptation by weighted entropy minimization for the classification of aerial images. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences V-2, 591–598.
Yang, C., Rottensteiner, F., Heipke, C., 2018. Classification of land cover and land use based on convolution neural networks. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-3, 251-258.
Yang, C., Rottensteiner, F., Heipke, C., 2020a. Exploring semantic relationships for hierarchical land use classification based on convolutional neural networks. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences V-2, 599–607.
Yang, C., Rottensteiner, F., Heipke, C., 2020b. Investigations on skip-connections with an additional cosine similarity loss for land cover classification. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences V-3, 339–346.
Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., Fraundorfer, F., 2017. Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine 5(4), 8-36.
About the Speaker
Dr. Rottensteiner received the Dipl.-Ing. degree in surveying and the Ph.D. degree and venia docendi in Photogrammetry, all from Vienna University of Technology, Vienna, Austria (TUW). Currently, he is an Associate Professor and leader of the research group “Photogrammetric Image Analysis” at the Institute of Photogrammetry and GeoInformation at the University of Hannover, Germany (LUH). His research interests include all aspects of image orientation, image classification, automated object detection and reconstruction from images and point clouds, and change detection from remote sensing data. Before joining LUH in 2008, he worked as a postdoctoral researcher at TUW and the Universities of New South Wales and Melbourne, both in Australia. He has authored or co-authored more than 100 scientific papers, 31 of which have appeared in peer-reviewed international journals. He received the Karl Rinner Award of the Austrian Geodetic Commission in 2004 and the Carl Pulfrich Award for Photogrammetry, sponsored by Leica Geosystems, in 2017. Since 2011, he has been the Associate Editor of the ISI-listed journal “Photogrammetrie Fernerkundung Geoinformation” of the German Society of Photogrammetry, Remote Sensing and Geoinformation. Being the Chairman of the working group II/4 of the International Society of Photogrammetry and Remote Sensing (ISPRS), he has initiated and conducted the ISPRS benchmark on urban object detection and 3D building reconstruction.
CLICK HERE TO REGISTER IN THIS EVENT
