Our project is done in close collaboration with the Technical University of Munich. In particular with the TUM Data Science in Earth Observation (Sipeo) group. The complete list of associated publications might be also interesting for you and is available here.
Publications
Publications
2021 |
|
Köninger, Julia; Lugato, Emanuele; Panagos, Panos; Kochupillai, Mrinalini; Orgiazzi, Alberto; Briones, Maria J I Manure management and soil biodiversity: Towards more sustainable food systems in the EU Journal Article Agricultural Systems, 194 , 2021, ISSN: 0308521X. Abstract | Links | BibTeX | Tags: @article{, title = {Manure management and soil biodiversity: Towards more sustainable food systems in the EU}, author = {Julia Köninger and Emanuele Lugato and Panos Panagos and Mrinalini Kochupillai and Alberto Orgiazzi and Maria J I Briones}, doi = {10.1016/J.AGSY.2021.103251}, issn = {0308521X}, year = {2021}, date = {2021-01-01}, journal = {Agricultural Systems}, volume = {194}, publisher = {Elsevier Ltd}, abstract = {CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals.}, keywords = {}, pubstate = {published}, tppubtype = {article} } CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals. | |
Vishwakarma, Bramha Dutt; Horwath, Martin; Groh, Andreas; Bamber, Jonathan L Accounting for GIA signal in GRACE products Journal Article Geophysical Journal International, 228 , pp. 2056-2060, 2021, ISSN: 0956-540X. Abstract | Links | BibTeX | Tags: @article{Vishwakarma2021, title = {Accounting for GIA signal in GRACE products}, author = {Bramha Dutt Vishwakarma and Martin Horwath and Andreas Groh and Jonathan L Bamber}, url = {https://academic.oup.com/gji/article/228/3/2056/6426181}, doi = {10.1093/GJI/GGAB464}, issn = {0956-540X}, year = {2021}, date = {2021-01-01}, journal = {Geophysical Journal International}, volume = {228}, pages = {2056-2060}, publisher = {Oxford Academic}, abstract = {The Gravity Recovery and Climate Experiment (GRACE) observes gravitational potential anomalies that include the effects of present-day surface mass change (PDSMC)-and glacial isostatic adjustment (GIA)-driven solid Earth mass redistribution. Therefore, GIA estimates from a forward model are commonly removed from GRACE to estimate PDSMC. There are several GIA models and to facilitate users in using a GIA model of their choice, both GRACE and GIA products are made available in terms of global gridded fields representing mass anomaly. GRACE-observed gravitational potential anomalies are represented in terms of equivalent water height (EWH) with a relation that accounts for an elastic solid Earth deformation due to PDSMC. However, for obtaining GIA EWH fields from GIA gravitational potential fields, two relations are being used: one that is similar to that being used for GRACE EWH and the other that does not include an elastic deformation effect. This leaves users with the possibility of obtaining different values for PDSMC with a given GRACE and GIA field. In this paper, we discuss the impact of this problem on regional mass change estimates and highlight the need for consistent treatment of GIA signals in GRACE observations.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The Gravity Recovery and Climate Experiment (GRACE) observes gravitational potential anomalies that include the effects of present-day surface mass change (PDSMC)-and glacial isostatic adjustment (GIA)-driven solid Earth mass redistribution. Therefore, GIA estimates from a forward model are commonly removed from GRACE to estimate PDSMC. There are several GIA models and to facilitate users in using a GIA model of their choice, both GRACE and GIA products are made available in terms of global gridded fields representing mass anomaly. GRACE-observed gravitational potential anomalies are represented in terms of equivalent water height (EWH) with a relation that accounts for an elastic solid Earth deformation due to PDSMC. However, for obtaining GIA EWH fields from GIA gravitational potential fields, two relations are being used: one that is similar to that being used for GRACE EWH and the other that does not include an elastic deformation effect. This leaves users with the possibility of obtaining different values for PDSMC with a given GRACE and GIA field. In this paper, we discuss the impact of this problem on regional mass change estimates and highlight the need for consistent treatment of GIA signals in GRACE observations. | |
Basirat, Mina; Roth, Peter M S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization Inproceedings Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, SCITEPRESS - Science and Technology Publications, 2021, ISBN: 9789897584886. Abstract | Links | BibTeX | Tags: @inproceedings{Basirat_2021, title = {S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization}, author = {Mina Basirat and Peter M Roth}, url = {https://orcid.org/0000-0001-9566-1298}, doi = {10.5220/0010338506450652}, isbn = {9789897584886}, year = {2021}, date = {2021-01-01}, booktitle = {Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications}, publisher = {SCITEPRESS - Science and Technology Publications}, abstract = {Recently, it was shown that using a properly parametrized Leaky ReLU (LReLU) as activation function yields significantly better results for a variety of image classification tasks. However, such methods are not feasible in practice. Either the only parameter (i.e., the slope of the negative part) needs to be set manually (L*ReLU), or the approach is vulnerable due to the gradient-based optimization and, thus, highly dependent on a proper initialization (PReLU). In this paper, we exploit the benefits of piecewise linear functions, avoiding these problems. To this end, we propose a fully automatic approach to estimate the slope parameter for LReLU from the data. We realize this via Stochastic Optimization, namely Particle Swarm Optimization (PSO): S*ReLU. In this way, we can show that, compared to widely-used activation functions (including PReLU), we can obtain better results on seven different benchmark datasets, however, also drastically reducing the computational effort.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Recently, it was shown that using a properly parametrized Leaky ReLU (LReLU) as activation function yields significantly better results for a variety of image classification tasks. However, such methods are not feasible in practice. Either the only parameter (i.e., the slope of the negative part) needs to be set manually (L*ReLU), or the approach is vulnerable due to the gradient-based optimization and, thus, highly dependent on a proper initialization (PReLU). In this paper, we exploit the benefits of piecewise linear functions, avoiding these problems. To this end, we propose a fully automatic approach to estimate the slope parameter for LReLU from the data. We realize this via Stochastic Optimization, namely Particle Swarm Optimization (PSO): S*ReLU. In this way, we can show that, compared to widely-used activation functions (including PReLU), we can obtain better results on seven different benchmark datasets, however, also drastically reducing the computational effort. | |
Musa, Osman; Jung, Peter; Caire, Giuseppe Plug-And-Play Learned Gaussian-mixture Approximate Message Passing Inproceedings pp. 4855-4859, IEEE, 2021, ISBN: 978-1-7281-7605-5. @inproceedings{Musa2021, title = {Plug-And-Play Learned Gaussian-mixture Approximate Message Passing}, author = {Osman Musa and Peter Jung and Giuseppe Caire}, doi = {10.1109/ICASSP39728.2021.9414910}, isbn = {978-1-7281-7605-5}, year = {2021}, date = {2021-01-01}, journal = {ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pages = {4855-4859}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Sauder, Jonathan; Genzel, Martin; Jung, Peter Learning Structured Sparse Matrices for Signal Recovery via Unrolled Optimization | OpenReview Inproceedings 2021. @inproceedings{Sauder2021, title = {Learning Structured Sparse Matrices for Signal Recovery via Unrolled Optimization | OpenReview}, author = {Jonathan Sauder and Martin Genzel and Peter Jung}, url = {https://openreview.net/forum?id=IxKSqOq1TKQ}, year = {2021}, date = {2021-01-01}, journal = {NeurIPS 2021 Workshop on Deep Learning and Inverse Problems}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Sánchez-Pastor, Jesús; Thanthrige, Udaya Miriya S K P; Ilgac, Furkan; Jiménez-Sáez, Alejandro; Jung, Peter; Sezgin, Aydin; Jakoby, Rolf Clutter Suppression for Indoor Self-Localization Systems by Iteratively Reweighted Low-Rank Plus Sparse Recovery Journal Article Sensors 2021, Vol. 21, Page 6842, 21 , pp. 6842, 2021, ISSN: 1424-8220. Abstract | Links | BibTeX | Tags: @article{SanchezPastor2021, title = {Clutter Suppression for Indoor Self-Localization Systems by Iteratively Reweighted Low-Rank Plus Sparse Recovery}, author = {Jesús Sánchez-Pastor and Udaya Miriya S K P Thanthrige and Furkan Ilgac and Alejandro Jiménez-Sáez and Peter Jung and Aydin Sezgin and Rolf Jakoby}, url = {https://www.mdpi.com/1424-8220/21/20/6842/htm https://www.mdpi.com/1424-8220/21/20/6842}, doi = {10.3390/S21206842}, issn = {1424-8220}, year = {2021}, date = {2021-01-01}, journal = {Sensors 2021, Vol. 21, Page 6842}, volume = {21}, pages = {6842}, publisher = {Multidisciplinary Digital Publishing Institute}, abstract = {Self-localization based on passive RFID-based has many potential applications. One of the main challenges it faces is the suppression of the reflected signals from unwanted objects (i.e., clutter). Typically, the clutter echoes are much stronger than the backscattered signals of the passive tag landmarks used in such scenarios. Therefore, successful tag detection can be very challenging. We consider two types of tags, namely low-Q and high-Q tags. The high-Q tag features a sparse frequency response, whereas the low-Q tag presents a broad frequency response. Further, the clutter usually showcases a short-lived response. In this work, we propose an iterative algorithm based on a low-rank plus sparse recovery approach (RPCA) to mitigate clutter and retrieve the landmark response. In addition to that, we compare the proposed approach with the well-known time-gating technique. It turns out that RPCA outperforms significantly time-gating for low-Q tags, achieving clutter suppression and tag identification when clutter encroaches on the time-gating window span, whereas it also increases the backscattered power at resonance by approximately 12 dB at 80 cm for high-Q tags. Altogether, RPCA seems a promising approach to improve the identification of passive indoor self-localization tag landmarks.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Self-localization based on passive RFID-based has many potential applications. One of the main challenges it faces is the suppression of the reflected signals from unwanted objects (i.e., clutter). Typically, the clutter echoes are much stronger than the backscattered signals of the passive tag landmarks used in such scenarios. Therefore, successful tag detection can be very challenging. We consider two types of tags, namely low-Q and high-Q tags. The high-Q tag features a sparse frequency response, whereas the low-Q tag presents a broad frequency response. Further, the clutter usually showcases a short-lived response. In this work, we propose an iterative algorithm based on a low-rank plus sparse recovery approach (RPCA) to mitigate clutter and retrieve the landmark response. In addition to that, we compare the proposed approach with the well-known time-gating technique. It turns out that RPCA outperforms significantly time-gating for low-Q tags, achieving clutter suppression and tag identification when clutter encroaches on the time-gating window span, whereas it also increases the backscattered power at resonance by approximately 12 dB at 80 cm for high-Q tags. Altogether, RPCA seems a promising approach to improve the identification of passive indoor self-localization tag landmarks. | |
Damara, Muhammad Fadli; Kornhardt, Gregor; Jung, Peter Solving Inverse Problems with Conditional-GAN Prior via Fast Network-Projected Gradient Descent Journal Article 2021. Abstract | Links | BibTeX | Tags: @article{Damara2021, title = {Solving Inverse Problems with Conditional-GAN Prior via Fast Network-Projected Gradient Descent}, author = {Muhammad Fadli Damara and Gregor Kornhardt and Peter Jung}, url = {https://arxiv.org/abs/2109.01105v1}, doi = {10.48550/arxiv.2109.01105}, year = {2021}, date = {2021-01-01}, abstract = {The projected gradient descent (PGD) method has shown to be effective in recovering compressed signals described in a data-driven way by a generative model, i.e., a generator which has learned the data distribution. Further reconstruction improvements for such inverse problems can be achieved by conditioning the generator on the measurement. The boundary equilibrium generative adversarial network (BEGAN) implements an equilibrium based loss function and an auto-encoding discriminator to better balance the performance of the generator and the discriminator. In this work we investigate a network-based projected gradient descent (NPGD) algorithm for measurement-conditional generative models to solve the inverse problem much faster than regular PGD. We combine the NPGD with conditional GAN/BEGAN to evaluate their effectiveness in solving compressed sensing type problems. Our experiments on the MNIST and CelebA datasets show that the combination of measurement conditional model with NPGD works well in recovering the compressed signal while achieving similar or in some cases even better performance along with a much faster reconstruction. The achieved reconstruction speed-up in our experiments is up to 140-175.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The projected gradient descent (PGD) method has shown to be effective in recovering compressed signals described in a data-driven way by a generative model, i.e., a generator which has learned the data distribution. Further reconstruction improvements for such inverse problems can be achieved by conditioning the generator on the measurement. The boundary equilibrium generative adversarial network (BEGAN) implements an equilibrium based loss function and an auto-encoding discriminator to better balance the performance of the generator and the discriminator. In this work we investigate a network-based projected gradient descent (NPGD) algorithm for measurement-conditional generative models to solve the inverse problem much faster than regular PGD. We combine the NPGD with conditional GAN/BEGAN to evaluate their effectiveness in solving compressed sensing type problems. Our experiments on the MNIST and CelebA datasets show that the combination of measurement conditional model with NPGD works well in recovering the compressed signal while achieving similar or in some cases even better performance along with a much faster reconstruction. The achieved reconstruction speed-up in our experiments is up to 140-175. | |
Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Out-of-distribution Detection in Satellite Image Classification Conference RobustML workshop at ICLR 2021, 2021. BibTeX | Tags: @conference{Gawlikowski2021b, title = {Out-of-distribution Detection in Satellite Image Classification}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {RobustML workshop at ICLR 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Saha, Sudipan; Zhu, Xiao Xiang Patch-level unsupervised planetary change detection Journal Article IEEE Geoscience and Remote Sensing Letters, 19 , 2021, ISSN: 15580571. Abstract | Links | BibTeX | Tags: @article{Saha2021d, title = {Patch-level unsupervised planetary change detection}, author = {Sudipan Saha and Xiao Xiang Zhu}, doi = {10.1109/LGRS.2021.3130862}, issn = {15580571}, year = {2021}, date = {2021-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, volume = {19}, publisher = {IEEE}, abstract = {Change detection (CD) is critical for analyzing data collected by planetary exploration missions, e.g., for identification of new impact craters. However, CD is still a relatively new topic in the context of planetary exploration. Sheer variation of planetary data makes CD much more challenging than in the case of Earth observation (EO). Unlike CD for EO, patch-level decision is preferred in planetary exploration as it is difficult to obtain perfect pixelwise alignment/coregistration between the bi-temporal planetary images. Lack of labeled bi-temporal data impedes supervised CD. To overcome these challenges, we propose an unsupervised CD method that exploits a pretrained feature extractor to obtain bi-temporal deep features that are further processed using global max-pooling to obtain patch-level feature description. Bi-temporal patch-level features are further analyzed based on difference to determine whether a patch is changed. Additionally, a self-supervised method is proposed to estimate the decision boundary between the changed and unchanged patches. Experimental results on three planetary CD datasets from two different planetary bodies (Mars and Moon) demonstrate that the proposed method often outperforms supervised planetary CD methods. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/planetaryCDUnsup.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Change detection (CD) is critical for analyzing data collected by planetary exploration missions, e.g., for identification of new impact craters. However, CD is still a relatively new topic in the context of planetary exploration. Sheer variation of planetary data makes CD much more challenging than in the case of Earth observation (EO). Unlike CD for EO, patch-level decision is preferred in planetary exploration as it is difficult to obtain perfect pixelwise alignment/coregistration between the bi-temporal planetary images. Lack of labeled bi-temporal data impedes supervised CD. To overcome these challenges, we propose an unsupervised CD method that exploits a pretrained feature extractor to obtain bi-temporal deep features that are further processed using global max-pooling to obtain patch-level feature description. Bi-temporal patch-level features are further analyzed based on difference to determine whether a patch is changed. Additionally, a self-supervised method is proposed to estimate the decision boundary between the changed and unchanged patches. Experimental results on three planetary CD datasets from two different planetary bodies (Mars and Moon) demonstrate that the proposed method often outperforms supervised planetary CD methods. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/planetaryCDUnsup. | |
Saha, Sudipan; Kondmann, Lukas; Song, Qian; Zhu, Xiao Xiang Change Detection in Hyperdimensional Images using Untrained Models Journal Article IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14 , pp. 11029, 2021. Abstract | Links | BibTeX | Tags: @article{Saha2021e, title = {Change Detection in Hyperdimensional Images using Untrained Models}, author = {Sudipan Saha and Lukas Kondmann and Qian Song and Xiao Xiang Zhu}, doi = {10.1109/JSTARS.2021.3121556}, year = {2021}, date = {2021-01-01}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, volume = {14}, pages = {11029}, publisher = {IEEE}, abstract = {Deep transfer-learning-based change detection methods are dependent on the availability of sensor-specific pretrained feature extractors. Such feature extractors are not always available due to lack of training data, especially for hyperspectral sensors and other hyperdimensional images. Moreover models trained on easily available multispectral (RGB/RGB-NIR) images cannot be reused on such hyperdimensional images due to their irregular number of bands. While hyperdimensional images show large number of spectral bands, they generally show much less spatial complexity, thus reducing the requirement of large receptive fields of convo-lution filters. Recent works in the computer vision have shown that even untrained deep models can yield remarkable result in some tasks like super-resolution and surface reconstruction. This motivates us to make a bold proposition that untrained lightweight deep model, initialized with some weight initialization strategy, can be used to extract useful semantic features from bi-temporal hyperdimensional images. Based on this proposition, we design a novel change detection framework for hyperdimensional images by extracting bitemporal features using an untrained model and further comparing the extracted features using deep change vector analysis to distinguish changed pixels from the unchanged ones. We further use the deep change hypervectors to cluster the changed pixels into different semantic groups. We conduct experiments on four change detection datasets: three hyperspectral datasets and a hyperdimensional polarimetric synthetic aperture radar dataset. The results clearly demonstrate that the proposed method is suitable for change detection in hyperdimensional remote sensing data.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Deep transfer-learning-based change detection methods are dependent on the availability of sensor-specific pretrained feature extractors. Such feature extractors are not always available due to lack of training data, especially for hyperspectral sensors and other hyperdimensional images. Moreover models trained on easily available multispectral (RGB/RGB-NIR) images cannot be reused on such hyperdimensional images due to their irregular number of bands. While hyperdimensional images show large number of spectral bands, they generally show much less spatial complexity, thus reducing the requirement of large receptive fields of convo-lution filters. Recent works in the computer vision have shown that even untrained deep models can yield remarkable result in some tasks like super-resolution and surface reconstruction. This motivates us to make a bold proposition that untrained lightweight deep model, initialized with some weight initialization strategy, can be used to extract useful semantic features from bi-temporal hyperdimensional images. Based on this proposition, we design a novel change detection framework for hyperdimensional images by extracting bitemporal features using an untrained model and further comparing the extracted features using deep change vector analysis to distinguish changed pixels from the unchanged ones. We further use the deep change hypervectors to cluster the changed pixels into different semantic groups. We conduct experiments on four change detection datasets: three hyperspectral datasets and a hyperdimensional polarimetric synthetic aperture radar dataset. The results clearly demonstrate that the proposed method is suitable for change detection in hyperdimensional remote sensing data. | |
Qian, Kun; Wang, Yuanyuan; Shi, Yilei; Zhu, Xiao Xiang γ-Net: Superresolving SAR Tomographic Inversion via Deep Learning Journal Article Forthcoming IEEE Transactions on Geoscience and Remote Sensing, Forthcoming, (in press). @article{Qian2021, title = {γ-Net: Superresolving SAR Tomographic Inversion via Deep Learning}, author = {Kun Qian and Yuanyuan Wang and Yilei Shi and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Transactions on Geoscience and Remote Sensing}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, abstract = {(This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.) Synthetic aperture radar tomography (TomoSAR) has been extensively employed in 3-D reconstruction in dense urban areas using high-resolution SAR acquisitions. Compressive sensing (CS)-based algorithms are generally considered as the state of the art in super-resolving TomoSAR, in particular in the single look case. This superior performance comes at the cost of extra computational burdens, because of the sparse reconstruction, which cannot be solved analytically and we need to employ computationally expensive iterative solvers. In this paper, we propose a novel deep learning-based super-resolving TomoSAR inversion approach, γ-Net, to tackle this challenge. γ-Net adopts advanced complex-valued learned iterative shrinkage thresholding algorithm (CV-LISTA) to mimic the iterative optimization step in sparse reconstruction. Simulations show the height estimate from a well-trained γ-Net approaches the Cramér-Rao lower bound while improving the computational efficiency by 1 to 2 orders of magnitude comparing to the first-order CS-based methods. It also shows no degradation in the super-resolution power comparing to the state-of-the-art second-order TomoSAR solvers, which are much more computationally expensive than the first-order methods. Specifically, γ-Net reaches more than 90% detection rate in moderate super-resolving cases at 25 measurements at 6dB SNR. Moreover, simulation at limited baselines demonstrates that the proposed algorithm outperforms the second-order CS-based method by a fair margin. Test on real TerraSAR-X data with just 6 interferograms also shows high-quality 3-D reconstruction with high-density detected double scatterers.}, note = {in press}, keywords = {}, pubstate = {forthcoming}, tppubtype = {article} } (This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.) Synthetic aperture radar tomography (TomoSAR) has been extensively employed in 3-D reconstruction in dense urban areas using high-resolution SAR acquisitions. Compressive sensing (CS)-based algorithms are generally considered as the state of the art in super-resolving TomoSAR, in particular in the single look case. This superior performance comes at the cost of extra computational burdens, because of the sparse reconstruction, which cannot be solved analytically and we need to employ computationally expensive iterative solvers. In this paper, we propose a novel deep learning-based super-resolving TomoSAR inversion approach, γ-Net, to tackle this challenge. γ-Net adopts advanced complex-valued learned iterative shrinkage thresholding algorithm (CV-LISTA) to mimic the iterative optimization step in sparse reconstruction. Simulations show the height estimate from a well-trained γ-Net approaches the Cramér-Rao lower bound while improving the computational efficiency by 1 to 2 orders of magnitude comparing to the first-order CS-based methods. It also shows no degradation in the super-resolution power comparing to the state-of-the-art second-order TomoSAR solvers, which are much more computationally expensive than the first-order methods. Specifically, γ-Net reaches more than 90% detection rate in moderate super-resolving cases at 25 measurements at 6dB SNR. Moreover, simulation at limited baselines demonstrates that the proposed algorithm outperforms the second-order CS-based method by a fair margin. Test on real TerraSAR-X data with just 6 interferograms also shows high-quality 3-D reconstruction with high-density detected double scatterers. | |
Saha, Sudipan; Ahmad, Tahir Federated transfer learning: Concept and applications Journal Article Intelligenza Artificiale, 15 (1), pp. 35-44, 2021, ISSN: 1724-8035. Abstract | Links | BibTeX | Tags: @article{Saha2021f, title = {Federated transfer learning: Concept and applications}, author = {Sudipan Saha and Tahir Ahmad}, doi = {10.3233/IA-200075}, issn = {1724-8035}, year = {2021}, date = {2021-01-01}, journal = {Intelligenza Artificiale}, volume = {15}, number = {1}, pages = {35-44}, publisher = {IOS Press}, abstract = {Development of Artificial Intelligence (AI) is inherently tied to the development of data. However, in most industries data exists in form of isolated islands, with limited scope of sharing between different organizations. This is an hindrance to the further development of AI. Federated learning has emerged as a possible solution to this problem in the last few years without compromising user privacy. Among different variants of the federated learning, noteworthy is federated transfer learning (FTL) that allows knowledge to be transferred across domains that do not have many overlapping features and users. In this work we provide a comprehensive survey of the existing works on this topic. In more details, we study the background of FTL and its different existing applications. We further analyze FTL from privacy and machine learning perspective.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Development of Artificial Intelligence (AI) is inherently tied to the development of data. However, in most industries data exists in form of isolated islands, with limited scope of sharing between different organizations. This is an hindrance to the further development of AI. Federated learning has emerged as a possible solution to this problem in the last few years without compromising user privacy. Among different variants of the federated learning, noteworthy is federated transfer learning (FTL) that allows knowledge to be transferred across domains that do not have many overlapping features and users. In this work we provide a comprehensive survey of the existing works on this topic. In more details, we study the background of FTL and its different existing applications. We further analyze FTL from privacy and machine learning perspective. | |
Köninger, Julia; Lugato, Emanuele; Panagos, Panos; Kochupillai, Mrinalini; Orgiazzi, Alberto; Briones, Maria J I Manure management and soil biodiversity: Towards more sustainable food systems in the EU Journal Article Agricultural Systems, 194 , 2021, ISSN: 0308521X. Abstract | Links | BibTeX | Tags: @article{Koeninger2021, title = {Manure management and soil biodiversity: Towards more sustainable food systems in the EU}, author = {Julia Köninger and Emanuele Lugato and Panos Panagos and Mrinalini Kochupillai and Alberto Orgiazzi and Maria J I Briones}, url = {http://creativecommons.org/licenses/by/4.0/}, doi = {10.1016/J.AGSY.2021.103251}, issn = {0308521X}, year = {2021}, date = {2021-01-01}, journal = {Agricultural Systems}, volume = {194}, publisher = {Elsevier Ltd}, abstract = {CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals.}, keywords = {}, pubstate = {published}, tppubtype = {article} } CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals. | |
Prexl, Jonathan; Saha, Sudipan; Zhu, Xiao Xiang Mitigating Spatial and Spectral Differences for Change Detection using Super-resolution and Unsupervised learning Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), IEEE, 2021. @conference{Prexl2021, title = {Mitigating Spatial and Spectral Differences for Change Detection using Super-resolution and Unsupervised learning}, author = {Jonathan Prexl and Sudipan Saha and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, publisher = {IEEE}, abstract = {Change detection (CD) is one of the most researched areas in remote sensing. However, most CD methods assume that the pre-change and post-change images are acquired by the same sensor, having the same set of spectral bands and same spatial resolution. This severely limits the applicability of CD methods. It is not trivial to apply the existing CD methods in multi-sensor scenario. Towards this direction, we propose an unsu-pervised CD method that can handle large differences in spatial resolution and can work with completely different set of spectral bands. The proposed method uses a self-supervised super-resolution strategy to upsample the lower resolution image , thus mitigating differences in spatial resolution. To mitigate spectral differences, a self-supervised learning strategy is used that ingests both images as input and trains a network using self-supervised loss accounting for the spectral differences in both images. Once trained this network is used in deep change vector analysis framework for change detection. We validated the proposed method in an experimental setup where the pre-change and post-change images have different spatial resolution (10 m and 20 m/pixel) and completely dis-joint set of spectral bands.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Change detection (CD) is one of the most researched areas in remote sensing. However, most CD methods assume that the pre-change and post-change images are acquired by the same sensor, having the same set of spectral bands and same spatial resolution. This severely limits the applicability of CD methods. It is not trivial to apply the existing CD methods in multi-sensor scenario. Towards this direction, we propose an unsu-pervised CD method that can handle large differences in spatial resolution and can work with completely different set of spectral bands. The proposed method uses a self-supervised super-resolution strategy to upsample the lower resolution image , thus mitigating differences in spatial resolution. To mitigate spectral differences, a self-supervised learning strategy is used that ingests both images as input and trains a network using self-supervised loss accounting for the spectral differences in both images. Once trained this network is used in deep change vector analysis framework for change detection. We validated the proposed method in an experimental setup where the pre-change and post-change images have different spatial resolution (10 m and 20 m/pixel) and completely dis-joint set of spectral bands. | |
Kondmann, Lukas; Toker, Aysim; Saha, Sudipan; Schölkopf, Bernhard; Leal-Taixé, Laura; Zhu, Xiao Xiang Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images Journal Article arXiv preprint arXiv:2110.02068, 2021. Abstract | Links | BibTeX | Tags: @article{Kondmann2021b, title = {Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images}, author = {Lukas Kondmann and Aysim Toker and Sudipan Saha and Bernhard Schölkopf and Laura Leal-Taixé and Xiao Xiang Zhu}, doi = {10.1109/tgrs.2021.3130842}, year = {2021}, date = {2021-01-01}, journal = {arXiv preprint arXiv:2110.02068}, abstract = {Detecting changes on the ground in multitemporal Earth observation data is one of the key problems in remote sensing. In this paper, we introduce Sibling Regression for Optical Change detection (SiROC), an unsupervised method for change detection in optical satellite images with medium and high resolution. SiROC is a spatial context-based method that models a pixel as a linear combination of its distant neighbors. It uses this model to analyze differences in the pixel and its spatial context-based predictions in subsequent time periods for change detection. We combine this spatial context-based change detection with ensembling over mutually exclusive neighborhoods and transitioning from pixel to object-level changes with morphological operations. SiROC achieves competitive performance for change detection with medium-resolution Sentinel-2 and high-resolution Planetscope imagery on four datasets. Besides accurate predictions without the need for training, SiROC also provides a well-calibrated uncertainty of its predictions. This makes the method especially useful in conjunction with deep-learning based methods for applications such as pseudo-labeling.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Detecting changes on the ground in multitemporal Earth observation data is one of the key problems in remote sensing. In this paper, we introduce Sibling Regression for Optical Change detection (SiROC), an unsupervised method for change detection in optical satellite images with medium and high resolution. SiROC is a spatial context-based method that models a pixel as a linear combination of its distant neighbors. It uses this model to analyze differences in the pixel and its spatial context-based predictions in subsequent time periods for change detection. We combine this spatial context-based change detection with ensembling over mutually exclusive neighborhoods and transitioning from pixel to object-level changes with morphological operations. SiROC achieves competitive performance for change detection with medium-resolution Sentinel-2 and high-resolution Planetscope imagery on four datasets. Besides accurate predictions without the need for training, SiROC also provides a well-calibrated uncertainty of its predictions. This makes the method especially useful in conjunction with deep-learning based methods for applications such as pseudo-labeling. | |
Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Towards Out-of-Distribution Detection for Remote Sensing Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), Institute of Electrical and Electronics Engineers (IEEE), 2021, ISBN: 9781665403696. Abstract | Links | BibTeX | Tags: @conference{Gawlikowski2021b, title = {Towards Out-of-Distribution Detection for Remote Sensing}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553266}, isbn = {9781665403696}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, pages = {8676-8679}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, abstract = {In remote sensing, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data, differences in the geographic area, and multi-sensor differences. Deep learning based models may behave in unexpected manners when subjected to test data that has such distributional shifts from the training data, also called out-of-distribution (OOD) examples. Vulnerability to OOD data severely reduces the reliability of deep learning based models. In this work, we address this issue by proposing a model to quantify distributional uncertainty of deep learning based remote sensing models. In particular, we adopt a Dirichlet Prior Network for remote sensing data. The approach seeks to maximize the representation gap between the in-domain and OOD examples for a better identification of unknown examples at test time. Experimental results on three exemplary test scenarios show that the proposed model can detect OOD images in remote sensing.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } In remote sensing, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data, differences in the geographic area, and multi-sensor differences. Deep learning based models may behave in unexpected manners when subjected to test data that has such distributional shifts from the training data, also called out-of-distribution (OOD) examples. Vulnerability to OOD data severely reduces the reliability of deep learning based models. In this work, we address this issue by proposing a model to quantify distributional uncertainty of deep learning based remote sensing models. In particular, we adopt a Dirichlet Prior Network for remote sensing data. The approach seeks to maximize the representation gap between the in-domain and OOD examples for a better identification of unknown examples at test time. Experimental results on three exemplary test scenarios show that the proposed model can detect OOD images in remote sensing. | |
Saha, Sudipan; Banerjee, Biplab; Zhu, Xiao Xiang Trusting small training dataset for supervised change detection Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), Institute of Electrical and Electronics Engineers (IEEE), 2021, ISBN: 9781665403696. Abstract | Links | BibTeX | Tags: @conference{Saha2021g, title = {Trusting small training dataset for supervised change detection}, author = {Sudipan Saha and Biplab Banerjee and Xiao Xiang Zhu}, url = {https://arxiv.org/abs/2104.05443v1}, doi = {10.48550/arxiv.2104.05443}, isbn = {9781665403696}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, pages = {2031-2034}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, abstract = {Deep learning (DL) based supervised change detection (CD) models require large labeled training data. Due to the difficulty of collecting labeled multi-temporal data, unsupervised methods are preferred in the CD literature. However, unsupervised methods cannot fully exploit the potentials of data-driven deep learning and thus they are not absolute alternative to the supervised methods. This motivates us to look deeper into the supervised DL methods and investigate how they can be adopted intelligently for CD by minimizing the requirement of labeled training data. Towards this, in this work we show that geographically diverse training dataset can yield significant improvement over less diverse training datasets of the same size. We propose a simple confidence indicator for verifying the trustworthiness/confidence of supervised models trained with small labeled dataset. Moreover, we show that for the test cases where supervised CD model is found to be less confident/trustworthy, unsupervised methods often produce better result than the supervised ones.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Deep learning (DL) based supervised change detection (CD) models require large labeled training data. Due to the difficulty of collecting labeled multi-temporal data, unsupervised methods are preferred in the CD literature. However, unsupervised methods cannot fully exploit the potentials of data-driven deep learning and thus they are not absolute alternative to the supervised methods. This motivates us to look deeper into the supervised DL methods and investigate how they can be adopted intelligently for CD by minimizing the requirement of labeled training data. Towards this, in this work we show that geographically diverse training dataset can yield significant improvement over less diverse training datasets of the same size. We propose a simple confidence indicator for verifying the trustworthiness/confidence of supervised models trained with small labeled dataset. Moreover, we show that for the test cases where supervised CD model is found to be less confident/trustworthy, unsupervised methods often produce better result than the supervised ones. | |
Nandy, Jay; Saha, Sudipan; Hsu, Wynne; Lee, Mong; Zhu, Xiao Covariate Shift Adaptation for Adversarially Robust Classifier Conference ICLR 2021 Workshop on Security and Safety in Machine Learning Systems, 2021. BibTeX | Tags: @conference{Nandy2021, title = {Covariate Shift Adaptation for Adversarially Robust Classifier}, author = {Jay Nandy and Sudipan Saha and Wynne Hsu and Mong Lee and Xiao Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {ICLR 2021 Workshop on Security and Safety in Machine Learning Systems}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Tuia, Xiao Xiang Zhu Markus Reichstein Gustau Camps-Valls Devis Wiley & Sons, 2021, ISBN: 978-1-119-64614-3. Abstract | Links | BibTeX | Tags: @book{EditorsCampsValls2021, title = {Deep learning for the earth sciences : a comprehensive approach to remote sensing, climate science and geosciences}, author = {Xiao Xiang Zhu Markus Reichstein Gustau Camps-Valls Devis Tuia}, editor = {Xiao Xiang Zhu Markus Reichstein Gustau Camps-Valls Devis Tuia}, url = {https://www.wiley-vch.de/de/fachgebiete/ingenieurwesen/elektrotechnik-und-elektronik-10ee/bildgebende-systeme-u-verfahren-10eec/fernerkundung-10eec1/deep-learning-for-the-earth-sciences-978-1-119-64614-3}, isbn = {978-1-119-64614-3}, year = {2021}, date = {2021-01-01}, publisher = {Wiley & Sons}, abstract = {"The research in deep learning for the geosciences and Earth observation is growing fast and goes beyond the mere application of algorithms to new data. This is a huge interdisciplinary field. Applying new algorithms to the data deluge is a hot topic in all these cross-sectorial fields. Academic research on this area is strongly involved, and many specialized conferences and special issues in journals are arising each year. The book will provide the reader with the landscape, skills, and principles to quickly become familiar with both fields? needs and applications and will give a principled status of where are we now. The practitioner will be ready to use the technology and principles in his/her own research field in a short period of time. The highlights on future research at the end of each chapter will provide new ideas, particularly for those people involved in advanced research education, who will find these highlights of special interest for PhD Thesis orientations"--}, keywords = {}, pubstate = {published}, tppubtype = {book} } "The research in deep learning for the geosciences and Earth observation is growing fast and goes beyond the mere application of algorithms to new data. This is a huge interdisciplinary field. Applying new algorithms to the data deluge is a hot topic in all these cross-sectorial fields. Academic research on this area is strongly involved, and many specialized conferences and special issues in journals are arising each year. The book will provide the reader with the landscape, skills, and principles to quickly become familiar with both fields? needs and applications and will give a principled status of where are we now. The practitioner will be ready to use the technology and principles in his/her own research field in a short period of time. The highlights on future research at the end of each chapter will provide new ideas, particularly for those people involved in advanced research education, who will find these highlights of special interest for PhD Thesis orientations"-- | |
Saha, Sudipan; Kondmann, Lukas; Zhu, Xiao Xiang Deep No Learning Approach for Unsupervised Change Detection in Hyperspectral Images Conference XXIV ISPRS Congress 2021, 2021. @conference{Saha2021bb, title = {Deep No Learning Approach for Unsupervised Change Detection in Hyperspectral Images}, author = {Sudipan Saha and Lukas Kondmann and Xiao Xiang Zhu}, url = {https://www.researchgate.net/publication/352887191_Deep_No_Learning_Approach_for_Unsupervised_Change_Detection_in_Hyperspectral_Images}, year = {2021}, date = {2021-01-01}, booktitle = {XXIV ISPRS Congress 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Ebel, Patrick; Saha, Sudipan; Zhu, Xiao Xiang Fusing Multi-modal Data for Supervised Change Detection Conference XXIV ISPRS Congress 2021, 2021. Abstract | Links | BibTeX | Tags: @conference{Ebel2021, title = {Fusing Multi-modal Data for Supervised Change Detection}, author = {Patrick Ebel and Sudipan Saha and Xiao Xiang Zhu}, url = {https://doi.org/10.5194/isprs-archives-XLIII-B3-2021-243-2021}, doi = {10.5194/isprs-archives-XLIII-B3-2021-243-2021}, year = {2021}, date = {2021-01-01}, booktitle = {XXIV ISPRS Congress 2021}, abstract = {With the rapid development of remote sensing technology in the last decade, different modalities of remote sensing data recorded via a variety of sensors are now easily accessible. Different sensors often provide complementary information and thus a more detailed and accurate Earth observation is possible by integrating their joint information. While change detection methods have been traditionally proposed for homogeneous data, combining multi-sensor multi-temporal data with different characteristics and resolution may provide a more robust interpretation of spatio-temporal evolution. However, integration of multi-temporal information from disparate sensory sources is challenging. Moreover, research in this direction is often hindered by a lack of available multi-modal data sets. To resolve these current shortcomings we curate a novel data set for multi-modal change detection. We further propose a novel Siamese architecture for fusion of SAR and optical observations for multi-modal change detection, which underlines the value of our newly gathered data. An experimental validation on the aforementioned data set demonstrates the potentials of the proposed model, which outperforms common mono-modal methods compared against.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } With the rapid development of remote sensing technology in the last decade, different modalities of remote sensing data recorded via a variety of sensors are now easily accessible. Different sensors often provide complementary information and thus a more detailed and accurate Earth observation is possible by integrating their joint information. While change detection methods have been traditionally proposed for homogeneous data, combining multi-sensor multi-temporal data with different characteristics and resolution may provide a more robust interpretation of spatio-temporal evolution. However, integration of multi-temporal information from disparate sensory sources is challenging. Moreover, research in this direction is often hindered by a lack of available multi-modal data sets. To resolve these current shortcomings we curate a novel data set for multi-modal change detection. We further propose a novel Siamese architecture for fusion of SAR and optical observations for multi-modal change detection, which underlines the value of our newly gathered data. An experimental validation on the aforementioned data set demonstrates the potentials of the proposed model, which outperforms common mono-modal methods compared against. | |
Klemmer, Konstantin; Saha, Sudipan; Kahl, Matthias; Xu, Tianlin; Zhu, Xiao Xiang Generative modeling of spatio-temporal weather patterns with extreme event conditioning Inproceedings AI: Modeling Oceans and Climate Change (AIMOCC 2021) Workshop, ICLR 2021, 2021. Abstract | Links | BibTeX | Tags: @inproceedings{Klemmer2021b, title = {Generative modeling of spatio-temporal weather patterns with extreme event conditioning}, author = {Konstantin Klemmer and Sudipan Saha and Matthias Kahl and Tianlin Xu and Xiao Xiang Zhu}, url = {https://arxiv.org/abs/2104.12469v1}, doi = {10.48550/arxiv.2104.12469}, year = {2021}, date = {2021-01-01}, booktitle = {AI: Modeling Oceans and Climate Change (AIMOCC 2021) Workshop, ICLR 2021}, abstract = {Deep generative models are increasingly used to gain insights in the geospatial data domain, e.g., for climate data. However, most existing approaches work with temporal snapshots or assume 1D time-series; few are able to capture spatio-temporal processes simultaneously. Beyond this, Earth-systems data often exhibit highly irregular and complex patterns, for example caused by extreme weather events. Because of climate change, these phenomena are only increasing in frequency. Here, we proposed a novel GAN-based approach for generating spatio-temporal weather patterns conditioned on detected extreme events. Our approach augments GAN generator and discriminator with an encoded extreme weather event segmentation mask. These segmentation masks can be created from raw input using existing event detection frameworks. As such, our approach is highly modular and can be combined with custom GAN architectures. We highlight the applicability of our proposed approach in experiments with real-world surface radiation and zonal wind data.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Deep generative models are increasingly used to gain insights in the geospatial data domain, e.g., for climate data. However, most existing approaches work with temporal snapshots or assume 1D time-series; few are able to capture spatio-temporal processes simultaneously. Beyond this, Earth-systems data often exhibit highly irregular and complex patterns, for example caused by extreme weather events. Because of climate change, these phenomena are only increasing in frequency. Here, we proposed a novel GAN-based approach for generating spatio-temporal weather patterns conditioned on detected extreme events. Our approach augments GAN generator and discriminator with an encoded extreme weather event segmentation mask. These segmentation masks can be created from raw input using existing event detection frameworks. As such, our approach is highly modular and can be combined with custom GAN architectures. We highlight the applicability of our proposed approach in experiments with real-world surface radiation and zonal wind data. | |
Kochupillai, Mrinalini; Gallersdörfer, Ulrich; Köninger, Julia; Beck, Roman Incentivizing research & innovation with agrobiodiversity conserved in situ: Possibilities and limitations of a blockchain-based solution Journal Article Journal of Cleaner Production, 309 , pp. 127155, 2021, ISSN: 0959-6526. @article{Kochupillai2021a, title = {Incentivizing research & innovation with agrobiodiversity conserved in situ: Possibilities and limitations of a blockchain-based solution}, author = {Mrinalini Kochupillai and Ulrich Gallersdörfer and Julia Köninger and Roman Beck}, issn = {0959-6526}, year = {2021}, date = {2021-01-01}, journal = {Journal of Cleaner Production}, volume = {309}, pages = {127155}, abstract = {This article explores the use of blockchain for agrobiodiversity (B4A) with a specific focus on (i) providing an overview of the existing regulatory challenges when it comes to conserving agrobiodiversity, which results in a lack of research and innovation when it comes to agrobiodiversity conserved in situ, (ii) investigating how a blockchain-based solution may help overcome these challenges, and (iii) illustrating how incentive mechanisms can help to overcome existing intellectual property regimes that prevent effective conservation, research and innovation (CRI). Our research identifies (i) lack of incentives, (ii) lack of trust among stakeholders, and (iii) lack of traceability options as main hindering reasons for in situ CRI with agrobiodiversity. Further, We find that blockchain solutions may empower data providers, including small farmers, to collectively track, control and monetize the use of data and assets shared, while minimizing fraudulent activities. Transaction costs may also be lowered by removing complex and expensive interaction processes. However, further research and development is necessary to design an ethical and sustainable blockchain-based solution to incentivize in situ conservation, research and innovation with agrobiodiversity. Some future directions of research are recommended.}, keywords = {}, pubstate = {published}, tppubtype = {article} } This article explores the use of blockchain for agrobiodiversity (B4A) with a specific focus on (i) providing an overview of the existing regulatory challenges when it comes to conserving agrobiodiversity, which results in a lack of research and innovation when it comes to agrobiodiversity conserved in situ, (ii) investigating how a blockchain-based solution may help overcome these challenges, and (iii) illustrating how incentive mechanisms can help to overcome existing intellectual property regimes that prevent effective conservation, research and innovation (CRI). Our research identifies (i) lack of incentives, (ii) lack of trust among stakeholders, and (iii) lack of traceability options as main hindering reasons for in situ CRI with agrobiodiversity. Further, We find that blockchain solutions may empower data providers, including small farmers, to collectively track, control and monetize the use of data and assets shared, while minimizing fraudulent activities. Transaction costs may also be lowered by removing complex and expensive interaction processes. However, further research and development is necessary to design an ethical and sustainable blockchain-based solution to incentivize in situ conservation, research and innovation with agrobiodiversity. Some future directions of research are recommended. | |
Kochupillai, Mrinalini; Köninger, Julia 2021. @conference{Kochupillai2021c, title = {Creating a Digital Marketplace for Agrobiodiversity and Plant Genetic Sequence Data: Legal and Ethical Considerations of an AI and Blockchain Based Solution Sustainable Innovation in Plant Varieties View project TRIPs, Patent Law and Policy View project}, author = {Mrinalini Kochupillai and Julia Köninger}, url = {https://www.researchgate.net/publication/350546174}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Ahmed, Nouman; Saha, Sudipan; Mohsin, Maaz; Shahzad, Muhammad; Zhu, Xiao Xiang Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image Inproceedings International Conference on Computer Vision (ICCV), pp. 752–761, 2021. Abstract | Links | BibTeX | Tags: @inproceedings{Ahmed2021b, title = {Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image}, author = {Nouman Ahmed and Sudipan Saha and Maaz Mohsin and Muhammad Shahzad and Xiao Xiang Zhu}, url = {https://elib.dlr.de/145759/}, year = {2021}, date = {2021-01-01}, booktitle = {International Conference on Computer Vision (ICCV)}, journal = {ICCV 2021 workshop on Learning to Understand Aerial Images (LUAI)}, pages = {752--761}, abstract = {Automated forest mapping is important to understand our forests that play a key role in ecological system. However, efforts towards forest mapping is impeded by difficulty to collect labeled forest images that show large intraclass variation. Recently unsupervised learning has shown promising capability when exploiting limited labeled data. Motivated by this, we propose a progressive unsupervised deep transfer learning method for forest mapping. The proposed method exploits a pre-trained model that is subsequently fine-tuned over the target forest domain. We propose two different fine-tuning echanism, one works in a totally unsupervised setting by jointly learning the parameters of CNN and the k-means based cluster assignments of the resulting features and the other one works in a semi-supervised setting by exploiting the extracted k-nearest neighbor based pseudo labels. The proposed progressive scheme is evaluated on publicly available EuroSAT dataset using the relevant base model trained on BigEarth-Net labels. The results show that the proposed method greatly improves the forest regions classification accuracy as compared to the unsupervised baseline, nearly approaching the supervised classification approach.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Automated forest mapping is important to understand our forests that play a key role in ecological system. However, efforts towards forest mapping is impeded by difficulty to collect labeled forest images that show large intraclass variation. Recently unsupervised learning has shown promising capability when exploiting limited labeled data. Motivated by this, we propose a progressive unsupervised deep transfer learning method for forest mapping. The proposed method exploits a pre-trained model that is subsequently fine-tuned over the target forest domain. We propose two different fine-tuning echanism, one works in a totally unsupervised setting by jointly learning the parameters of CNN and the k-means based cluster assignments of the resulting features and the other one works in a semi-supervised setting by exploiting the extracted k-nearest neighbor based pseudo labels. The proposed progressive scheme is evaluated on publicly available EuroSAT dataset using the relevant base model trained on BigEarth-Net labels. The results show that the proposed method greatly improves the forest regions classification accuracy as compared to the unsupervised baseline, nearly approaching the supervised classification approach. | |
Thanthrige, Udaya Miriya S K P; Jung, Peter; Sezgin, Aydin Deep Unfolding of Iteratively Reweighted ADMM for Wireless RF Sensing Miscellaneous 2021. Abstract | Links | BibTeX | Tags: @misc{Thanthrige2021, title = {Deep Unfolding of Iteratively Reweighted ADMM for Wireless RF Sensing}, author = {Udaya Miriya S K P Thanthrige and Peter Jung and Aydin Sezgin}, url = {https://arxiv.org/abs/2106.03686v3}, doi = {10.48550/arxiv.2106.03686}, year = {2021}, date = {2021-01-01}, publisher = {arXiv}, abstract = {We address the detection of material defects, which are inside a layered material structure using compressive sensing based multiple-input and multiple-output (MIMO) wireless radar. Here, the strong clutter due to the reflection of the layered structure's surface often makes the detection of the defects challenging. Thus, sophisticated signal separation methods are required for improved defect detection. In many scenarios, the number of defects that we are interested in is limited and the signaling response of the layered structure can be modeled as a low-rank structure. Therefore, we propose joint rank and sparsity minimization for defect detection. In particular, we propose a non-convex approach based on the iteratively reweighted nuclear and $ell_1-$norm (a double-reweighted approach) to obtain a higher accuracy compared to the conventional nuclear norm and $ell_1-$norm minimization. To this end, an iterative algorithm is designed to estimate the low-rank and sparse contributions. Further, we propose deep learning to learn the parameters of the algorithm (i.e., algorithm unfolding) to improve the accuracy and the speed of convergence of the algorithm. Our numerical results show that the proposed approach outperforms the conventional approaches in terms of mean square errors of the recovered low-rank and sparse components and the speed of convergence.}, keywords = {}, pubstate = {published}, tppubtype = {misc} } We address the detection of material defects, which are inside a layered material structure using compressive sensing based multiple-input and multiple-output (MIMO) wireless radar. Here, the strong clutter due to the reflection of the layered structure's surface often makes the detection of the defects challenging. Thus, sophisticated signal separation methods are required for improved defect detection. In many scenarios, the number of defects that we are interested in is limited and the signaling response of the layered structure can be modeled as a low-rank structure. Therefore, we propose joint rank and sparsity minimization for defect detection. In particular, we propose a non-convex approach based on the iteratively reweighted nuclear and $ell_1-$norm (a double-reweighted approach) to obtain a higher accuracy compared to the conventional nuclear norm and $ell_1-$norm minimization. To this end, an iterative algorithm is designed to estimate the low-rank and sparse contributions. Further, we propose deep learning to learn the parameters of the algorithm (i.e., algorithm unfolding) to improve the accuracy and the speed of convergence of the algorithm. Our numerical results show that the proposed approach outperforms the conventional approaches in terms of mean square errors of the recovered low-rank and sparse components and the speed of convergence. | |
Basirat, Mina; Geiger, Bernhard; Roth, Peter M A Geometric Perspective on Information Plane Analysis Journal Article Entropy, 23 , pp. 711, 2021. @article{article, title = {A Geometric Perspective on Information Plane Analysis}, author = {Mina Basirat and Bernhard Geiger and Peter M Roth}, doi = {10.3390/e23060711}, year = {2021}, date = {2021-01-01}, journal = {Entropy}, volume = {23}, pages = {711}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Mandl, David; Peter M, ; Langlotz, Tobias; Ebner, Christoph; Mori, Shohei; Zollmann, Stefanie; Mohr, Peter; Kalkofen, Denis Neural Cameras: Learning Camera Characteristics for Coherent Mixed Reality Rendering Inproceedings 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 508–516, IEEE Publications, 2021, ISBN: 978-1-7281-9777-7, (null ; Conference date: 04-10-2021 Through 08-10-2021). Abstract | Links | BibTeX | Tags: @inproceedings{238e0aabb29b487b99c5315be5d8d2ed, title = {Neural Cameras: Learning Camera Characteristics for Coherent Mixed Reality Rendering}, author = {David Mandl and {Peter M } Roth and Tobias Langlotz and Christoph Ebner and Shohei Mori and Stefanie Zollmann and Peter Mohr and Denis Kalkofen}, doi = {10.1109/ISMAR52148.2021.00068}, isbn = {978-1-7281-9777-7}, year = {2021}, date = {2021-01-01}, booktitle = {2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)}, pages = {508--516}, publisher = {IEEE Publications}, abstract = {Coherent rendering is important for generating plausible Mixed Reality presentations of virtual objects within a usertextquoterights real-world environment. Besides photo-realistic rendering and correct lighting, visual coherence requires simulating the imaging system that is used to capture the real environment. While existing approaches either focus on a specific camera or a specific component of the imaging system, we introduce Neural Cameras, the first approach that jointly simulates all major components of an arbitrary modern camera using neural networks. Our system allows for adding new cameras to the framework by learning the visual properties from a database of images that has been captured using the physical camera. We present qualitative and quantitative results and discuss future direction for research that emerge from using Neural Cameras.}, note = {null ; Conference date: 04-10-2021 Through 08-10-2021}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Coherent rendering is important for generating plausible Mixed Reality presentations of virtual objects within a usertextquoterights real-world environment. Besides photo-realistic rendering and correct lighting, visual coherence requires simulating the imaging system that is used to capture the real environment. While existing approaches either focus on a specific camera or a specific component of the imaging system, we introduce Neural Cameras, the first approach that jointly simulates all major components of an arbitrary modern camera using neural networks. Our system allows for adding new cameras to the framework by learning the visual properties from a database of images that has been captured using the physical camera. We present qualitative and quantitative results and discuss future direction for research that emerge from using Neural Cameras. | |
Pepe, Antonio; Egger, Jan; Codari, Marina; Willemink, Martin J; Gsaxner, Christina; Li, Jianning; Roth, Peter M; Mistelbauer, Gabriel; Schmalstieg, Dieter; Fleischmann, Dominik 2021. @misc{https://doi.org/10.48550/arxiv.2111.11269, title = {Automated cross-sectional view selection in CT angiography of aortic dissections with uncertainty awareness and retrospective clinical annotations}, author = {Antonio Pepe and Jan Egger and Marina Codari and Martin J Willemink and Christina Gsaxner and Jianning Li and Peter M Roth and Gabriel Mistelbauer and Dieter Schmalstieg and Dominik Fleischmann}, url = {https://arxiv.org/abs/2111.11269}, doi = {10.48550/ARXIV.2111.11269}, year = {2021}, date = {2021-01-01}, publisher = {arXiv}, keywords = {}, pubstate = {published}, tppubtype = {misc} } | |
Zhu, Xiao Xiang; Montazeri, Sina; Ali, Mohsin; Hua, Yuansheng; Wang, Yuanyuan; Mou, Lichao; Shi, Yilei; Xu, Feng; Bamler, Richard Deep Learning Meets SAR: Concepts, models, pitfalls, and perspectives Journal Article IEEE Geoscience and Remote Sensing Magazine, 9 (4), pp. 143-172, 2021. @article{9351574, title = {Deep Learning Meets SAR: Concepts, models, pitfalls, and perspectives}, author = {Xiao Xiang Zhu and Sina Montazeri and Mohsin Ali and Yuansheng Hua and Yuanyuan Wang and Lichao Mou and Yilei Shi and Feng Xu and Richard Bamler}, doi = {10.1109/MGRS.2020.3046356}, year = {2021}, date = {2021-01-01}, journal = {IEEE Geoscience and Remote Sensing Magazine}, volume = {9}, number = {4}, pages = {143-172}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Lin, Jianzhe; Mou, Lichao; Zhu, Xiao Xiang; Ji, Xiangyang; Wang, Jane Z Attention-Aware Pseudo-3-D Convolutional Neural Network for Hyperspectral Image Classification Journal Article IEEE Transactions on Geoscience and Remote Sensing, 59 (9), pp. 7790-7802, 2021. @article{9347797, title = {Attention-Aware Pseudo-3-D Convolutional Neural Network for Hyperspectral Image Classification}, author = {Jianzhe Lin and Lichao Mou and Xiao Xiang Zhu and Xiangyang Ji and Jane Z Wang}, doi = {10.1109/TGRS.2020.3038212}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, volume = {59}, number = {9}, pages = {7790-7802}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Lin, Jianzhe; Yu, Tianze; Mou, Lichao; Zhu, Xiaoxiang; Ward, Rabab Kreidieh; Wang, Jane Z Unifying Top–Down Views by Task-Specific Domain Adaptation Journal Article IEEE Transactions on Geoscience and Remote Sensing, 59 (6), pp. 4689-4702, 2021. @article{9210589, title = {Unifying Top–Down Views by Task-Specific Domain Adaptation}, author = {Jianzhe Lin and Tianze Yu and Lichao Mou and Xiaoxiang Zhu and Rabab Kreidieh Ward and Jane Z Wang}, doi = {10.1109/TGRS.2020.3022608}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, volume = {59}, number = {6}, pages = {4689-4702}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Saha, Sudipan; Mou, Lichao; Zhu, Xiao Xiang; Bovolo, Francesca; Bruzzone, Lorenzo Semisupervised Change Detection Using Graph Convolutional Network Journal Article IEEE Geoscience and Remote Sensing Letters, 18 (4), pp. 607-611, 2021. @article{9069898, title = {Semisupervised Change Detection Using Graph Convolutional Network}, author = {Sudipan Saha and Lichao Mou and Xiao Xiang Zhu and Francesca Bovolo and Lorenzo Bruzzone}, doi = {10.1109/LGRS.2020.2985340}, year = {2021}, date = {2021-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, volume = {18}, number = {4}, pages = {607-611}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Heidler, Konrad; Mou, Lichao; Zhu, Xiao Xiang Seeing the Bigger Picture: Enabling Large Context Windows in Neural Networks by Combining Multiple Zoom Levels Inproceedings 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 3033-3036, 2021. @inproceedings{9554434, title = {Seeing the Bigger Picture: Enabling Large Context Windows in Neural Networks by Combining Multiple Zoom Levels}, author = {Konrad Heidler and Lichao Mou and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9554434}, year = {2021}, date = {2021-01-01}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, pages = {3033-3036}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Heidler, Konrad; Mou, Lichao; Baumhoer, Celia; Dietz, Andreas; Zhu, Xiao Xiang Hed-Unet: A Multi-Scale Framework for Simultaneous Segmentation and Edge Detection Inproceedings 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 3037-3040, 2021. @inproceedings{9553585, title = {Hed-Unet: A Multi-Scale Framework for Simultaneous Segmentation and Edge Detection}, author = {Konrad Heidler and Lichao Mou and Celia Baumhoer and Andreas Dietz and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553585}, year = {2021}, date = {2021-01-01}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, pages = {3037-3040}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Jin, Pu; Mou, Lichao; Xia, Gui-Song; Zhu, Xiao Xiang Anomaly Detection in Aerial Videos Via Future Frame Prediction Networks Inproceedings 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 8237-8240, 2021. @inproceedings{9554396, title = {Anomaly Detection in Aerial Videos Via Future Frame Prediction Networks}, author = {Pu Jin and Lichao Mou and Gui-Song Xia and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9554396}, year = {2021}, date = {2021-01-01}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, pages = {8237-8240}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Jin, Pu; Mou, Lichao; Hua, Yuansheng; Xia, Gui-Song; Zhu, Xiao Xiang Temporal Relations Matter: A Two-Pathway Network for Aerial Video Recognition Inproceedings 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 8221-8224, 2021. @inproceedings{9554868, title = {Temporal Relations Matter: A Two-Pathway Network for Aerial Video Recognition}, author = {Pu Jin and Lichao Mou and Yuansheng Hua and Gui-Song Xia and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9554868}, year = {2021}, date = {2021-01-01}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, pages = {8221-8224}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Chen, Sining; Mou, Lichao; Li, Qingyu; Sun, Yao; Zhu, Xiao Xiang Mask-Height R-CNN: An End-to-End Network for 3D Building Reconstruction from Monocular Remote Sensing Imagery Inproceedings 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 1202-1205, 2021. @inproceedings{9553121, title = {Mask-Height R-CNN: An End-to-End Network for 3D Building Reconstruction from Monocular Remote Sensing Imagery}, author = {Sining Chen and Lichao Mou and Qingyu Li and Yao Sun and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553121}, year = {2021}, date = {2021-01-01}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, pages = {1202-1205}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Sun, Yao; Hua, Yuansheng; Mou, Lichao; Zhu, Xiao Xiang Conditional GIS-Aware Network for Individual Building Segmentation in a VHR SAR Image Inproceedings 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 4532-4535, 2021. @inproceedings{9553135, title = {Conditional GIS-Aware Network for Individual Building Segmentation in a VHR SAR Image}, author = {Yao Sun and Yuansheng Hua and Lichao Mou and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553135}, year = {2021}, date = {2021-01-01}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, pages = {4532-4535}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Luo, Cong; Hua, Yuansheng; Mou, Lichao; Zhu, Xiao Xiang Improving Land Cover Classification with a Shift-Invariant Center-Focusing Convolutional Neural Network Inproceedings 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 2863-2866, 2021. @inproceedings{9554678, title = {Improving Land Cover Classification with a Shift-Invariant Center-Focusing Convolutional Neural Network}, author = {Cong Luo and Yuansheng Hua and Lichao Mou and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9554678}, year = {2021}, date = {2021-01-01}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, pages = {2863-2866}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Abid, Nosheen; Shahzad, Muhammad; Malik, Muhammad Imran; Schwanecke, Ulrich; Ulges, Adrian; Kovács, György; Shafait, Faisal UCL: Unsupervised Curriculum Learning for water body classification from remote sensing imagery Journal Article International Journal of Applied Earth Observation and Geoinformation, 105 , pp. 102568, 2021, ISSN: 1569-8432. Abstract | Links | BibTeX | Tags: @article{Abid2021, title = {UCL: Unsupervised Curriculum Learning for water body classification from remote sensing imagery}, author = {Nosheen Abid and Muhammad Shahzad and Muhammad Imran Malik and Ulrich Schwanecke and Adrian Ulges and György Kovács and Faisal Shafait}, url = {https://www.sciencedirect.com/science/article/pii/S0303243421002750}, doi = {https://doi.org/10.1016/j.jag.2021.102568}, issn = {1569-8432}, year = {2021}, date = {2021-01-01}, journal = {International Journal of Applied Earth Observation and Geoinformation}, volume = {105}, pages = {102568}, abstract = {This paper presents a Convolutional Neural Networks (CNN) based Unsupervised Curriculum Learning approach for the recognition of water bodies to overcome the stated challenges for remote sensing based RGB imagery. The unsupervised nature of the presented algorithm eliminates the need for labelled training data. The problem is cast as a two class clustering problem (water and non-water), while clustering is done on deep features obtained by a pre-trained CNN. After initial clusters have been identified, representative samples from each cluster are chosen by the unsupervised curriculum learning algorithm for fine-tuning the feature extractor. The stated process is repeated iteratively until convergence. Three datasets have been used to evaluate the approach and show its effectiveness on varying scales: (i) SAT-6 dataset comprising high resolution aircraft images, (ii) Sentinel-2 of EuroSAT, comprising remote sensing images with low resolution, and (iii) PakSAT, a new dataset we created for this study. PakSAT is the first Pakistani Sentinel-2 dataset designed to classify water bodies of Pakistan. Extensive experiments on these datasets demonstrate the progressive learning behaviour of UCL and reported promising results of water classification on all three datasets. The obtained accuracies outperform the supervised methods in domain adaptation, demonstrating the effectiveness of the proposed algorithm.}, keywords = {}, pubstate = {published}, tppubtype = {article} } This paper presents a Convolutional Neural Networks (CNN) based Unsupervised Curriculum Learning approach for the recognition of water bodies to overcome the stated challenges for remote sensing based RGB imagery. The unsupervised nature of the presented algorithm eliminates the need for labelled training data. The problem is cast as a two class clustering problem (water and non-water), while clustering is done on deep features obtained by a pre-trained CNN. After initial clusters have been identified, representative samples from each cluster are chosen by the unsupervised curriculum learning algorithm for fine-tuning the feature extractor. The stated process is repeated iteratively until convergence. Three datasets have been used to evaluate the approach and show its effectiveness on varying scales: (i) SAT-6 dataset comprising high resolution aircraft images, (ii) Sentinel-2 of EuroSAT, comprising remote sensing images with low resolution, and (iii) PakSAT, a new dataset we created for this study. PakSAT is the first Pakistani Sentinel-2 dataset designed to classify water bodies of Pakistan. Extensive experiments on these datasets demonstrate the progressive learning behaviour of UCL and reported promising results of water classification on all three datasets. The obtained accuracies outperform the supervised methods in domain adaptation, demonstrating the effectiveness of the proposed algorithm. | |
Mughal, Muhammad Hamza; Khokhar, Muhammad Jawad; Shahzad, Muhammad Assisting UAV Localization Via Deep Contextual Image Matching Journal Article IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14 , pp. 2445-2457, 2021. @article{9336232, title = {Assisting UAV Localization Via Deep Contextual Image Matching}, author = {Muhammad Hamza Mughal and Muhammad Jawad Khokhar and Muhammad Shahzad}, doi = {10.1109/JSTARS.2021.3054832}, year = {2021}, date = {2021-01-01}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, volume = {14}, pages = {2445-2457}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Usmani, Fehmida; Khan, Ihtesham; Masood, Muhammad Umar; Ahmad, Arsalan; Shahzad, Muhammad; Curri, Vittorio Convolutional neural network for quality of transmission prediction of unestablished lightpaths Journal Article Microwave and Optical Technology Letters, 63 (10), pp. 2461-2469, 2021. Abstract | Links | BibTeX | Tags: @article{https://doi.org/10.1002/mop.32996, title = {Convolutional neural network for quality of transmission prediction of unestablished lightpaths}, author = {Fehmida Usmani and Ihtesham Khan and Muhammad Umar Masood and Arsalan Ahmad and Muhammad Shahzad and Vittorio Curri}, url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/mop.32996}, doi = {https://doi.org/10.1002/mop.32996}, year = {2021}, date = {2021-01-01}, journal = {Microwave and Optical Technology Letters}, volume = {63}, number = {10}, pages = {2461-2469}, abstract = {Abstract With the advancement in evolving concepts of software-defined networks and elastic-optical-network, the number of design parameters is growing dramatically, making the lightpath (LP) deployment more complex. Typically, worst-case assumptions are utilized to calculate the quality-of-transmission (QoT) with the provisioning of high-margin requirements. To this aim, precise and advanced estimation of the QoT of the LP is essential for reducing this provisioning margin. In this investigation, we present convolutional-neural-networks (CNN) based architecture to accurately calculate QoT before the actual deployment of LP in an unseen network. The proposed model is trained on the data acquired from already established LP of a completely different network. The metric considered to evaluate the QoT of LP is the generalized signal-to-noise ratio (GSNR). The synthetic dataset is generated by utilizing well appraised GNPy simulation tool. Promising results are achieved, showing that the proposed CNN model considerably minimizes the GSNR uncertainty and, consequently, the provisioning margin.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Abstract With the advancement in evolving concepts of software-defined networks and elastic-optical-network, the number of design parameters is growing dramatically, making the lightpath (LP) deployment more complex. Typically, worst-case assumptions are utilized to calculate the quality-of-transmission (QoT) with the provisioning of high-margin requirements. To this aim, precise and advanced estimation of the QoT of the LP is essential for reducing this provisioning margin. In this investigation, we present convolutional-neural-networks (CNN) based architecture to accurately calculate QoT before the actual deployment of LP in an unseen network. The proposed model is trained on the data acquired from already established LP of a completely different network. The metric considered to evaluate the QoT of LP is the generalized signal-to-noise ratio (GSNR). The synthetic dataset is generated by utilizing well appraised GNPy simulation tool. Promising results are achieved, showing that the proposed CNN model considerably minimizes the GSNR uncertainty and, consequently, the provisioning margin. | |
Zulfiqar, Annus; Ghaffar, Muhammad M; Shahzad, Muhammad; Weis, Christian; Malik, Muhammad I; Shafait, Faisal; Wehn, Norbert Journal of Applied Remote Sensing, 15 (2), pp. 1 – 21, 2021. Abstract | Links | BibTeX | Tags: @article{Zulfiqar2021, title = {AI-ForestWatch: semantic segmentation based end-to-end framework for forest estimation and change detection using multi-spectral remote sensing imagery}, author = {Annus Zulfiqar and Muhammad M Ghaffar and Muhammad Shahzad and Christian Weis and Muhammad I Malik and Faisal Shafait and Norbert Wehn}, url = {https://doi.org/10.1117/1.JRS.15.024518}, doi = {10.1117/1.JRS.15.024518}, year = {2021}, date = {2021-01-01}, journal = {Journal of Applied Remote Sensing}, volume = {15}, number = {2}, pages = {1 -- 21}, publisher = {SPIE}, abstract = {Forest change detection is crucial for sustainable forest management. The changes in the forest area due to deforestation (such as wild fires or logging due to development activities) or afforestation alter the total forest area. Additionally, it impacts the available stock for commercial purposes, climate change due to carbon emissions, and biodiversity of the forest habitat estimations, which are essential for disaster management and policy making. In recent years, foresters have relied on hand-crafted features or bi-temporal change detection methods to detect change in the remote sensing imagery to estimate the forest area. Due to manual processing steps, these methods are fragile and prone to errors and can generate inaccurate (i.e., under or over) segmentation results. In contrast to traditional methods, we present AI-ForestWatch, an end to end framework for forest estimation and change analysis. The proposed approach uses deep convolution neural network-based semantic segmentation to process multi-spectral space-borne images to quantitatively monitor the forest cover change patterns by automatically extracting features from the dataset. Our analysis is completely data driven and has been performed using extended (with vegetation indices) Landsat-8 multi-spectral imagery from 2014 to 2020. As a case study, we estimated the forest area in 15 districts of Pakistan and generated forest change maps from 2014 to 2020, where major afforestation activity is carried out during this period. Our critical analysis shows an improvement of forest cover in 14 out of 15 districts. The AI-ForestWatch framework along with the associated dataset will be made public upon publication so that it can be adapted by other countries or regions.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Forest change detection is crucial for sustainable forest management. The changes in the forest area due to deforestation (such as wild fires or logging due to development activities) or afforestation alter the total forest area. Additionally, it impacts the available stock for commercial purposes, climate change due to carbon emissions, and biodiversity of the forest habitat estimations, which are essential for disaster management and policy making. In recent years, foresters have relied on hand-crafted features or bi-temporal change detection methods to detect change in the remote sensing imagery to estimate the forest area. Due to manual processing steps, these methods are fragile and prone to errors and can generate inaccurate (i.e., under or over) segmentation results. In contrast to traditional methods, we present AI-ForestWatch, an end to end framework for forest estimation and change analysis. The proposed approach uses deep convolution neural network-based semantic segmentation to process multi-spectral space-borne images to quantitatively monitor the forest cover change patterns by automatically extracting features from the dataset. Our analysis is completely data driven and has been performed using extended (with vegetation indices) Landsat-8 multi-spectral imagery from 2014 to 2020. As a case study, we estimated the forest area in 15 districts of Pakistan and generated forest change maps from 2014 to 2020, where major afforestation activity is carried out during this period. Our critical analysis shows an improvement of forest cover in 14 out of 15 districts. The AI-ForestWatch framework along with the associated dataset will be made public upon publication so that it can be adapted by other countries or regions. | |
Rizvi, Syed; Aslam, Warda; Shahzad, Muhammad; Fraz, Muhammad PROUD-MAL: static analysis-based progressive framework for deep unsupervised malware classification of windows portable executable Journal Article Complex & Intelligent Systems, pp. 1-13, 2021. @article{articleb, title = {PROUD-MAL: static analysis-based progressive framework for deep unsupervised malware classification of windows portable executable}, author = {Syed Rizvi and Warda Aslam and Muhammad Shahzad and Muhammad Fraz}, doi = {10.1007/s40747-021-00560-1}, year = {2021}, date = {2021-01-01}, journal = {Complex & Intelligent Systems}, pages = {1-13}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Nouman, Ahmed; Saha, Sudipan; Shahzad, Muhammad; Fraz, Muhammad Moazam; Zhu, Xiao Xiang Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image Inproceedings International Conference on Computer Vision (ICCV), pp. 752–761, 2021. Abstract | Links | BibTeX | Tags: @inproceedings{dlr145759, title = {Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image}, author = {Ahmed Nouman and Sudipan Saha and Muhammad Shahzad and Muhammad Moazam Fraz and Xiao Xiang Zhu}, url = {https://elib.dlr.de/145759/}, year = {2021}, date = {2021-01-01}, booktitle = {International Conference on Computer Vision (ICCV)}, journal = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops}, pages = {752--761}, abstract = {Automated forest mapping is important to understand our forests that play a key role in ecological system. However, efforts towards forest mapping is impeded by difficulty to collect labeled forest images that show large intraclass variation. Recently unsupervised learning has shown promising capability when exploiting limited labeled data. Motivated by this, we propose a progressive unsupervised deep transfer learning method for forest mapping. The proposed method exploits a pre-trained model that is subsequently fine-tuned over the target forest domain. We propose two different fine-tuning echanism, one works in a totally unsupervised setting by jointly learning the parameters of CNN and the k-means based cluster assignments of the resulting features and the other one works in a semi-supervised setting by exploiting the extracted k-nearest neighbor based pseudo labels. The proposed progressive scheme is evaluated on publicly available EuroSAT dataset using the relevant base model trained on BigEarth-Net labels. The results show that the proposed method greatly improves the forest regions classification accuracy as compared to the unsupervised baseline, nearly approaching the supervised classification approach.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Automated forest mapping is important to understand our forests that play a key role in ecological system. However, efforts towards forest mapping is impeded by difficulty to collect labeled forest images that show large intraclass variation. Recently unsupervised learning has shown promising capability when exploiting limited labeled data. Motivated by this, we propose a progressive unsupervised deep transfer learning method for forest mapping. The proposed method exploits a pre-trained model that is subsequently fine-tuned over the target forest domain. We propose two different fine-tuning echanism, one works in a totally unsupervised setting by jointly learning the parameters of CNN and the k-means based cluster assignments of the resulting features and the other one works in a semi-supervised setting by exploiting the extracted k-nearest neighbor based pseudo labels. The proposed progressive scheme is evaluated on publicly available EuroSAT dataset using the relevant base model trained on BigEarth-Net labels. The results show that the proposed method greatly improves the forest regions classification accuracy as compared to the unsupervised baseline, nearly approaching the supervised classification approach. | |
Abid, Nosheen; Malik, Muhammad Imran; Shahzad, Muhammad; Shafait, Faisal; Ali, Haider; Ghaffar, Muhammad Mohsin; Weis, Christian; Wehn, Norbert; Liwicki, Marcus Burnt Forest Estimation from Sentinel-2 Imagery of Australia using Unsupervised Deep Learning Inproceedings 2021 Digital Image Computing: Techniques and Applications (DICTA), pp. 01-08, 2021. @inproceedings{9647174, title = {Burnt Forest Estimation from Sentinel-2 Imagery of Australia using Unsupervised Deep Learning}, author = {Nosheen Abid and Muhammad Imran Malik and Muhammad Shahzad and Faisal Shafait and Haider Ali and Muhammad Mohsin Ghaffar and Christian Weis and Norbert Wehn and Marcus Liwicki}, doi = {10.1109/DICTA52665.2021.9647174}, year = {2021}, date = {2021-01-01}, booktitle = {2021 Digital Image Computing: Techniques and Applications (DICTA)}, pages = {01-08}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Javed, Muhammad Gohar; Raza, Minahil; Ghaffar, Muhammad Mohsin; Weis, Christian; Wehn, Norbert; Shahzad, Muhammad; Shafait, Faisal QuantYOLO: A High-Throughput and Power-Efficient Object Detection Network for Resource and Power Constrained UAVs Inproceedings 2021 Digital Image Computing: Techniques and Applications (DICTA), pp. 01-08, 2021. @inproceedings{9647121, title = {QuantYOLO: A High-Throughput and Power-Efficient Object Detection Network for Resource and Power Constrained UAVs}, author = {Muhammad Gohar Javed and Minahil Raza and Muhammad Mohsin Ghaffar and Christian Weis and Norbert Wehn and Muhammad Shahzad and Faisal Shafait}, doi = {10.1109/DICTA52665.2021.9647121}, year = {2021}, date = {2021-01-01}, booktitle = {2021 Digital Image Computing: Techniques and Applications (DICTA)}, pages = {01-08}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Saha, Sudipan; Mou, Lichao; Shahzad, Muhammad; Zhu, Xiao Xiang Segmentation of VHR EO Images using Unsupervised Learning Inproceedings arXiv, 2021, ISSN: 16130073. Abstract | Links | BibTeX | Tags: @inproceedings{Saha2021i, title = {Segmentation of VHR EO Images using Unsupervised Learning}, author = {Sudipan Saha and Lichao Mou and Muhammad Shahzad and Xiao Xiang Zhu}, url = {https://arxiv.org/abs/2108.04222v2}, doi = {10.48550/arxiv.2108.04222}, issn = {16130073}, year = {2021}, date = {2021-01-01}, journal = {ECML PKDD 2021 workshop Machine Learning for Earth Observation (MACLEAN)}, publisher = {arXiv}, abstract = {Semantic segmentation is a crucial step in many Earth observation tasks. Large quantity of pixel-level annotation is required to train deep networks for semantic segmentation. Earth observation techniques are applied to varieties of applications and since classes vary widely depending on the applications, therefore, domain knowledge is often required to label Earth observation images, impeding availability of labeled training data in many Earth observation applications. To tackle these challenges, in this paper we propose an unsupervised semantic segmentation method that can be trained using just a single unlabeled scene. Remote sensing scenes are generally large. The proposed method exploits this property to sample smaller patches from the larger scene and uses deep clustering and contrastive learning to refine the weights of a lightweight deep model composed of a series of the convolution layers along with an embedded channel attention. After unsupervised training on the target image/scene, the model automatically segregates the major classes present in the scene and produces the segmentation map. Experimental results on the Vaihingen dataset demonstrate the efficacy of the proposed method.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Semantic segmentation is a crucial step in many Earth observation tasks. Large quantity of pixel-level annotation is required to train deep networks for semantic segmentation. Earth observation techniques are applied to varieties of applications and since classes vary widely depending on the applications, therefore, domain knowledge is often required to label Earth observation images, impeding availability of labeled training data in many Earth observation applications. To tackle these challenges, in this paper we propose an unsupervised semantic segmentation method that can be trained using just a single unlabeled scene. Remote sensing scenes are generally large. The proposed method exploits this property to sample smaller patches from the larger scene and uses deep clustering and contrastive learning to refine the weights of a lightweight deep model composed of a series of the convolution layers along with an embedded channel attention. After unsupervised training on the target image/scene, the model automatically segregates the major classes present in the scene and produces the segmentation map. Experimental results on the Vaihingen dataset demonstrate the efficacy of the proposed method. | |
Baumhoer, Celia; Dietz, Andreas; Heidler, Konrad; Mou, LiChao; Zhu, Xiao Xiang; ü, Claudia K Towards an Antarctic Ice Shelf and Glacier Front Monitoring Service based on Sentinel-1 Data Inproceedings UK Antarctic Science Conference 2021, 2021. Abstract | Links | BibTeX | Tags: @inproceedings{dlr141526, title = {Towards an Antarctic Ice Shelf and Glacier Front Monitoring Service based on Sentinel-1 Data}, author = {Celia Baumhoer and Andreas Dietz and Konrad Heidler and LiChao Mou and Xiao Xiang Zhu and Claudia K ü}, url = {https://elib.dlr.de/141526/}, year = {2021}, date = {2021-00-01}, booktitle = {UK Antarctic Science Conference 2021}, abstract = {The extent of Antarctic ice shelves and glacier tongues is important for accurate sea level rise predictions. If ice shelf areas with strong buttressing effects are lost the ice discharge of the Antarctic Ice Sheet increases. Therefore, it is crucial to monitor ice shelf front positions and keep track of glacier and ice shelf front dynamics. The German Aerospace Center (DLR) is currently facilitating a project aiming to bring together the expertise in polar research: The Polar Monitor Project. Within this project, we are establishing an automated calving front monitoring service providing monthly Ice Shelf and Glacier Front Time Series (IceLines) via the EOC GeoService hosted by DLR. IceLines is based on a deep learning architecture combining segmentation and edge detection optimized for calving front extraction. The monthly front positions are extracted from Sentinel-1 SAR data over the Antarctic coastline. We will outline the implementation of IceLines within the DLR processing infrastructure, present first results and highlight the potential future usage for the scientific community. Future implementations within the Polar Monitor Project will also provide access to daily global snow cover (GlobalSnowPack) and the Antarctic grounding line derived from Sentinel-1 imagery.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The extent of Antarctic ice shelves and glacier tongues is important for accurate sea level rise predictions. If ice shelf areas with strong buttressing effects are lost the ice discharge of the Antarctic Ice Sheet increases. Therefore, it is crucial to monitor ice shelf front positions and keep track of glacier and ice shelf front dynamics. The German Aerospace Center (DLR) is currently facilitating a project aiming to bring together the expertise in polar research: The Polar Monitor Project. Within this project, we are establishing an automated calving front monitoring service providing monthly Ice Shelf and Glacier Front Time Series (IceLines) via the EOC GeoService hosted by DLR. IceLines is based on a deep learning architecture combining segmentation and edge detection optimized for calving front extraction. The monthly front positions are extracted from Sentinel-1 SAR data over the Antarctic coastline. We will outline the implementation of IceLines within the DLR processing infrastructure, present first results and highlight the potential future usage for the scientific community. Future implementations within the Polar Monitor Project will also provide access to daily global snow cover (GlobalSnowPack) and the Antarctic grounding line derived from Sentinel-1 imagery. |