Our project is done in close collaboration with the Technical University of Munich. In particular with the TUM Data Science in Earth Observation (Sipeo) group. The complete list of associated publications might be also interesting for you and is available here.
Publications
Publications
2022 |
|
Saha, Sudipan; Zhan, Shao; Shahzad, Muhammad; Zhu, Xiao Xiang Mitigating distribution shift for multi-sensor classification Inproceedings IEEE Geoscience and Remote Sensing Symposium 2022, IEEE, 2022. BibTeX | Tags: @inproceedings{Saha2022bb, title = {Mitigating distribution shift for multi-sensor classification}, author = {Sudipan Saha and Shao Zhan and Muhammad Shahzad and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2022}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Gawlikowski, Jakob; Saha, Sudipan; Niebling, Julia; Zhu, Xiao Xiang Robust Distribution-Shift Aware SAR-Optical Data Fusion for Multi-Label Scene Classification Inproceedings IEEE Geoscience and Remote Sensing Symposium 2022, IEEE, 2022. BibTeX | Tags: @inproceedings{Gawlikowski2022a, title = {Robust Distribution-Shift Aware SAR-Optical Data Fusion for Multi-Label Scene Classification}, author = {Jakob Gawlikowski and Sudipan Saha and Julia Niebling and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2022}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Hermann, Martin; Saha, Sudipan; Zhu, Xiao Xiang Few-Shot Filtering for the Detection of Specialized Change in Remote Sensing Inproceedings ICLR 2022 Workshop on Practical ML for Developing Countries, 2022. BibTeX | Tags: @inproceedings{Hermann2022, title = {Few-Shot Filtering for the Detection of Specialized Change in Remote Sensing}, author = {Martin Hermann and Sudipan Saha and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {ICLR 2022 Workshop on Practical ML for Developing Countries}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Li, Qingyu; Shi, Yilei; Zhu, Xiao Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training 2022. @unknown{unknown, title = {Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training}, author = {Qingyu Li and Yilei Shi and Xiao Zhu}, doi = {10.48550/arXiv.2205.08416}, year = {2022}, date = {2022-01-01}, keywords = {}, pubstate = {published}, tppubtype = {unknown} } | |
Diaconu, Codruț-Andrei; Saha, Sudipan; Günnemann, Stephan; Zhu, Xiao Xiang Understanding the Role of Weather Data for Earth Surface Forecasting using a ConvLSTM-based Model Inproceedings CVPR 2022 Workshop Earthvision, 2022. BibTeX | Tags: @inproceedings{Diaconu2022, title = {Understanding the Role of Weather Data for Earth Surface Forecasting using a ConvLSTM-based Model}, author = {Codruț-Andrei Diaconu and Sudipan Saha and Stephan Günnemann and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {CVPR 2022 Workshop Earthvision}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Gu, Ziqi; Ebel, Patrick; Yuan, Qiangqiang; Schmitt, Michael; Zhu, Xiao Xiang Explicit Haze & Cloud Removal for Global Land Cover Classification Inproceedings CVPR Workshops, pp. 1–6, 2022. BibTeX | Tags: @inproceedings{gu2022explicithcr, title = {Explicit Haze & Cloud Removal for Global Land Cover Classification}, author = {Ziqi Gu and Patrick Ebel and Qiangqiang Yuan and Michael Schmitt and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {CVPR Workshops}, pages = {1--6}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Saha, Sudipan; Zhao, Shan; Sheikh, Nasrullah; Zhu, Xiao Xiang Reiterative Domain Aware Multi-Target Adaptation Inproceedings German Conference on Pattern Recognition (GCPR), 2022. BibTeX | Tags: @inproceedings{Saha2022d, title = {Reiterative Domain Aware Multi-Target Adaptation}, author = {Sudipan Saha and Shan Zhao and Nasrullah Sheikh and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {German Conference on Pattern Recognition (GCPR)}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
M., Mou Song Zhu Saha Shahzad L Q X S Unsupervised Single-Scene Semantic Segmentation for Earth Observation Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2022. BibTeX | Tags: @article{Saha2022cb, title = {Unsupervised Single-Scene Semantic Segmentation for Earth Observation}, author = {Mou Song Zhu L Q X Saha S. Shahzad M.}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
2021 |
|
Usmani, Fehmida; Khan, Ihtesham; Siddiqui, Mehek; Khan, Mahnoor; Bilal, Muhammad; Masood, M; Ahmad, Arsalan; Shahzad, Muhammad; Curri, Vittorio Cross-feature trained machine learning models for QoT-estimation in optical networks Journal Article Optical Engineering, 60 (12), pp. 125106, 2021. @article{Usmani2021, title = {Cross-feature trained machine learning models for QoT-estimation in optical networks}, author = {Fehmida Usmani and Ihtesham Khan and Mehek Siddiqui and Mahnoor Khan and Muhammad Bilal and M Masood and Arsalan Ahmad and Muhammad Shahzad and Vittorio Curri}, doi = {10.1117/1.oe.60.12.125106}, year = {2021}, date = {2021-12-03}, journal = {Optical Engineering}, volume = {60}, number = {12}, pages = {125106}, publisher = {SPIE-Intl Soc Optical Eng}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Bloice, Marcus D; Roth, Peter M; Holzinger, Andreas Performing Arithmetic Using a Neural Network Trained on Images of Digit Permutation Pairs Journal Article J. Intell. Inf. Syst., 57 (3), pp. 547–562, 2021, ISSN: 0925-9902. Abstract | Links | BibTeX | Tags: @article{10.1007/s10844-021-00662-9, title = {Performing Arithmetic Using a Neural Network Trained on Images of Digit Permutation Pairs}, author = {Marcus D Bloice and Peter M Roth and Andreas Holzinger}, url = {https://doi.org/10.1007/s10844-021-00662-9}, doi = {10.1007/s10844-021-00662-9}, issn = {0925-9902}, year = {2021}, date = {2021-12-01}, journal = {J. Intell. Inf. Syst.}, volume = {57}, number = {3}, pages = {547–562}, publisher = {Kluwer Academic Publishers}, address = {USA}, abstract = {In this paper, a neural network is trained to perform simple arithmetic using images of concatenated handwritten digit pairs. A convolutional neural network was trained with images consisting of two side-by-side handwritten digits, where the image’s label is the summation of the two digits contained in the combined image. Crucially, the network was tested on permutation pairs that were not present during training in an effort to see if the network could learn the task of addition, as opposed to simply mapping images to labels. A dataset was generated for all possible permutation pairs of length 2 for the digits 0–9 using MNIST as a basis for the images, with one thousand samples generated for each permutation pair. For testing the network, samples generated from previously unseen permutation pairs were fed into the trained network, and its predictions measured. Results were encouraging, with the network achieving an accuracy of over 90% on some permutation train/test splits. This suggests that the network learned at first digit recognition, and subsequently the further task of addition based on the two recognised digits. As far as the authors are aware, no previous work has concentrated on learning a mathematical operation in this way. This paper is an attempt to demonstrate that a network can learn more than a direct mapping from image to label, but is learning to analyse two separate regions of an image and combining what was recognised to produce the final output label.}, keywords = {}, pubstate = {published}, tppubtype = {article} } In this paper, a neural network is trained to perform simple arithmetic using images of concatenated handwritten digit pairs. A convolutional neural network was trained with images consisting of two side-by-side handwritten digits, where the image’s label is the summation of the two digits contained in the combined image. Crucially, the network was tested on permutation pairs that were not present during training in an effort to see if the network could learn the task of addition, as opposed to simply mapping images to labels. A dataset was generated for all possible permutation pairs of length 2 for the digits 0–9 using MNIST as a basis for the images, with one thousand samples generated for each permutation pair. For testing the network, samples generated from previously unseen permutation pairs were fed into the trained network, and its predictions measured. Results were encouraging, with the network achieving an accuracy of over 90% on some permutation train/test splits. This suggests that the network learned at first digit recognition, and subsequently the further task of addition based on the two recognised digits. As far as the authors are aware, no previous work has concentrated on learning a mathematical operation in this way. This paper is an attempt to demonstrate that a network can learn more than a direct mapping from image to label, but is learning to analyse two separate regions of an image and combining what was recognised to produce the final output label. | |
Yuan, Zhenghang; Mou, Lichao; Zhu, Xiao Xiang Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data Conference IEEE International Symposium on Geoscience and Remote Sensing (IGARSS), Institute of Electrical and Electronics Engineers (IEEE), 2021, ISBN: 9781665403696. Abstract | Links | BibTeX | Tags: @conference{Yuan2021b, title = {Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data}, author = {Zhenghang Yuan and Lichao Mou and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553624}, isbn = {9781665403696}, year = {2021}, date = {2021-10-01}, booktitle = {IEEE International Symposium on Geoscience and Remote Sensing (IGARSS)}, journal = {IEEE International Symposium on Geoscience and Remote Sensing (IGARSS)}, pages = {2999-3002}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, abstract = {Answering questions with natural language by extracting information from image has great potential in various applications. Although visual question answering (VQA) for natural image has been broadly studied, VQA for remote sensing data is still in the early research stage. For the same remote sensing image, there exist questions with dramatically different difficulty-levels. Treating these questions equally may mislead the model and limit the VQA model performance. Considering this problem, in this work, we propose a self-paced curriculum learning (SPCL) based VQA model with hard and soft weighting strategies for remote sensing data. Like human learning process, the model is trained from easy to hard question samples gradually. Extensive experimental results on two datasets demonstrate that the proposed training method can achieve promising performance.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Answering questions with natural language by extracting information from image has great potential in various applications. Although visual question answering (VQA) for natural image has been broadly studied, VQA for remote sensing data is still in the early research stage. For the same remote sensing image, there exist questions with dramatically different difficulty-levels. Treating these questions equally may mislead the model and limit the VQA model performance. Considering this problem, in this work, we propose a self-paced curriculum learning (SPCL) based VQA model with hard and soft weighting strategies for remote sensing data. Like human learning process, the model is trained from easy to hard question samples gradually. Extensive experimental results on two datasets demonstrate that the proposed training method can achieve promising performance. | |
Bloice, M D; Roth, P M; Holzinger, A Performing arithmetic using a neural network trained on images of digit permutation pairs Journal Article Journal of Intelligent Information Systems, 57 , pp. 547–562, 2021. @article{Bloice2021, title = {Performing arithmetic using a neural network trained on images of digit permutation pairs}, author = {Bloice, M.D. and Roth, P.M. and Holzinger, A.}, url = {https://link.springer.com/article/10.1007/s10844-021-00662-9}, doi = {https://doi.org/10.3390/jimaging7020021 }, year = {2021}, date = {2021-08-06}, journal = {Journal of Intelligent Information Systems}, volume = {57}, pages = {547–562}, keywords = {.}, pubstate = {published}, tppubtype = {article} } | |
Perko, Roland; Almer, Alexander; Theuermann, Mario; Klopschitz, Manfred; Schnsbel, Thomas; Roth, Peter Protocol Design Issues for Object Density Estimation and Counting in Remote Sensing Book Chapter 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, IEEE, 2021. @inbook{Perko2021b, title = {Protocol Design Issues for Object Density Estimation and Counting in Remote Sensing}, author = {Roland Perko and Alexander Almer and Mario Theuermann and Manfred Klopschitz and Thomas Schnsbel and Peter Roth}, doi = {10.1109/igarss47720.2021.9553934}, year = {2021}, date = {2021-07-11}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } | |
Hua, Yuansheng; Mou, Lichao; Jin, Pu; Zhu, Xiao Unconstrained Aerial Scene Recognition with Deep Neural Networks and a New Dataset Book Chapter 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 11–16, IEEE, Brussels, Belgium, 2021. @inbook{Hua2021a, title = {Unconstrained Aerial Scene Recognition with Deep Neural Networks and a New Dataset}, author = {Yuansheng Hua and Lichao Mou and Pu Jin and Xiao Zhu}, doi = {10.1109/igarss47720.2021.9554633}, year = {2021}, date = {2021-07-11}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, pages = {11--16}, publisher = {IEEE}, address = {Brussels, Belgium}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } | |
Perwaiz, Nazia; Fraz, Muhammad; Shahzad, Muhammad Stochastic attentions and context learning for person re-identification Journal Article PeerJ Computer Science, 7 , pp. e447, 2021. @article{Perwaiz2021, title = {Stochastic attentions and context learning for person re-identification}, author = {Nazia Perwaiz and Muhammad Fraz and Muhammad Shahzad}, doi = {10.7717/peerj-cs.447}, year = {2021}, date = {2021-05-05}, journal = {PeerJ Computer Science}, volume = {7}, pages = {e447}, publisher = {PeerJ}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
 | Hua, Yuansheng; Mou, Lichao; Lin, Jianzhe; Heidler, Konrad; Zhu, Xiao Xiang Aerial Scene Understanding in The Wild: Multi-Scene Recognition via Prototype-based Memory Networks Journal Article ISPRS Journal of Photogrammetry and Remote Sensing, 2021. BibTeX | Tags: @article{hua2021prototype, title = {Aerial Scene Understanding in The Wild: Multi-Scene Recognition via Prototype-based Memory Networks}, author = {Yuansheng Hua and Lichao Mou and Jianzhe Lin and Konrad Heidler and Xiao Xiang Zhu}, year = {2021}, date = {2021-04-09}, journal = {ISPRS Journal of Photogrammetry and Remote Sensing}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Klemmer, Konstantin; Saha, Sudipan; Kahl, Matthias; Xu, Tianlin; Zhu, Xiao Xiang Generative modeling of spatio-temporal weather patterns with extreme event conditioning Inproceedings AI: Modeling Oceans and Climate Change (AIMOCC 2021) Workshop, ICLR 2021, 2021. Abstract | Links | BibTeX | Tags: @inproceedings{Klemmer2021, title = {Generative modeling of spatio-temporal weather patterns with extreme event conditioning}, author = {Konstantin Klemmer and Sudipan Saha and Matthias Kahl and Tianlin Xu and Xiao Xiang Zhu}, url = {http://arxiv.org/abs/2104.12469}, year = {2021}, date = {2021-04-01}, booktitle = {AI: Modeling Oceans and Climate Change (AIMOCC 2021) Workshop, ICLR 2021}, abstract = {Deep generative models are increasingly used to gain insights in the geospatial data domain, e.g., for climate data. However, most existing approaches work with temporal snapshots or assume 1D time-series; few are able to capture spatio-temporal processes simultaneously. Beyond this, Earth-systems data often exhibit highly irregular and complex patterns, for example caused by extreme weather events. Because of climate change, these phenomena are only increasing in frequency. Here, we proposed a novel GAN-based approach for generating spatio-temporal weather patterns conditioned on detected extreme events. Our approach augments GAN generator and discriminator with an encoded extreme weather event segmentation mask. These segmentation masks can be created from raw input using existing event detection frameworks. As such, our approach is highly modular and can be combined with custom GAN architectures. We highlight the applicability of our proposed approach in experiments with real-world surface radiation and zonal wind data.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Deep generative models are increasingly used to gain insights in the geospatial data domain, e.g., for climate data. However, most existing approaches work with temporal snapshots or assume 1D time-series; few are able to capture spatio-temporal processes simultaneously. Beyond this, Earth-systems data often exhibit highly irregular and complex patterns, for example caused by extreme weather events. Because of climate change, these phenomena are only increasing in frequency. Here, we proposed a novel GAN-based approach for generating spatio-temporal weather patterns conditioned on detected extreme events. Our approach augments GAN generator and discriminator with an encoded extreme weather event segmentation mask. These segmentation masks can be created from raw input using existing event detection frameworks. As such, our approach is highly modular and can be combined with custom GAN architectures. We highlight the applicability of our proposed approach in experiments with real-world surface radiation and zonal wind data. |
Hua, Yuansheng; Mou, Lichao; Lin, Jianzhe; Heidler, Konrad; Zhu, Xiao Xiang Aerial scene understanding in the wild: Multi-scene recognition via prototype-based memory networks Journal Article ISPRS Journal of Photogrammetry and Remote Sensing, 177 , pp. 89-102, 2021, ISSN: 0924-2716. Abstract | Links | BibTeX | Tags: @article{Hua2021d, title = {Aerial scene understanding in the wild: Multi-scene recognition via prototype-based memory networks}, author = {Yuansheng Hua and Lichao Mou and Jianzhe Lin and Konrad Heidler and Xiao Xiang Zhu}, url = {https://www.sciencedirect.com/science/article/pii/S0924271621001015}, doi = {10.1016/j.isprsjprs.2021.04.006}, issn = {0924-2716}, year = {2021}, date = {2021-04-01}, journal = {ISPRS Journal of Photogrammetry and Remote Sensing}, volume = {177}, pages = {89-102}, abstract = {A R T I C L E I N F O Keywords: Convolutional neural network (CNN) Multi-scene recognition in single images Memory network Multi-scene aerial image dataset Multi-head attention-based memory retrieval Prototype learning A B S T R A C T Aerial scene recognition is a fundamental visual task and has attracted an increasing research interest in the last few years. Most of current researches mainly deploy efforts to categorize an aerial image into one scene-level label, while in real-world scenarios, there often exist multiple scenes in a single image. Therefore, in this paper, we propose to take a step forward to a more practical and challenging task, namely multi-scene recognition in single images. Moreover, we note that manually yielding annotations for such a task is extraordinarily time-and labor-consuming. To address this, we propose a prototype-based memory network to recognize multiple scenes in a single image by leveraging massive well-annotated single-scene images. The proposed network consists of three key components: 1) a prototype learning module, 2) a prototype-inhabiting external memory, and 3) a multi-head attention-based memory retrieval module. To be more specific, we first learn the prototype representation of each aerial scene from single-scene aerial image datasets and store it in an external memory. Afterwards, a multi-head attention-based memory retrieval module is devised to retrieve scene prototypes relevant to query multi-scene images for final predictions. Notably, only a limited number of annotated multi-scene images are needed in the training phase. To facilitate the progress of aerial scene recognition, we produce a new multi-scene aerial image (MAI) dataset. Experimental results on variant dataset configurations demonstrate the effectiveness of our network. Our dataset and codes are publicly available 1 .}, keywords = {}, pubstate = {published}, tppubtype = {article} } A R T I C L E I N F O Keywords: Convolutional neural network (CNN) Multi-scene recognition in single images Memory network Multi-scene aerial image dataset Multi-head attention-based memory retrieval Prototype learning A B S T R A C T Aerial scene recognition is a fundamental visual task and has attracted an increasing research interest in the last few years. Most of current researches mainly deploy efforts to categorize an aerial image into one scene-level label, while in real-world scenarios, there often exist multiple scenes in a single image. Therefore, in this paper, we propose to take a step forward to a more practical and challenging task, namely multi-scene recognition in single images. Moreover, we note that manually yielding annotations for such a task is extraordinarily time-and labor-consuming. To address this, we propose a prototype-based memory network to recognize multiple scenes in a single image by leveraging massive well-annotated single-scene images. The proposed network consists of three key components: 1) a prototype learning module, 2) a prototype-inhabiting external memory, and 3) a multi-head attention-based memory retrieval module. To be more specific, we first learn the prototype representation of each aerial scene from single-scene aerial image datasets and store it in an external memory. Afterwards, a multi-head attention-based memory retrieval module is devised to retrieve scene prototypes relevant to query multi-scene images for final predictions. Notably, only a limited number of annotated multi-scene images are needed in the training phase. To facilitate the progress of aerial scene recognition, we produce a new multi-scene aerial image (MAI) dataset. Experimental results on variant dataset configurations demonstrate the effectiveness of our network. Our dataset and codes are publicly available 1 . | |
Perko, Roland; Klopschitz, Manfred; Almer, Alexander; Roth, Peter M Critical Aspects of Person Counting and Density Estimation Journal Article Journal of Imaging 2021, Vol. 7, Page 21, 7 (2), pp. 21, 2021, ISSN: 2313-433X. Abstract | Links | BibTeX | Tags: @article{Perko2021b, title = {Critical Aspects of Person Counting and Density Estimation}, author = {Roland Perko and Manfred Klopschitz and Alexander Almer and Peter M Roth}, url = {https://www.mdpi.com/2313-433X/7/2/21/htm https://www.mdpi.com/2313-433X/7/2/21}, doi = {10.3390/JIMAGING7020021}, issn = {2313-433X}, year = {2021}, date = {2021-01-31}, journal = {Journal of Imaging 2021, Vol. 7, Page 21}, volume = {7}, number = {2}, pages = {21}, publisher = {Multidisciplinary Digital Publishing Institute}, abstract = {Many scientific studies deal with person counting and density estimation from single images. Recently, convolutional neural networks (CNNs) have been applied for these tasks. Even though often better results are reported, it is often not clear where the improvements are resulting from, and if the proposed approaches would generalize. Thus, the main goal of this paper was to identify the critical aspects of these tasks and to show how these limit state-of-the-art approaches. Based on these findings, we show how to mitigate these limitations. To this end, we implemented a CNN-based baseline approach, which we extended to deal with identified problems. These include the discovery of bias in the reference data sets, ambiguity in ground truth generation, and mismatching of evaluation metrics w.r.t. the training loss function. The experimental results show that our modifications allow for significantly outperforming the baseline in terms of the accuracy of person counts and density estimation. In this way, we get a deeper understanding of CNN-based person density estimation beyond the network architecture. Furthermore, our insights would allow to advance the field of person density estimation in general by highlighting current limitations in the evaluation protocols.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Many scientific studies deal with person counting and density estimation from single images. Recently, convolutional neural networks (CNNs) have been applied for these tasks. Even though often better results are reported, it is often not clear where the improvements are resulting from, and if the proposed approaches would generalize. Thus, the main goal of this paper was to identify the critical aspects of these tasks and to show how these limit state-of-the-art approaches. Based on these findings, we show how to mitigate these limitations. To this end, we implemented a CNN-based baseline approach, which we extended to deal with identified problems. These include the discovery of bias in the reference data sets, ambiguity in ground truth generation, and mismatching of evaluation metrics w.r.t. the training loss function. The experimental results show that our modifications allow for significantly outperforming the baseline in terms of the accuracy of person counts and density estimation. In this way, we get a deeper understanding of CNN-based person density estimation beyond the network architecture. Furthermore, our insights would allow to advance the field of person density estimation in general by highlighting current limitations in the evaluation protocols. | |
 | Perko, Roland; Klopschitz, Manfred; Almer, Alexander; Roth, Peter M Critical Aspects of Person Couting and Density Estimation Journal Article Journal of Imaging, 7 (2), pp. 21, 2021. BibTeX | Tags: @article{perko21a, title = {Critical Aspects of Person Couting and Density Estimation}, author = {Roland Perko and Manfred Klopschitz and Alexander Almer and Peter M Roth}, year = {2021}, date = {2021-01-01}, journal = {Journal of Imaging}, volume = {7}, number = {2}, pages = {21}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Basirat, Mina; Roth, Peter M S*ReLU: Learning Piecewise Linear Activation Functions via ParticleSwarm Optimization Inproceedings International Conf. on Computer Vision Theory and Applications, 2021. BibTeX | Tags: @inproceedings{basirat21ab, title = {S*ReLU: Learning Piecewise Linear Activation Functions via ParticleSwarm Optimization}, author = {Mina Basirat and Peter M Roth}, year = {2021}, date = {2021-01-01}, booktitle = {International Conf. on Computer Vision Theory and Applications}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
 | Editors:Camps-Valls, Gustau; Tuia, Devis; Zhu, Xiao Xiang; Reichstein, Markus Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences Book Wiley & Sons, 2021, ISBN: 978-1-119-64614-3. BibTeX | Tags: @book{CampsValls21wiley, title = {Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences}, author = {Gustau Editors:Camps-Valls and Devis Tuia and Xiao Xiang Zhu and Markus Reichstein }, isbn = {978-1-119-64614-3}, year = {2021}, date = {2021-01-01}, publisher = {Wiley & Sons}, keywords = {}, pubstate = {published}, tppubtype = {book} } |
 | Mou, Lichao; Saha, Sudipan; Hua, Yuansheng; Bovolo, Francesca; Bruzzone, Lorenzo; Zhu, Xiao Xiang Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2021. BibTeX | Tags: @article{deepReinforcementTgrs2021, title = {Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification}, author = {Lichao Mou and Sudipan Saha and Yuansheng Hua and Francesca Bovolo and Lorenzo Bruzzone and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Saha, Sudipan; Banerjee, Biplab; Zhu, Xiao Xiang Trusting Small Training Dataset for Supervised Change Detection Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), IEEE, 2021. BibTeX | Tags: @conference{trustingSmallDatasetSudipanIgarss2021, title = {Trusting Small Training Dataset for Supervised Change Detection}, author = {Sudipan Saha and Biplab Banerjee and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Prexl, Jonathan; Saha, Sudipan; Zhu, Xiao Xiang Mitigating Spatial and Spectral Differences for Change Detection using Super-resolution and Unsupervised learning Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), IEEE, 2021. BibTeX | Tags: @conference{mitigatingJonathanIgarss2021, title = {Mitigating Spatial and Spectral Differences for Change Detection using Super-resolution and Unsupervised learning}, author = {Jonathan Prexl and Sudipan Saha and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Towards Out-of-distribution Detection for Remote Sensing Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), IEEE, 2021. BibTeX | Tags: @conference{towardsGawlikowskiIgarss2021, title = {Towards Out-of-distribution Detection for Remote Sensing}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Saha, Sudipan; Kondmann, Lukas; Zhu, Xiao Xiang Deep No Learning Approach for Unsupervised Change Detection in Hyperspectral Images Conference XXIV ISPRS Congress 2021, 2021. BibTeX | Tags: @conference{deepNoLearningSudipanIsprs2021, title = {Deep No Learning Approach for Unsupervised Change Detection in Hyperspectral Images}, author = {Sudipan Saha and Lukas Kondmann and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {XXIV ISPRS Congress 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Ebel, Patrick; Saha, Sudipan; Zhu, Xiao Xiang Fusing Multi-modal Data for Supervised Change Detection Conference XXIV ISPRS Congress 2021, 2021. BibTeX | Tags: @conference{fusingPatrickIsprs2021, title = {Fusing Multi-modal Data for Supervised Change Detection}, author = {Patrick Ebel and Sudipan Saha and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {XXIV ISPRS Congress 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Nandy, Jay; Saha, Sudipan; Hsu, Wynne; Lee, Mong Li; Zhu, Xiao Xiang Covariate Shift Adaptation for Adversarially Robust Classifier Conference ICLR 2021 Workshop on Security and Safety in Machine Learning Systems, 2021. BibTeX | Tags: @conference{covariateJayIclrW2021, title = {Covariate Shift Adaptation for Adversarially Robust Classifier}, author = {Jay Nandy and Sudipan Saha and Wynne Hsu and Mong Li Lee and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {ICLR 2021 Workshop on Security and Safety in Machine Learning Systems}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
| Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Out-of-distribution Detection in Satellite Image Classification Conference RobustML workshop at ICLR 2021, 2021. BibTeX | Tags: @conference{oodJakobIclrW2021, title = {Out-of-distribution Detection in Satellite Image Classification}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {RobustML workshop at ICLR 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Kochupillai, Mrinalini; Gallersdörfer, Ulrich; Köninger, Julia; Beck, Roman Incentivizing research & innovation with agrobiodiversity conserved in situ: Possibilities and limitations of a blockchain-based solution Journal Article Journal of Cleaner Production, 309 , pp. 127155, 2021, ISSN: 0959-6526. BibTeX | Tags: @article{RN2468, title = {Incentivizing research & innovation with agrobiodiversity conserved in situ: Possibilities and limitations of a blockchain-based solution}, author = {Mrinalini Kochupillai and Ulrich Gallersdörfer and Julia Köninger and Roman Beck}, issn = {0959-6526}, year = {2021}, date = {2021-01-01}, journal = {Journal of Cleaner Production}, volume = {309}, pages = {127155}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Kochupillai, M Creating a Digital Marketplace for Agrobiodiversity and Plant Genetic Sequence Data: Legal and Ethical Considerations of an AI and Blackchain Based Solution, Conference paper, Towards Responsible Plant Data Linkage: Global Challenges for Food Security and Governance, Alan Turing Institute and University of Exeter, March 2021 (forthcoming in an edited volume with Springer Nature) Conference 2021. BibTeX | Tags: @conference{RN2474b, title = {Creating a Digital Marketplace for Agrobiodiversity and Plant Genetic Sequence Data: Legal and Ethical Considerations of an AI and Blackchain Based Solution, Conference paper, Towards Responsible Plant Data Linkage: Global Challenges for Food Security and Governance, Alan Turing Institute and University of Exeter, March 2021 (forthcoming in an edited volume with Springer Nature)}, author = {M Kochupillai}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Saha, Sudipan; Ebel, Patrick; Zhu, Xiao Xiang Self-supervised multisensor change detection Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2021. BibTeX | Tags: @article{selfSupervisedMultisensorTgrs2021, title = {Self-supervised multisensor change detection}, author = {Sudipan Saha and Patrick Ebel and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Saha, Sudipan; Ahmad, Tahir Federated transfer learning: Concept and applications Journal Article Intelligenza Artificiale, 15 (1), pp. 35–44, 2021. BibTeX | Tags: @article{ftlSurvey2021, title = {Federated transfer learning: Concept and applications}, author = {Sudipan Saha and Tahir Ahmad}, year = {2021}, date = {2021-01-01}, journal = {Intelligenza Artificiale}, volume = {15}, number = {1}, pages = {35--44}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Saha, Sudipan; Zhu, Xiao Xiang Patch-level unsupervised planetary change detection Journal Article IEEE Geoscience and Remote Sensing Letters, 2021. BibTeX | Tags: @article{Saha2021, title = {Patch-level unsupervised planetary change detection}, author = {Sudipan Saha and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Saha, Sudipan; Mou, Lichao; Shahzad, Muhammad; Zhu, Xiao Xiang Segmentation of VHR EO Images using Unsupervised Learning Inproceedings 2021. BibTeX | Tags: @inproceedings{Saha2021b, title = {Segmentation of VHR EO Images using Unsupervised Learning}, author = {Sudipan Saha and Lichao Mou and Muhammad Shahzad and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {ECML PKDD 2021 workshop Machine Learning for Earth Observation (MACLEAN)}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Ahmed, Nouman; Saha, Sudipan; Mohsin, Maaz; Shahzad, Muhammad; Zhu, Xiao Xiang Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image Inproceedings 2021. BibTeX | Tags: @inproceedings{Ahmed2021, title = {Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image}, author = {Nouman Ahmed and Sudipan Saha and Maaz Mohsin and Muhammad Shahzad and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {ICCV 2021 workshop on Learning to Understand Aerial Images (LUAI)}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Saha, Sudipan; Kondmann, Lukas; Song, Qian; Zhu, Xiao Xiang Change Detection in Hyperdimensional Images using Untrained Models Journal Article IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021. BibTeX | Tags: @article{Saha2021c, title = {Change Detection in Hyperdimensional Images using Untrained Models}, author = {Sudipan Saha and Lukas Kondmann and Qian Song and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Kondmann, Lukas; Toker, Aysim; Saha, Sudipan; Schölkopf, Bernhard; Leal-Taixé, Laura; Zhu, Xiao Xiang Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images Journal Article arXiv preprint arXiv:2110.02068, 2021. BibTeX | Tags: @article{Kondmann2021, title = {Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images}, author = {Lukas Kondmann and Aysim Toker and Sudipan Saha and Bernhard Schölkopf and Laura Leal-Taixé and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {arXiv preprint arXiv:2110.02068}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Yuan, Zhenghang; Mou, Lichao; Zhu, Xiao Xiang Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data Conference IEEE International Symposium on Geoscience and Remote Sensing (IGARSS) , Institute of Electrical and Electronics Engineers (IEEE), 2021, ISBN: 9781665403696. Abstract | Links | BibTeX | Tags: @conference{Yuan2021, title = {Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data}, author = {Zhenghang Yuan and Lichao Mou and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553624}, isbn = {9781665403696}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE International Symposium on Geoscience and Remote Sensing (IGARSS) }, journal = {IEEE International Symposium on Geoscience and Remote Sensing (IGARSS) }, pages = {2999-3002}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, abstract = {Answering questions with natural language by extracting information from image has great potential in various applications. Although visual question answering (VQA) for natural image has been broadly studied, VQA for remote sensing data is still in the early research stage. For the same remote sensing image, there exist questions with dramatically different difficulty-levels. Treating these questions equally may mislead the model and limit the VQA model performance. Considering this problem, in this work, we propose a self-paced curriculum learning (SPCL) based VQA model with hard and soft weighting strategies for remote sensing data. Like human learning process, the model is trained from easy to hard question samples gradually. Extensive experimental results on two datasets demonstrate that the proposed training method can achieve promising performance.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Answering questions with natural language by extracting information from image has great potential in various applications. Although visual question answering (VQA) for natural image has been broadly studied, VQA for remote sensing data is still in the early research stage. For the same remote sensing image, there exist questions with dramatically different difficulty-levels. Treating these questions equally may mislead the model and limit the VQA model performance. Considering this problem, in this work, we propose a self-paced curriculum learning (SPCL) based VQA model with hard and soft weighting strategies for remote sensing data. Like human learning process, the model is trained from easy to hard question samples gradually. Extensive experimental results on two datasets demonstrate that the proposed training method can achieve promising performance. | |
Köninger, Julia; Lugato, Emanuele; Panagos, Panos; Kochupillai, Mrinalini; Orgiazzi, Alberto; Briones, Maria J I Manure management and soil biodiversity: Towards more sustainable food systems in the EU Journal Article Agricultural Systems, 194 , 2021, ISSN: 0308521X. Abstract | Links | BibTeX | Tags: @article{, title = {Manure management and soil biodiversity: Towards more sustainable food systems in the EU}, author = {Julia Köninger and Emanuele Lugato and Panos Panagos and Mrinalini Kochupillai and Alberto Orgiazzi and Maria J I Briones}, doi = {10.1016/J.AGSY.2021.103251}, issn = {0308521X}, year = {2021}, date = {2021-01-01}, journal = {Agricultural Systems}, volume = {194}, publisher = {Elsevier Ltd}, abstract = {CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals.}, keywords = {}, pubstate = {published}, tppubtype = {article} } CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals. | |
Vishwakarma, Bramha Dutt; Horwath, Martin; Groh, Andreas; Bamber, Jonathan L Accounting for GIA signal in GRACE products Journal Article Geophysical Journal International, 228 , pp. 2056-2060, 2021, ISSN: 0956-540X. Abstract | Links | BibTeX | Tags: @article{Vishwakarma2021, title = {Accounting for GIA signal in GRACE products}, author = {Bramha Dutt Vishwakarma and Martin Horwath and Andreas Groh and Jonathan L Bamber}, url = {https://academic.oup.com/gji/article/228/3/2056/6426181}, doi = {10.1093/GJI/GGAB464}, issn = {0956-540X}, year = {2021}, date = {2021-01-01}, journal = {Geophysical Journal International}, volume = {228}, pages = {2056-2060}, publisher = {Oxford Academic}, abstract = {The Gravity Recovery and Climate Experiment (GRACE) observes gravitational potential anomalies that include the effects of present-day surface mass change (PDSMC)-and glacial isostatic adjustment (GIA)-driven solid Earth mass redistribution. Therefore, GIA estimates from a forward model are commonly removed from GRACE to estimate PDSMC. There are several GIA models and to facilitate users in using a GIA model of their choice, both GRACE and GIA products are made available in terms of global gridded fields representing mass anomaly. GRACE-observed gravitational potential anomalies are represented in terms of equivalent water height (EWH) with a relation that accounts for an elastic solid Earth deformation due to PDSMC. However, for obtaining GIA EWH fields from GIA gravitational potential fields, two relations are being used: one that is similar to that being used for GRACE EWH and the other that does not include an elastic deformation effect. This leaves users with the possibility of obtaining different values for PDSMC with a given GRACE and GIA field. In this paper, we discuss the impact of this problem on regional mass change estimates and highlight the need for consistent treatment of GIA signals in GRACE observations.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The Gravity Recovery and Climate Experiment (GRACE) observes gravitational potential anomalies that include the effects of present-day surface mass change (PDSMC)-and glacial isostatic adjustment (GIA)-driven solid Earth mass redistribution. Therefore, GIA estimates from a forward model are commonly removed from GRACE to estimate PDSMC. There are several GIA models and to facilitate users in using a GIA model of their choice, both GRACE and GIA products are made available in terms of global gridded fields representing mass anomaly. GRACE-observed gravitational potential anomalies are represented in terms of equivalent water height (EWH) with a relation that accounts for an elastic solid Earth deformation due to PDSMC. However, for obtaining GIA EWH fields from GIA gravitational potential fields, two relations are being used: one that is similar to that being used for GRACE EWH and the other that does not include an elastic deformation effect. This leaves users with the possibility of obtaining different values for PDSMC with a given GRACE and GIA field. In this paper, we discuss the impact of this problem on regional mass change estimates and highlight the need for consistent treatment of GIA signals in GRACE observations. | |
Basirat, Mina; Roth, Peter M S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization Inproceedings Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, SCITEPRESS - Science and Technology Publications, 2021, ISBN: 9789897584886. Abstract | Links | BibTeX | Tags: @inproceedings{Basirat_2021, title = {S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization}, author = {Mina Basirat and Peter M Roth}, url = {https://orcid.org/0000-0001-9566-1298}, doi = {10.5220/0010338506450652}, isbn = {9789897584886}, year = {2021}, date = {2021-01-01}, booktitle = {Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications}, publisher = {SCITEPRESS - Science and Technology Publications}, abstract = {Recently, it was shown that using a properly parametrized Leaky ReLU (LReLU) as activation function yields significantly better results for a variety of image classification tasks. However, such methods are not feasible in practice. Either the only parameter (i.e., the slope of the negative part) needs to be set manually (L*ReLU), or the approach is vulnerable due to the gradient-based optimization and, thus, highly dependent on a proper initialization (PReLU). In this paper, we exploit the benefits of piecewise linear functions, avoiding these problems. To this end, we propose a fully automatic approach to estimate the slope parameter for LReLU from the data. We realize this via Stochastic Optimization, namely Particle Swarm Optimization (PSO): S*ReLU. In this way, we can show that, compared to widely-used activation functions (including PReLU), we can obtain better results on seven different benchmark datasets, however, also drastically reducing the computational effort.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Recently, it was shown that using a properly parametrized Leaky ReLU (LReLU) as activation function yields significantly better results for a variety of image classification tasks. However, such methods are not feasible in practice. Either the only parameter (i.e., the slope of the negative part) needs to be set manually (L*ReLU), or the approach is vulnerable due to the gradient-based optimization and, thus, highly dependent on a proper initialization (PReLU). In this paper, we exploit the benefits of piecewise linear functions, avoiding these problems. To this end, we propose a fully automatic approach to estimate the slope parameter for LReLU from the data. We realize this via Stochastic Optimization, namely Particle Swarm Optimization (PSO): S*ReLU. In this way, we can show that, compared to widely-used activation functions (including PReLU), we can obtain better results on seven different benchmark datasets, however, also drastically reducing the computational effort. | |
Musa, Osman; Jung, Peter; Caire, Giuseppe Plug-And-Play Learned Gaussian-mixture Approximate Message Passing Inproceedings pp. 4855-4859, IEEE, 2021, ISBN: 978-1-7281-7605-5. @inproceedings{Musa2021, title = {Plug-And-Play Learned Gaussian-mixture Approximate Message Passing}, author = {Osman Musa and Peter Jung and Giuseppe Caire}, doi = {10.1109/ICASSP39728.2021.9414910}, isbn = {978-1-7281-7605-5}, year = {2021}, date = {2021-01-01}, journal = {ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pages = {4855-4859}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Sauder, Jonathan; Genzel, Martin; Jung, Peter Learning Structured Sparse Matrices for Signal Recovery via Unrolled Optimization | OpenReview Inproceedings 2021. @inproceedings{Sauder2021, title = {Learning Structured Sparse Matrices for Signal Recovery via Unrolled Optimization | OpenReview}, author = {Jonathan Sauder and Martin Genzel and Peter Jung}, url = {https://openreview.net/forum?id=IxKSqOq1TKQ}, year = {2021}, date = {2021-01-01}, journal = {NeurIPS 2021 Workshop on Deep Learning and Inverse Problems}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Sánchez-Pastor, Jesús; Thanthrige, Udaya Miriya S K P; Ilgac, Furkan; Jiménez-Sáez, Alejandro; Jung, Peter; Sezgin, Aydin; Jakoby, Rolf Clutter Suppression for Indoor Self-Localization Systems by Iteratively Reweighted Low-Rank Plus Sparse Recovery Journal Article Sensors 2021, Vol. 21, Page 6842, 21 , pp. 6842, 2021, ISSN: 1424-8220. Abstract | Links | BibTeX | Tags: @article{SanchezPastor2021, title = {Clutter Suppression for Indoor Self-Localization Systems by Iteratively Reweighted Low-Rank Plus Sparse Recovery}, author = {Jesús Sánchez-Pastor and Udaya Miriya S K P Thanthrige and Furkan Ilgac and Alejandro Jiménez-Sáez and Peter Jung and Aydin Sezgin and Rolf Jakoby}, url = {https://www.mdpi.com/1424-8220/21/20/6842/htm https://www.mdpi.com/1424-8220/21/20/6842}, doi = {10.3390/S21206842}, issn = {1424-8220}, year = {2021}, date = {2021-01-01}, journal = {Sensors 2021, Vol. 21, Page 6842}, volume = {21}, pages = {6842}, publisher = {Multidisciplinary Digital Publishing Institute}, abstract = {Self-localization based on passive RFID-based has many potential applications. One of the main challenges it faces is the suppression of the reflected signals from unwanted objects (i.e., clutter). Typically, the clutter echoes are much stronger than the backscattered signals of the passive tag landmarks used in such scenarios. Therefore, successful tag detection can be very challenging. We consider two types of tags, namely low-Q and high-Q tags. The high-Q tag features a sparse frequency response, whereas the low-Q tag presents a broad frequency response. Further, the clutter usually showcases a short-lived response. In this work, we propose an iterative algorithm based on a low-rank plus sparse recovery approach (RPCA) to mitigate clutter and retrieve the landmark response. In addition to that, we compare the proposed approach with the well-known time-gating technique. It turns out that RPCA outperforms significantly time-gating for low-Q tags, achieving clutter suppression and tag identification when clutter encroaches on the time-gating window span, whereas it also increases the backscattered power at resonance by approximately 12 dB at 80 cm for high-Q tags. Altogether, RPCA seems a promising approach to improve the identification of passive indoor self-localization tag landmarks.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Self-localization based on passive RFID-based has many potential applications. One of the main challenges it faces is the suppression of the reflected signals from unwanted objects (i.e., clutter). Typically, the clutter echoes are much stronger than the backscattered signals of the passive tag landmarks used in such scenarios. Therefore, successful tag detection can be very challenging. We consider two types of tags, namely low-Q and high-Q tags. The high-Q tag features a sparse frequency response, whereas the low-Q tag presents a broad frequency response. Further, the clutter usually showcases a short-lived response. In this work, we propose an iterative algorithm based on a low-rank plus sparse recovery approach (RPCA) to mitigate clutter and retrieve the landmark response. In addition to that, we compare the proposed approach with the well-known time-gating technique. It turns out that RPCA outperforms significantly time-gating for low-Q tags, achieving clutter suppression and tag identification when clutter encroaches on the time-gating window span, whereas it also increases the backscattered power at resonance by approximately 12 dB at 80 cm for high-Q tags. Altogether, RPCA seems a promising approach to improve the identification of passive indoor self-localization tag landmarks. | |
Damara, Muhammad Fadli; Kornhardt, Gregor; Jung, Peter Solving Inverse Problems with Conditional-GAN Prior via Fast Network-Projected Gradient Descent Journal Article 2021. Abstract | Links | BibTeX | Tags: @article{Damara2021, title = {Solving Inverse Problems with Conditional-GAN Prior via Fast Network-Projected Gradient Descent}, author = {Muhammad Fadli Damara and Gregor Kornhardt and Peter Jung}, url = {https://arxiv.org/abs/2109.01105v1}, doi = {10.48550/arxiv.2109.01105}, year = {2021}, date = {2021-01-01}, abstract = {The projected gradient descent (PGD) method has shown to be effective in recovering compressed signals described in a data-driven way by a generative model, i.e., a generator which has learned the data distribution. Further reconstruction improvements for such inverse problems can be achieved by conditioning the generator on the measurement. The boundary equilibrium generative adversarial network (BEGAN) implements an equilibrium based loss function and an auto-encoding discriminator to better balance the performance of the generator and the discriminator. In this work we investigate a network-based projected gradient descent (NPGD) algorithm for measurement-conditional generative models to solve the inverse problem much faster than regular PGD. We combine the NPGD with conditional GAN/BEGAN to evaluate their effectiveness in solving compressed sensing type problems. Our experiments on the MNIST and CelebA datasets show that the combination of measurement conditional model with NPGD works well in recovering the compressed signal while achieving similar or in some cases even better performance along with a much faster reconstruction. The achieved reconstruction speed-up in our experiments is up to 140-175.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The projected gradient descent (PGD) method has shown to be effective in recovering compressed signals described in a data-driven way by a generative model, i.e., a generator which has learned the data distribution. Further reconstruction improvements for such inverse problems can be achieved by conditioning the generator on the measurement. The boundary equilibrium generative adversarial network (BEGAN) implements an equilibrium based loss function and an auto-encoding discriminator to better balance the performance of the generator and the discriminator. In this work we investigate a network-based projected gradient descent (NPGD) algorithm for measurement-conditional generative models to solve the inverse problem much faster than regular PGD. We combine the NPGD with conditional GAN/BEGAN to evaluate their effectiveness in solving compressed sensing type problems. Our experiments on the MNIST and CelebA datasets show that the combination of measurement conditional model with NPGD works well in recovering the compressed signal while achieving similar or in some cases even better performance along with a much faster reconstruction. The achieved reconstruction speed-up in our experiments is up to 140-175. | |
Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Out-of-distribution Detection in Satellite Image Classification Conference RobustML workshop at ICLR 2021, 2021. BibTeX | Tags: @conference{Gawlikowski2021b, title = {Out-of-distribution Detection in Satellite Image Classification}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {RobustML workshop at ICLR 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Saha, Sudipan; Zhu, Xiao Xiang Patch-level unsupervised planetary change detection Journal Article IEEE Geoscience and Remote Sensing Letters, 19 , 2021, ISSN: 15580571. Abstract | Links | BibTeX | Tags: @article{Saha2021d, title = {Patch-level unsupervised planetary change detection}, author = {Sudipan Saha and Xiao Xiang Zhu}, doi = {10.1109/LGRS.2021.3130862}, issn = {15580571}, year = {2021}, date = {2021-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, volume = {19}, publisher = {IEEE}, abstract = {Change detection (CD) is critical for analyzing data collected by planetary exploration missions, e.g., for identification of new impact craters. However, CD is still a relatively new topic in the context of planetary exploration. Sheer variation of planetary data makes CD much more challenging than in the case of Earth observation (EO). Unlike CD for EO, patch-level decision is preferred in planetary exploration as it is difficult to obtain perfect pixelwise alignment/coregistration between the bi-temporal planetary images. Lack of labeled bi-temporal data impedes supervised CD. To overcome these challenges, we propose an unsupervised CD method that exploits a pretrained feature extractor to obtain bi-temporal deep features that are further processed using global max-pooling to obtain patch-level feature description. Bi-temporal patch-level features are further analyzed based on difference to determine whether a patch is changed. Additionally, a self-supervised method is proposed to estimate the decision boundary between the changed and unchanged patches. Experimental results on three planetary CD datasets from two different planetary bodies (Mars and Moon) demonstrate that the proposed method often outperforms supervised planetary CD methods. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/planetaryCDUnsup.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Change detection (CD) is critical for analyzing data collected by planetary exploration missions, e.g., for identification of new impact craters. However, CD is still a relatively new topic in the context of planetary exploration. Sheer variation of planetary data makes CD much more challenging than in the case of Earth observation (EO). Unlike CD for EO, patch-level decision is preferred in planetary exploration as it is difficult to obtain perfect pixelwise alignment/coregistration between the bi-temporal planetary images. Lack of labeled bi-temporal data impedes supervised CD. To overcome these challenges, we propose an unsupervised CD method that exploits a pretrained feature extractor to obtain bi-temporal deep features that are further processed using global max-pooling to obtain patch-level feature description. Bi-temporal patch-level features are further analyzed based on difference to determine whether a patch is changed. Additionally, a self-supervised method is proposed to estimate the decision boundary between the changed and unchanged patches. Experimental results on three planetary CD datasets from two different planetary bodies (Mars and Moon) demonstrate that the proposed method often outperforms supervised planetary CD methods. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/planetaryCDUnsup. | |
Saha, Sudipan; Kondmann, Lukas; Song, Qian; Zhu, Xiao Xiang Change Detection in Hyperdimensional Images using Untrained Models Journal Article IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14 , pp. 11029, 2021. Abstract | Links | BibTeX | Tags: @article{Saha2021e, title = {Change Detection in Hyperdimensional Images using Untrained Models}, author = {Sudipan Saha and Lukas Kondmann and Qian Song and Xiao Xiang Zhu}, doi = {10.1109/JSTARS.2021.3121556}, year = {2021}, date = {2021-01-01}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, volume = {14}, pages = {11029}, publisher = {IEEE}, abstract = {Deep transfer-learning-based change detection methods are dependent on the availability of sensor-specific pretrained feature extractors. Such feature extractors are not always available due to lack of training data, especially for hyperspectral sensors and other hyperdimensional images. Moreover models trained on easily available multispectral (RGB/RGB-NIR) images cannot be reused on such hyperdimensional images due to their irregular number of bands. While hyperdimensional images show large number of spectral bands, they generally show much less spatial complexity, thus reducing the requirement of large receptive fields of convo-lution filters. Recent works in the computer vision have shown that even untrained deep models can yield remarkable result in some tasks like super-resolution and surface reconstruction. This motivates us to make a bold proposition that untrained lightweight deep model, initialized with some weight initialization strategy, can be used to extract useful semantic features from bi-temporal hyperdimensional images. Based on this proposition, we design a novel change detection framework for hyperdimensional images by extracting bitemporal features using an untrained model and further comparing the extracted features using deep change vector analysis to distinguish changed pixels from the unchanged ones. We further use the deep change hypervectors to cluster the changed pixels into different semantic groups. We conduct experiments on four change detection datasets: three hyperspectral datasets and a hyperdimensional polarimetric synthetic aperture radar dataset. The results clearly demonstrate that the proposed method is suitable for change detection in hyperdimensional remote sensing data.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Deep transfer-learning-based change detection methods are dependent on the availability of sensor-specific pretrained feature extractors. Such feature extractors are not always available due to lack of training data, especially for hyperspectral sensors and other hyperdimensional images. Moreover models trained on easily available multispectral (RGB/RGB-NIR) images cannot be reused on such hyperdimensional images due to their irregular number of bands. While hyperdimensional images show large number of spectral bands, they generally show much less spatial complexity, thus reducing the requirement of large receptive fields of convo-lution filters. Recent works in the computer vision have shown that even untrained deep models can yield remarkable result in some tasks like super-resolution and surface reconstruction. This motivates us to make a bold proposition that untrained lightweight deep model, initialized with some weight initialization strategy, can be used to extract useful semantic features from bi-temporal hyperdimensional images. Based on this proposition, we design a novel change detection framework for hyperdimensional images by extracting bitemporal features using an untrained model and further comparing the extracted features using deep change vector analysis to distinguish changed pixels from the unchanged ones. We further use the deep change hypervectors to cluster the changed pixels into different semantic groups. We conduct experiments on four change detection datasets: three hyperspectral datasets and a hyperdimensional polarimetric synthetic aperture radar dataset. The results clearly demonstrate that the proposed method is suitable for change detection in hyperdimensional remote sensing data. |