Our project is done in close collaboration with the Technical University of Munich. In particular with the TUM Data Science in Earth Observation (Sipeo) group. The complete list of associated publications might be also interesting for you and is available here.
Publications
Publications
2021 |
|
Hua, Yuansheng; Mou, Lichao; Lin, Jianzhe; Heidler, Konrad; Zhu, Xiao Xiang Aerial scene understanding in the wild: Multi-scene recognition via prototype-based memory networks Journal Article ISPRS Journal of Photogrammetry and Remote Sensing, 177 , pp. 89-102, 2021, ISSN: 0924-2716. Abstract | Links | BibTeX | Tags: @article{Hua2021d, title = {Aerial scene understanding in the wild: Multi-scene recognition via prototype-based memory networks}, author = {Yuansheng Hua and Lichao Mou and Jianzhe Lin and Konrad Heidler and Xiao Xiang Zhu}, url = {https://www.sciencedirect.com/science/article/pii/S0924271621001015}, doi = {10.1016/j.isprsjprs.2021.04.006}, issn = {0924-2716}, year = {2021}, date = {2021-04-01}, journal = {ISPRS Journal of Photogrammetry and Remote Sensing}, volume = {177}, pages = {89-102}, abstract = {A R T I C L E I N F O Keywords: Convolutional neural network (CNN) Multi-scene recognition in single images Memory network Multi-scene aerial image dataset Multi-head attention-based memory retrieval Prototype learning A B S T R A C T Aerial scene recognition is a fundamental visual task and has attracted an increasing research interest in the last few years. Most of current researches mainly deploy efforts to categorize an aerial image into one scene-level label, while in real-world scenarios, there often exist multiple scenes in a single image. Therefore, in this paper, we propose to take a step forward to a more practical and challenging task, namely multi-scene recognition in single images. Moreover, we note that manually yielding annotations for such a task is extraordinarily time-and labor-consuming. To address this, we propose a prototype-based memory network to recognize multiple scenes in a single image by leveraging massive well-annotated single-scene images. The proposed network consists of three key components: 1) a prototype learning module, 2) a prototype-inhabiting external memory, and 3) a multi-head attention-based memory retrieval module. To be more specific, we first learn the prototype representation of each aerial scene from single-scene aerial image datasets and store it in an external memory. Afterwards, a multi-head attention-based memory retrieval module is devised to retrieve scene prototypes relevant to query multi-scene images for final predictions. Notably, only a limited number of annotated multi-scene images are needed in the training phase. To facilitate the progress of aerial scene recognition, we produce a new multi-scene aerial image (MAI) dataset. Experimental results on variant dataset configurations demonstrate the effectiveness of our network. Our dataset and codes are publicly available 1 .}, keywords = {}, pubstate = {published}, tppubtype = {article} } A R T I C L E I N F O Keywords: Convolutional neural network (CNN) Multi-scene recognition in single images Memory network Multi-scene aerial image dataset Multi-head attention-based memory retrieval Prototype learning A B S T R A C T Aerial scene recognition is a fundamental visual task and has attracted an increasing research interest in the last few years. Most of current researches mainly deploy efforts to categorize an aerial image into one scene-level label, while in real-world scenarios, there often exist multiple scenes in a single image. Therefore, in this paper, we propose to take a step forward to a more practical and challenging task, namely multi-scene recognition in single images. Moreover, we note that manually yielding annotations for such a task is extraordinarily time-and labor-consuming. To address this, we propose a prototype-based memory network to recognize multiple scenes in a single image by leveraging massive well-annotated single-scene images. The proposed network consists of three key components: 1) a prototype learning module, 2) a prototype-inhabiting external memory, and 3) a multi-head attention-based memory retrieval module. To be more specific, we first learn the prototype representation of each aerial scene from single-scene aerial image datasets and store it in an external memory. Afterwards, a multi-head attention-based memory retrieval module is devised to retrieve scene prototypes relevant to query multi-scene images for final predictions. Notably, only a limited number of annotated multi-scene images are needed in the training phase. To facilitate the progress of aerial scene recognition, we produce a new multi-scene aerial image (MAI) dataset. Experimental results on variant dataset configurations demonstrate the effectiveness of our network. Our dataset and codes are publicly available 1 . | |
Perko, Roland; Klopschitz, Manfred; Almer, Alexander; Roth, Peter M Critical Aspects of Person Counting and Density Estimation Journal Article Journal of Imaging 2021, Vol. 7, Page 21, 7 (2), pp. 21, 2021, ISSN: 2313-433X. Abstract | Links | BibTeX | Tags: @article{Perko2021b, title = {Critical Aspects of Person Counting and Density Estimation}, author = {Roland Perko and Manfred Klopschitz and Alexander Almer and Peter M Roth}, url = {https://www.mdpi.com/2313-433X/7/2/21/htm https://www.mdpi.com/2313-433X/7/2/21}, doi = {10.3390/JIMAGING7020021}, issn = {2313-433X}, year = {2021}, date = {2021-01-31}, journal = {Journal of Imaging 2021, Vol. 7, Page 21}, volume = {7}, number = {2}, pages = {21}, publisher = {Multidisciplinary Digital Publishing Institute}, abstract = {Many scientific studies deal with person counting and density estimation from single images. Recently, convolutional neural networks (CNNs) have been applied for these tasks. Even though often better results are reported, it is often not clear where the improvements are resulting from, and if the proposed approaches would generalize. Thus, the main goal of this paper was to identify the critical aspects of these tasks and to show how these limit state-of-the-art approaches. Based on these findings, we show how to mitigate these limitations. To this end, we implemented a CNN-based baseline approach, which we extended to deal with identified problems. These include the discovery of bias in the reference data sets, ambiguity in ground truth generation, and mismatching of evaluation metrics w.r.t. the training loss function. The experimental results show that our modifications allow for significantly outperforming the baseline in terms of the accuracy of person counts and density estimation. In this way, we get a deeper understanding of CNN-based person density estimation beyond the network architecture. Furthermore, our insights would allow to advance the field of person density estimation in general by highlighting current limitations in the evaluation protocols.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Many scientific studies deal with person counting and density estimation from single images. Recently, convolutional neural networks (CNNs) have been applied for these tasks. Even though often better results are reported, it is often not clear where the improvements are resulting from, and if the proposed approaches would generalize. Thus, the main goal of this paper was to identify the critical aspects of these tasks and to show how these limit state-of-the-art approaches. Based on these findings, we show how to mitigate these limitations. To this end, we implemented a CNN-based baseline approach, which we extended to deal with identified problems. These include the discovery of bias in the reference data sets, ambiguity in ground truth generation, and mismatching of evaluation metrics w.r.t. the training loss function. The experimental results show that our modifications allow for significantly outperforming the baseline in terms of the accuracy of person counts and density estimation. In this way, we get a deeper understanding of CNN-based person density estimation beyond the network architecture. Furthermore, our insights would allow to advance the field of person density estimation in general by highlighting current limitations in the evaluation protocols. | |
 | Perko, Roland; Klopschitz, Manfred; Almer, Alexander; Roth, Peter M Critical Aspects of Person Couting and Density Estimation Journal Article Journal of Imaging, 7 (2), pp. 21, 2021. BibTeX | Tags: @article{perko21a, title = {Critical Aspects of Person Couting and Density Estimation}, author = {Roland Perko and Manfred Klopschitz and Alexander Almer and Peter M Roth}, year = {2021}, date = {2021-01-01}, journal = {Journal of Imaging}, volume = {7}, number = {2}, pages = {21}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Basirat, Mina; Roth, Peter M S*ReLU: Learning Piecewise Linear Activation Functions via ParticleSwarm Optimization Inproceedings International Conf. on Computer Vision Theory and Applications, 2021. BibTeX | Tags: @inproceedings{basirat21ab, title = {S*ReLU: Learning Piecewise Linear Activation Functions via ParticleSwarm Optimization}, author = {Mina Basirat and Peter M Roth}, year = {2021}, date = {2021-01-01}, booktitle = {International Conf. on Computer Vision Theory and Applications}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
 | Editors:Camps-Valls, Gustau; Tuia, Devis; Zhu, Xiao Xiang; Reichstein, Markus Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences Book Wiley & Sons, 2021, ISBN: 978-1-119-64614-3. BibTeX | Tags: @book{CampsValls21wiley, title = {Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences}, author = {Gustau Editors:Camps-Valls and Devis Tuia and Xiao Xiang Zhu and Markus Reichstein }, isbn = {978-1-119-64614-3}, year = {2021}, date = {2021-01-01}, publisher = {Wiley & Sons}, keywords = {}, pubstate = {published}, tppubtype = {book} } |
 | Mou, Lichao; Saha, Sudipan; Hua, Yuansheng; Bovolo, Francesca; Bruzzone, Lorenzo; Zhu, Xiao Xiang Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2021. BibTeX | Tags: @article{deepReinforcementTgrs2021, title = {Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification}, author = {Lichao Mou and Sudipan Saha and Yuansheng Hua and Francesca Bovolo and Lorenzo Bruzzone and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Saha, Sudipan; Banerjee, Biplab; Zhu, Xiao Xiang Trusting Small Training Dataset for Supervised Change Detection Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), IEEE, 2021. BibTeX | Tags: @conference{trustingSmallDatasetSudipanIgarss2021, title = {Trusting Small Training Dataset for Supervised Change Detection}, author = {Sudipan Saha and Biplab Banerjee and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Prexl, Jonathan; Saha, Sudipan; Zhu, Xiao Xiang Mitigating Spatial and Spectral Differences for Change Detection using Super-resolution and Unsupervised learning Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), IEEE, 2021. BibTeX | Tags: @conference{mitigatingJonathanIgarss2021, title = {Mitigating Spatial and Spectral Differences for Change Detection using Super-resolution and Unsupervised learning}, author = {Jonathan Prexl and Sudipan Saha and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Towards Out-of-distribution Detection for Remote Sensing Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), IEEE, 2021. BibTeX | Tags: @conference{towardsGawlikowskiIgarss2021, title = {Towards Out-of-distribution Detection for Remote Sensing}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Saha, Sudipan; Kondmann, Lukas; Zhu, Xiao Xiang Deep No Learning Approach for Unsupervised Change Detection in Hyperspectral Images Conference XXIV ISPRS Congress 2021, 2021. BibTeX | Tags: @conference{deepNoLearningSudipanIsprs2021, title = {Deep No Learning Approach for Unsupervised Change Detection in Hyperspectral Images}, author = {Sudipan Saha and Lukas Kondmann and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {XXIV ISPRS Congress 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Ebel, Patrick; Saha, Sudipan; Zhu, Xiao Xiang Fusing Multi-modal Data for Supervised Change Detection Conference XXIV ISPRS Congress 2021, 2021. BibTeX | Tags: @conference{fusingPatrickIsprs2021, title = {Fusing Multi-modal Data for Supervised Change Detection}, author = {Patrick Ebel and Sudipan Saha and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {XXIV ISPRS Congress 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Nandy, Jay; Saha, Sudipan; Hsu, Wynne; Lee, Mong Li; Zhu, Xiao Xiang Covariate Shift Adaptation for Adversarially Robust Classifier Conference ICLR 2021 Workshop on Security and Safety in Machine Learning Systems, 2021. BibTeX | Tags: @conference{covariateJayIclrW2021, title = {Covariate Shift Adaptation for Adversarially Robust Classifier}, author = {Jay Nandy and Sudipan Saha and Wynne Hsu and Mong Li Lee and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {ICLR 2021 Workshop on Security and Safety in Machine Learning Systems}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
| Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Out-of-distribution Detection in Satellite Image Classification Conference RobustML workshop at ICLR 2021, 2021. BibTeX | Tags: @conference{oodJakobIclrW2021, title = {Out-of-distribution Detection in Satellite Image Classification}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {RobustML workshop at ICLR 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Kochupillai, Mrinalini; Gallersdörfer, Ulrich; Köninger, Julia; Beck, Roman Incentivizing research & innovation with agrobiodiversity conserved in situ: Possibilities and limitations of a blockchain-based solution Journal Article Journal of Cleaner Production, 309 , pp. 127155, 2021, ISSN: 0959-6526. BibTeX | Tags: @article{RN2468, title = {Incentivizing research & innovation with agrobiodiversity conserved in situ: Possibilities and limitations of a blockchain-based solution}, author = {Mrinalini Kochupillai and Ulrich Gallersdörfer and Julia Köninger and Roman Beck}, issn = {0959-6526}, year = {2021}, date = {2021-01-01}, journal = {Journal of Cleaner Production}, volume = {309}, pages = {127155}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Kochupillai, M Creating a Digital Marketplace for Agrobiodiversity and Plant Genetic Sequence Data: Legal and Ethical Considerations of an AI and Blackchain Based Solution, Conference paper, Towards Responsible Plant Data Linkage: Global Challenges for Food Security and Governance, Alan Turing Institute and University of Exeter, March 2021 (forthcoming in an edited volume with Springer Nature) Conference 2021. BibTeX | Tags: @conference{RN2474b, title = {Creating a Digital Marketplace for Agrobiodiversity and Plant Genetic Sequence Data: Legal and Ethical Considerations of an AI and Blackchain Based Solution, Conference paper, Towards Responsible Plant Data Linkage: Global Challenges for Food Security and Governance, Alan Turing Institute and University of Exeter, March 2021 (forthcoming in an edited volume with Springer Nature)}, author = {M Kochupillai}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
 | Saha, Sudipan; Ebel, Patrick; Zhu, Xiao Xiang Self-supervised multisensor change detection Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2021. BibTeX | Tags: @article{selfSupervisedMultisensorTgrs2021, title = {Self-supervised multisensor change detection}, author = {Sudipan Saha and Patrick Ebel and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
 | Saha, Sudipan; Ahmad, Tahir Federated transfer learning: Concept and applications Journal Article Intelligenza Artificiale, 15 (1), pp. 35–44, 2021. BibTeX | Tags: @article{ftlSurvey2021, title = {Federated transfer learning: Concept and applications}, author = {Sudipan Saha and Tahir Ahmad}, year = {2021}, date = {2021-01-01}, journal = {Intelligenza Artificiale}, volume = {15}, number = {1}, pages = {35--44}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Saha, Sudipan; Zhu, Xiao Xiang Patch-level unsupervised planetary change detection Journal Article IEEE Geoscience and Remote Sensing Letters, 2021. BibTeX | Tags: @article{Saha2021, title = {Patch-level unsupervised planetary change detection}, author = {Sudipan Saha and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Saha, Sudipan; Mou, Lichao; Shahzad, Muhammad; Zhu, Xiao Xiang Segmentation of VHR EO Images using Unsupervised Learning Inproceedings 2021. BibTeX | Tags: @inproceedings{Saha2021b, title = {Segmentation of VHR EO Images using Unsupervised Learning}, author = {Sudipan Saha and Lichao Mou and Muhammad Shahzad and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {ECML PKDD 2021 workshop Machine Learning for Earth Observation (MACLEAN)}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Ahmed, Nouman; Saha, Sudipan; Mohsin, Maaz; Shahzad, Muhammad; Zhu, Xiao Xiang Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image Inproceedings 2021. BibTeX | Tags: @inproceedings{Ahmed2021, title = {Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image}, author = {Nouman Ahmed and Sudipan Saha and Maaz Mohsin and Muhammad Shahzad and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {ICCV 2021 workshop on Learning to Understand Aerial Images (LUAI)}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Saha, Sudipan; Kondmann, Lukas; Song, Qian; Zhu, Xiao Xiang Change Detection in Hyperdimensional Images using Untrained Models Journal Article IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021. BibTeX | Tags: @article{Saha2021c, title = {Change Detection in Hyperdimensional Images using Untrained Models}, author = {Sudipan Saha and Lukas Kondmann and Qian Song and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Kondmann, Lukas; Toker, Aysim; Saha, Sudipan; Schölkopf, Bernhard; Leal-Taixé, Laura; Zhu, Xiao Xiang Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images Journal Article arXiv preprint arXiv:2110.02068, 2021. BibTeX | Tags: @article{Kondmann2021, title = {Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images}, author = {Lukas Kondmann and Aysim Toker and Sudipan Saha and Bernhard Schölkopf and Laura Leal-Taixé and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, journal = {arXiv preprint arXiv:2110.02068}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Yuan, Zhenghang; Mou, Lichao; Zhu, Xiao Xiang Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data Conference IEEE International Symposium on Geoscience and Remote Sensing (IGARSS) , Institute of Electrical and Electronics Engineers (IEEE), 2021, ISBN: 9781665403696. Abstract | Links | BibTeX | Tags: @conference{Yuan2021, title = {Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data}, author = {Zhenghang Yuan and Lichao Mou and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553624}, isbn = {9781665403696}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE International Symposium on Geoscience and Remote Sensing (IGARSS) }, journal = {IEEE International Symposium on Geoscience and Remote Sensing (IGARSS) }, pages = {2999-3002}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, abstract = {Answering questions with natural language by extracting information from image has great potential in various applications. Although visual question answering (VQA) for natural image has been broadly studied, VQA for remote sensing data is still in the early research stage. For the same remote sensing image, there exist questions with dramatically different difficulty-levels. Treating these questions equally may mislead the model and limit the VQA model performance. Considering this problem, in this work, we propose a self-paced curriculum learning (SPCL) based VQA model with hard and soft weighting strategies for remote sensing data. Like human learning process, the model is trained from easy to hard question samples gradually. Extensive experimental results on two datasets demonstrate that the proposed training method can achieve promising performance.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Answering questions with natural language by extracting information from image has great potential in various applications. Although visual question answering (VQA) for natural image has been broadly studied, VQA for remote sensing data is still in the early research stage. For the same remote sensing image, there exist questions with dramatically different difficulty-levels. Treating these questions equally may mislead the model and limit the VQA model performance. Considering this problem, in this work, we propose a self-paced curriculum learning (SPCL) based VQA model with hard and soft weighting strategies for remote sensing data. Like human learning process, the model is trained from easy to hard question samples gradually. Extensive experimental results on two datasets demonstrate that the proposed training method can achieve promising performance. | |
Köninger, Julia; Lugato, Emanuele; Panagos, Panos; Kochupillai, Mrinalini; Orgiazzi, Alberto; Briones, Maria J I Manure management and soil biodiversity: Towards more sustainable food systems in the EU Journal Article Agricultural Systems, 194 , 2021, ISSN: 0308521X. Abstract | Links | BibTeX | Tags: @article{, title = {Manure management and soil biodiversity: Towards more sustainable food systems in the EU}, author = {Julia Köninger and Emanuele Lugato and Panos Panagos and Mrinalini Kochupillai and Alberto Orgiazzi and Maria J I Briones}, doi = {10.1016/J.AGSY.2021.103251}, issn = {0308521X}, year = {2021}, date = {2021-01-01}, journal = {Agricultural Systems}, volume = {194}, publisher = {Elsevier Ltd}, abstract = {CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals.}, keywords = {}, pubstate = {published}, tppubtype = {article} } CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals. | |
Vishwakarma, Bramha Dutt; Horwath, Martin; Groh, Andreas; Bamber, Jonathan L Accounting for GIA signal in GRACE products Journal Article Geophysical Journal International, 228 , pp. 2056-2060, 2021, ISSN: 0956-540X. Abstract | Links | BibTeX | Tags: @article{Vishwakarma2021, title = {Accounting for GIA signal in GRACE products}, author = {Bramha Dutt Vishwakarma and Martin Horwath and Andreas Groh and Jonathan L Bamber}, url = {https://academic.oup.com/gji/article/228/3/2056/6426181}, doi = {10.1093/GJI/GGAB464}, issn = {0956-540X}, year = {2021}, date = {2021-01-01}, journal = {Geophysical Journal International}, volume = {228}, pages = {2056-2060}, publisher = {Oxford Academic}, abstract = {The Gravity Recovery and Climate Experiment (GRACE) observes gravitational potential anomalies that include the effects of present-day surface mass change (PDSMC)-and glacial isostatic adjustment (GIA)-driven solid Earth mass redistribution. Therefore, GIA estimates from a forward model are commonly removed from GRACE to estimate PDSMC. There are several GIA models and to facilitate users in using a GIA model of their choice, both GRACE and GIA products are made available in terms of global gridded fields representing mass anomaly. GRACE-observed gravitational potential anomalies are represented in terms of equivalent water height (EWH) with a relation that accounts for an elastic solid Earth deformation due to PDSMC. However, for obtaining GIA EWH fields from GIA gravitational potential fields, two relations are being used: one that is similar to that being used for GRACE EWH and the other that does not include an elastic deformation effect. This leaves users with the possibility of obtaining different values for PDSMC with a given GRACE and GIA field. In this paper, we discuss the impact of this problem on regional mass change estimates and highlight the need for consistent treatment of GIA signals in GRACE observations.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The Gravity Recovery and Climate Experiment (GRACE) observes gravitational potential anomalies that include the effects of present-day surface mass change (PDSMC)-and glacial isostatic adjustment (GIA)-driven solid Earth mass redistribution. Therefore, GIA estimates from a forward model are commonly removed from GRACE to estimate PDSMC. There are several GIA models and to facilitate users in using a GIA model of their choice, both GRACE and GIA products are made available in terms of global gridded fields representing mass anomaly. GRACE-observed gravitational potential anomalies are represented in terms of equivalent water height (EWH) with a relation that accounts for an elastic solid Earth deformation due to PDSMC. However, for obtaining GIA EWH fields from GIA gravitational potential fields, two relations are being used: one that is similar to that being used for GRACE EWH and the other that does not include an elastic deformation effect. This leaves users with the possibility of obtaining different values for PDSMC with a given GRACE and GIA field. In this paper, we discuss the impact of this problem on regional mass change estimates and highlight the need for consistent treatment of GIA signals in GRACE observations. | |
Basirat, Mina; Roth, Peter M S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization Inproceedings Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, SCITEPRESS - Science and Technology Publications, 2021, ISBN: 9789897584886. Abstract | Links | BibTeX | Tags: @inproceedings{Basirat_2021, title = {S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization}, author = {Mina Basirat and Peter M Roth}, url = {https://orcid.org/0000-0001-9566-1298}, doi = {10.5220/0010338506450652}, isbn = {9789897584886}, year = {2021}, date = {2021-01-01}, booktitle = {Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications}, publisher = {SCITEPRESS - Science and Technology Publications}, abstract = {Recently, it was shown that using a properly parametrized Leaky ReLU (LReLU) as activation function yields significantly better results for a variety of image classification tasks. However, such methods are not feasible in practice. Either the only parameter (i.e., the slope of the negative part) needs to be set manually (L*ReLU), or the approach is vulnerable due to the gradient-based optimization and, thus, highly dependent on a proper initialization (PReLU). In this paper, we exploit the benefits of piecewise linear functions, avoiding these problems. To this end, we propose a fully automatic approach to estimate the slope parameter for LReLU from the data. We realize this via Stochastic Optimization, namely Particle Swarm Optimization (PSO): S*ReLU. In this way, we can show that, compared to widely-used activation functions (including PReLU), we can obtain better results on seven different benchmark datasets, however, also drastically reducing the computational effort.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Recently, it was shown that using a properly parametrized Leaky ReLU (LReLU) as activation function yields significantly better results for a variety of image classification tasks. However, such methods are not feasible in practice. Either the only parameter (i.e., the slope of the negative part) needs to be set manually (L*ReLU), or the approach is vulnerable due to the gradient-based optimization and, thus, highly dependent on a proper initialization (PReLU). In this paper, we exploit the benefits of piecewise linear functions, avoiding these problems. To this end, we propose a fully automatic approach to estimate the slope parameter for LReLU from the data. We realize this via Stochastic Optimization, namely Particle Swarm Optimization (PSO): S*ReLU. In this way, we can show that, compared to widely-used activation functions (including PReLU), we can obtain better results on seven different benchmark datasets, however, also drastically reducing the computational effort. | |
Musa, Osman; Jung, Peter; Caire, Giuseppe Plug-And-Play Learned Gaussian-mixture Approximate Message Passing Inproceedings pp. 4855-4859, IEEE, 2021, ISBN: 978-1-7281-7605-5. @inproceedings{Musa2021, title = {Plug-And-Play Learned Gaussian-mixture Approximate Message Passing}, author = {Osman Musa and Peter Jung and Giuseppe Caire}, doi = {10.1109/ICASSP39728.2021.9414910}, isbn = {978-1-7281-7605-5}, year = {2021}, date = {2021-01-01}, journal = {ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pages = {4855-4859}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Sauder, Jonathan; Genzel, Martin; Jung, Peter Learning Structured Sparse Matrices for Signal Recovery via Unrolled Optimization | OpenReview Inproceedings 2021. @inproceedings{Sauder2021, title = {Learning Structured Sparse Matrices for Signal Recovery via Unrolled Optimization | OpenReview}, author = {Jonathan Sauder and Martin Genzel and Peter Jung}, url = {https://openreview.net/forum?id=IxKSqOq1TKQ}, year = {2021}, date = {2021-01-01}, journal = {NeurIPS 2021 Workshop on Deep Learning and Inverse Problems}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Sánchez-Pastor, Jesús; Thanthrige, Udaya Miriya S K P; Ilgac, Furkan; Jiménez-Sáez, Alejandro; Jung, Peter; Sezgin, Aydin; Jakoby, Rolf Clutter Suppression for Indoor Self-Localization Systems by Iteratively Reweighted Low-Rank Plus Sparse Recovery Journal Article Sensors 2021, Vol. 21, Page 6842, 21 , pp. 6842, 2021, ISSN: 1424-8220. Abstract | Links | BibTeX | Tags: @article{SanchezPastor2021, title = {Clutter Suppression for Indoor Self-Localization Systems by Iteratively Reweighted Low-Rank Plus Sparse Recovery}, author = {Jesús Sánchez-Pastor and Udaya Miriya S K P Thanthrige and Furkan Ilgac and Alejandro Jiménez-Sáez and Peter Jung and Aydin Sezgin and Rolf Jakoby}, url = {https://www.mdpi.com/1424-8220/21/20/6842/htm https://www.mdpi.com/1424-8220/21/20/6842}, doi = {10.3390/S21206842}, issn = {1424-8220}, year = {2021}, date = {2021-01-01}, journal = {Sensors 2021, Vol. 21, Page 6842}, volume = {21}, pages = {6842}, publisher = {Multidisciplinary Digital Publishing Institute}, abstract = {Self-localization based on passive RFID-based has many potential applications. One of the main challenges it faces is the suppression of the reflected signals from unwanted objects (i.e., clutter). Typically, the clutter echoes are much stronger than the backscattered signals of the passive tag landmarks used in such scenarios. Therefore, successful tag detection can be very challenging. We consider two types of tags, namely low-Q and high-Q tags. The high-Q tag features a sparse frequency response, whereas the low-Q tag presents a broad frequency response. Further, the clutter usually showcases a short-lived response. In this work, we propose an iterative algorithm based on a low-rank plus sparse recovery approach (RPCA) to mitigate clutter and retrieve the landmark response. In addition to that, we compare the proposed approach with the well-known time-gating technique. It turns out that RPCA outperforms significantly time-gating for low-Q tags, achieving clutter suppression and tag identification when clutter encroaches on the time-gating window span, whereas it also increases the backscattered power at resonance by approximately 12 dB at 80 cm for high-Q tags. Altogether, RPCA seems a promising approach to improve the identification of passive indoor self-localization tag landmarks.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Self-localization based on passive RFID-based has many potential applications. One of the main challenges it faces is the suppression of the reflected signals from unwanted objects (i.e., clutter). Typically, the clutter echoes are much stronger than the backscattered signals of the passive tag landmarks used in such scenarios. Therefore, successful tag detection can be very challenging. We consider two types of tags, namely low-Q and high-Q tags. The high-Q tag features a sparse frequency response, whereas the low-Q tag presents a broad frequency response. Further, the clutter usually showcases a short-lived response. In this work, we propose an iterative algorithm based on a low-rank plus sparse recovery approach (RPCA) to mitigate clutter and retrieve the landmark response. In addition to that, we compare the proposed approach with the well-known time-gating technique. It turns out that RPCA outperforms significantly time-gating for low-Q tags, achieving clutter suppression and tag identification when clutter encroaches on the time-gating window span, whereas it also increases the backscattered power at resonance by approximately 12 dB at 80 cm for high-Q tags. Altogether, RPCA seems a promising approach to improve the identification of passive indoor self-localization tag landmarks. | |
Damara, Muhammad Fadli; Kornhardt, Gregor; Jung, Peter Solving Inverse Problems with Conditional-GAN Prior via Fast Network-Projected Gradient Descent Journal Article 2021. Abstract | Links | BibTeX | Tags: @article{Damara2021, title = {Solving Inverse Problems with Conditional-GAN Prior via Fast Network-Projected Gradient Descent}, author = {Muhammad Fadli Damara and Gregor Kornhardt and Peter Jung}, url = {https://arxiv.org/abs/2109.01105v1}, doi = {10.48550/arxiv.2109.01105}, year = {2021}, date = {2021-01-01}, abstract = {The projected gradient descent (PGD) method has shown to be effective in recovering compressed signals described in a data-driven way by a generative model, i.e., a generator which has learned the data distribution. Further reconstruction improvements for such inverse problems can be achieved by conditioning the generator on the measurement. The boundary equilibrium generative adversarial network (BEGAN) implements an equilibrium based loss function and an auto-encoding discriminator to better balance the performance of the generator and the discriminator. In this work we investigate a network-based projected gradient descent (NPGD) algorithm for measurement-conditional generative models to solve the inverse problem much faster than regular PGD. We combine the NPGD with conditional GAN/BEGAN to evaluate their effectiveness in solving compressed sensing type problems. Our experiments on the MNIST and CelebA datasets show that the combination of measurement conditional model with NPGD works well in recovering the compressed signal while achieving similar or in some cases even better performance along with a much faster reconstruction. The achieved reconstruction speed-up in our experiments is up to 140-175.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The projected gradient descent (PGD) method has shown to be effective in recovering compressed signals described in a data-driven way by a generative model, i.e., a generator which has learned the data distribution. Further reconstruction improvements for such inverse problems can be achieved by conditioning the generator on the measurement. The boundary equilibrium generative adversarial network (BEGAN) implements an equilibrium based loss function and an auto-encoding discriminator to better balance the performance of the generator and the discriminator. In this work we investigate a network-based projected gradient descent (NPGD) algorithm for measurement-conditional generative models to solve the inverse problem much faster than regular PGD. We combine the NPGD with conditional GAN/BEGAN to evaluate their effectiveness in solving compressed sensing type problems. Our experiments on the MNIST and CelebA datasets show that the combination of measurement conditional model with NPGD works well in recovering the compressed signal while achieving similar or in some cases even better performance along with a much faster reconstruction. The achieved reconstruction speed-up in our experiments is up to 140-175. | |
Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Out-of-distribution Detection in Satellite Image Classification Conference RobustML workshop at ICLR 2021, 2021. BibTeX | Tags: @conference{Gawlikowski2021b, title = {Out-of-distribution Detection in Satellite Image Classification}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {RobustML workshop at ICLR 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Saha, Sudipan; Zhu, Xiao Xiang Patch-level unsupervised planetary change detection Journal Article IEEE Geoscience and Remote Sensing Letters, 19 , 2021, ISSN: 15580571. Abstract | Links | BibTeX | Tags: @article{Saha2021d, title = {Patch-level unsupervised planetary change detection}, author = {Sudipan Saha and Xiao Xiang Zhu}, doi = {10.1109/LGRS.2021.3130862}, issn = {15580571}, year = {2021}, date = {2021-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, volume = {19}, publisher = {IEEE}, abstract = {Change detection (CD) is critical for analyzing data collected by planetary exploration missions, e.g., for identification of new impact craters. However, CD is still a relatively new topic in the context of planetary exploration. Sheer variation of planetary data makes CD much more challenging than in the case of Earth observation (EO). Unlike CD for EO, patch-level decision is preferred in planetary exploration as it is difficult to obtain perfect pixelwise alignment/coregistration between the bi-temporal planetary images. Lack of labeled bi-temporal data impedes supervised CD. To overcome these challenges, we propose an unsupervised CD method that exploits a pretrained feature extractor to obtain bi-temporal deep features that are further processed using global max-pooling to obtain patch-level feature description. Bi-temporal patch-level features are further analyzed based on difference to determine whether a patch is changed. Additionally, a self-supervised method is proposed to estimate the decision boundary between the changed and unchanged patches. Experimental results on three planetary CD datasets from two different planetary bodies (Mars and Moon) demonstrate that the proposed method often outperforms supervised planetary CD methods. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/planetaryCDUnsup.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Change detection (CD) is critical for analyzing data collected by planetary exploration missions, e.g., for identification of new impact craters. However, CD is still a relatively new topic in the context of planetary exploration. Sheer variation of planetary data makes CD much more challenging than in the case of Earth observation (EO). Unlike CD for EO, patch-level decision is preferred in planetary exploration as it is difficult to obtain perfect pixelwise alignment/coregistration between the bi-temporal planetary images. Lack of labeled bi-temporal data impedes supervised CD. To overcome these challenges, we propose an unsupervised CD method that exploits a pretrained feature extractor to obtain bi-temporal deep features that are further processed using global max-pooling to obtain patch-level feature description. Bi-temporal patch-level features are further analyzed based on difference to determine whether a patch is changed. Additionally, a self-supervised method is proposed to estimate the decision boundary between the changed and unchanged patches. Experimental results on three planetary CD datasets from two different planetary bodies (Mars and Moon) demonstrate that the proposed method often outperforms supervised planetary CD methods. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/planetaryCDUnsup. | |
Saha, Sudipan; Kondmann, Lukas; Song, Qian; Zhu, Xiao Xiang Change Detection in Hyperdimensional Images using Untrained Models Journal Article IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14 , pp. 11029, 2021. Abstract | Links | BibTeX | Tags: @article{Saha2021e, title = {Change Detection in Hyperdimensional Images using Untrained Models}, author = {Sudipan Saha and Lukas Kondmann and Qian Song and Xiao Xiang Zhu}, doi = {10.1109/JSTARS.2021.3121556}, year = {2021}, date = {2021-01-01}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, volume = {14}, pages = {11029}, publisher = {IEEE}, abstract = {Deep transfer-learning-based change detection methods are dependent on the availability of sensor-specific pretrained feature extractors. Such feature extractors are not always available due to lack of training data, especially for hyperspectral sensors and other hyperdimensional images. Moreover models trained on easily available multispectral (RGB/RGB-NIR) images cannot be reused on such hyperdimensional images due to their irregular number of bands. While hyperdimensional images show large number of spectral bands, they generally show much less spatial complexity, thus reducing the requirement of large receptive fields of convo-lution filters. Recent works in the computer vision have shown that even untrained deep models can yield remarkable result in some tasks like super-resolution and surface reconstruction. This motivates us to make a bold proposition that untrained lightweight deep model, initialized with some weight initialization strategy, can be used to extract useful semantic features from bi-temporal hyperdimensional images. Based on this proposition, we design a novel change detection framework for hyperdimensional images by extracting bitemporal features using an untrained model and further comparing the extracted features using deep change vector analysis to distinguish changed pixels from the unchanged ones. We further use the deep change hypervectors to cluster the changed pixels into different semantic groups. We conduct experiments on four change detection datasets: three hyperspectral datasets and a hyperdimensional polarimetric synthetic aperture radar dataset. The results clearly demonstrate that the proposed method is suitable for change detection in hyperdimensional remote sensing data.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Deep transfer-learning-based change detection methods are dependent on the availability of sensor-specific pretrained feature extractors. Such feature extractors are not always available due to lack of training data, especially for hyperspectral sensors and other hyperdimensional images. Moreover models trained on easily available multispectral (RGB/RGB-NIR) images cannot be reused on such hyperdimensional images due to their irregular number of bands. While hyperdimensional images show large number of spectral bands, they generally show much less spatial complexity, thus reducing the requirement of large receptive fields of convo-lution filters. Recent works in the computer vision have shown that even untrained deep models can yield remarkable result in some tasks like super-resolution and surface reconstruction. This motivates us to make a bold proposition that untrained lightweight deep model, initialized with some weight initialization strategy, can be used to extract useful semantic features from bi-temporal hyperdimensional images. Based on this proposition, we design a novel change detection framework for hyperdimensional images by extracting bitemporal features using an untrained model and further comparing the extracted features using deep change vector analysis to distinguish changed pixels from the unchanged ones. We further use the deep change hypervectors to cluster the changed pixels into different semantic groups. We conduct experiments on four change detection datasets: three hyperspectral datasets and a hyperdimensional polarimetric synthetic aperture radar dataset. The results clearly demonstrate that the proposed method is suitable for change detection in hyperdimensional remote sensing data. | |
Qian, Kun; Wang, Yuanyuan; Shi, Yilei; Zhu, Xiao Xiang γ-Net: Superresolving SAR Tomographic Inversion via Deep Learning Journal Article Forthcoming IEEE Transactions on Geoscience and Remote Sensing, Forthcoming, (in press). @article{Qian2021, title = {γ-Net: Superresolving SAR Tomographic Inversion via Deep Learning}, author = {Kun Qian and Yuanyuan Wang and Yilei Shi and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Transactions on Geoscience and Remote Sensing}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, abstract = {(This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.) Synthetic aperture radar tomography (TomoSAR) has been extensively employed in 3-D reconstruction in dense urban areas using high-resolution SAR acquisitions. Compressive sensing (CS)-based algorithms are generally considered as the state of the art in super-resolving TomoSAR, in particular in the single look case. This superior performance comes at the cost of extra computational burdens, because of the sparse reconstruction, which cannot be solved analytically and we need to employ computationally expensive iterative solvers. In this paper, we propose a novel deep learning-based super-resolving TomoSAR inversion approach, γ-Net, to tackle this challenge. γ-Net adopts advanced complex-valued learned iterative shrinkage thresholding algorithm (CV-LISTA) to mimic the iterative optimization step in sparse reconstruction. Simulations show the height estimate from a well-trained γ-Net approaches the Cramér-Rao lower bound while improving the computational efficiency by 1 to 2 orders of magnitude comparing to the first-order CS-based methods. It also shows no degradation in the super-resolution power comparing to the state-of-the-art second-order TomoSAR solvers, which are much more computationally expensive than the first-order methods. Specifically, γ-Net reaches more than 90% detection rate in moderate super-resolving cases at 25 measurements at 6dB SNR. Moreover, simulation at limited baselines demonstrates that the proposed algorithm outperforms the second-order CS-based method by a fair margin. Test on real TerraSAR-X data with just 6 interferograms also shows high-quality 3-D reconstruction with high-density detected double scatterers.}, note = {in press}, keywords = {}, pubstate = {forthcoming}, tppubtype = {article} } (This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.) Synthetic aperture radar tomography (TomoSAR) has been extensively employed in 3-D reconstruction in dense urban areas using high-resolution SAR acquisitions. Compressive sensing (CS)-based algorithms are generally considered as the state of the art in super-resolving TomoSAR, in particular in the single look case. This superior performance comes at the cost of extra computational burdens, because of the sparse reconstruction, which cannot be solved analytically and we need to employ computationally expensive iterative solvers. In this paper, we propose a novel deep learning-based super-resolving TomoSAR inversion approach, γ-Net, to tackle this challenge. γ-Net adopts advanced complex-valued learned iterative shrinkage thresholding algorithm (CV-LISTA) to mimic the iterative optimization step in sparse reconstruction. Simulations show the height estimate from a well-trained γ-Net approaches the Cramér-Rao lower bound while improving the computational efficiency by 1 to 2 orders of magnitude comparing to the first-order CS-based methods. It also shows no degradation in the super-resolution power comparing to the state-of-the-art second-order TomoSAR solvers, which are much more computationally expensive than the first-order methods. Specifically, γ-Net reaches more than 90% detection rate in moderate super-resolving cases at 25 measurements at 6dB SNR. Moreover, simulation at limited baselines demonstrates that the proposed algorithm outperforms the second-order CS-based method by a fair margin. Test on real TerraSAR-X data with just 6 interferograms also shows high-quality 3-D reconstruction with high-density detected double scatterers. | |
Saha, Sudipan; Ahmad, Tahir Federated transfer learning: Concept and applications Journal Article Intelligenza Artificiale, 15 (1), pp. 35-44, 2021, ISSN: 1724-8035. Abstract | Links | BibTeX | Tags: @article{Saha2021f, title = {Federated transfer learning: Concept and applications}, author = {Sudipan Saha and Tahir Ahmad}, doi = {10.3233/IA-200075}, issn = {1724-8035}, year = {2021}, date = {2021-01-01}, journal = {Intelligenza Artificiale}, volume = {15}, number = {1}, pages = {35-44}, publisher = {IOS Press}, abstract = {Development of Artificial Intelligence (AI) is inherently tied to the development of data. However, in most industries data exists in form of isolated islands, with limited scope of sharing between different organizations. This is an hindrance to the further development of AI. Federated learning has emerged as a possible solution to this problem in the last few years without compromising user privacy. Among different variants of the federated learning, noteworthy is federated transfer learning (FTL) that allows knowledge to be transferred across domains that do not have many overlapping features and users. In this work we provide a comprehensive survey of the existing works on this topic. In more details, we study the background of FTL and its different existing applications. We further analyze FTL from privacy and machine learning perspective.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Development of Artificial Intelligence (AI) is inherently tied to the development of data. However, in most industries data exists in form of isolated islands, with limited scope of sharing between different organizations. This is an hindrance to the further development of AI. Federated learning has emerged as a possible solution to this problem in the last few years without compromising user privacy. Among different variants of the federated learning, noteworthy is federated transfer learning (FTL) that allows knowledge to be transferred across domains that do not have many overlapping features and users. In this work we provide a comprehensive survey of the existing works on this topic. In more details, we study the background of FTL and its different existing applications. We further analyze FTL from privacy and machine learning perspective. | |
Köninger, Julia; Lugato, Emanuele; Panagos, Panos; Kochupillai, Mrinalini; Orgiazzi, Alberto; Briones, Maria J I Manure management and soil biodiversity: Towards more sustainable food systems in the EU Journal Article Agricultural Systems, 194 , 2021, ISSN: 0308521X. Abstract | Links | BibTeX | Tags: @article{Koeninger2021, title = {Manure management and soil biodiversity: Towards more sustainable food systems in the EU}, author = {Julia Köninger and Emanuele Lugato and Panos Panagos and Mrinalini Kochupillai and Alberto Orgiazzi and Maria J I Briones}, url = {http://creativecommons.org/licenses/by/4.0/}, doi = {10.1016/J.AGSY.2021.103251}, issn = {0308521X}, year = {2021}, date = {2021-01-01}, journal = {Agricultural Systems}, volume = {194}, publisher = {Elsevier Ltd}, abstract = {CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals.}, keywords = {}, pubstate = {published}, tppubtype = {article} } CONTEXT: In the European Union (EU-27) and UK, animal farming generated annually more than 1.4 billion tonnes of manure during the period 2016–2019. Of this, more than 90% is directly re-applied to soils as organic fertiliser. Manure promotes plant growth, provides nutritious food to soil organisms, adds genetic and functional diversity to soils and improves the chemical and physical soil properties. However, it can also cause pollution by introducing toxic elements (i.e., heavy metals, antibiotics, pathogens) and contribute to nutrient losses. Soil organisms play an essential role in manure transformation into the soil and the degradation of any potential toxic constitutes; however, manure management practices often neglect soil biodiversity. OBJECTIVE: In this review, we explored the impact of manure from farmed animals on soil biodiversity by considering factors that determine the effects of manure and vice versa. By evaluating manure's potential to enhance soil biodiversity, but also its environmental risks, we assessed current and future EU policy and legislations with the ultimate aim of providing recommendations that can enable a more sustainable management of farm manures. METHODS: This review explored the relationship between manure and soil biodiversity by considering 407 published papers and relevant legislative provisions. In addition, we evaluated whether benefits and risks on soil biodiversity are considered in manure management. Thereafter, we analysed the current legislation in the European Union relevant to manure, an important driver for its treatment, application and storage. RESULTS AND CONCLUSIONS: This review found that coupling manure management with soil biodiversity can mitigate present and future environmental risks. Our analyses showed that manure quality is more important to soil biodiversity than manure quantity and therefore, agricultural practices that protect and promote soil biodiversity with the application of appropriate, high-quality manure or biostimulant preparations based on manure, could accelerate the move towards more sustainable food production systems. Soil biodiversity needs to be appropriately factored in when assessing manure amendments to provide better guidelines on the use of manure and to reduce costs and environmental risks. However, radical changes in current philosophies and practices are needed so that soil biodiversity can be enhanced by manure management. SIGNIFICANCE: Manure quality in the EU requires greater attention, calling for more targeted policies. Our proposed approach could be applied by European Union Member States to include soil protection measures in national legislation, and at the EU level, can enable the implementation of strategic goals. | |
Prexl, Jonathan; Saha, Sudipan; Zhu, Xiao Xiang Mitigating Spatial and Spectral Differences for Change Detection using Super-resolution and Unsupervised learning Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), IEEE, 2021. @conference{Prexl2021, title = {Mitigating Spatial and Spectral Differences for Change Detection using Super-resolution and Unsupervised learning}, author = {Jonathan Prexl and Sudipan Saha and Xiao Xiang Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, publisher = {IEEE}, abstract = {Change detection (CD) is one of the most researched areas in remote sensing. However, most CD methods assume that the pre-change and post-change images are acquired by the same sensor, having the same set of spectral bands and same spatial resolution. This severely limits the applicability of CD methods. It is not trivial to apply the existing CD methods in multi-sensor scenario. Towards this direction, we propose an unsu-pervised CD method that can handle large differences in spatial resolution and can work with completely different set of spectral bands. The proposed method uses a self-supervised super-resolution strategy to upsample the lower resolution image , thus mitigating differences in spatial resolution. To mitigate spectral differences, a self-supervised learning strategy is used that ingests both images as input and trains a network using self-supervised loss accounting for the spectral differences in both images. Once trained this network is used in deep change vector analysis framework for change detection. We validated the proposed method in an experimental setup where the pre-change and post-change images have different spatial resolution (10 m and 20 m/pixel) and completely dis-joint set of spectral bands.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Change detection (CD) is one of the most researched areas in remote sensing. However, most CD methods assume that the pre-change and post-change images are acquired by the same sensor, having the same set of spectral bands and same spatial resolution. This severely limits the applicability of CD methods. It is not trivial to apply the existing CD methods in multi-sensor scenario. Towards this direction, we propose an unsu-pervised CD method that can handle large differences in spatial resolution and can work with completely different set of spectral bands. The proposed method uses a self-supervised super-resolution strategy to upsample the lower resolution image , thus mitigating differences in spatial resolution. To mitigate spectral differences, a self-supervised learning strategy is used that ingests both images as input and trains a network using self-supervised loss accounting for the spectral differences in both images. Once trained this network is used in deep change vector analysis framework for change detection. We validated the proposed method in an experimental setup where the pre-change and post-change images have different spatial resolution (10 m and 20 m/pixel) and completely dis-joint set of spectral bands. | |
Kondmann, Lukas; Toker, Aysim; Saha, Sudipan; Schölkopf, Bernhard; Leal-Taixé, Laura; Zhu, Xiao Xiang Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images Journal Article arXiv preprint arXiv:2110.02068, 2021. Abstract | Links | BibTeX | Tags: @article{Kondmann2021b, title = {Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images}, author = {Lukas Kondmann and Aysim Toker and Sudipan Saha and Bernhard Schölkopf and Laura Leal-Taixé and Xiao Xiang Zhu}, doi = {10.1109/tgrs.2021.3130842}, year = {2021}, date = {2021-01-01}, journal = {arXiv preprint arXiv:2110.02068}, abstract = {Detecting changes on the ground in multitemporal Earth observation data is one of the key problems in remote sensing. In this paper, we introduce Sibling Regression for Optical Change detection (SiROC), an unsupervised method for change detection in optical satellite images with medium and high resolution. SiROC is a spatial context-based method that models a pixel as a linear combination of its distant neighbors. It uses this model to analyze differences in the pixel and its spatial context-based predictions in subsequent time periods for change detection. We combine this spatial context-based change detection with ensembling over mutually exclusive neighborhoods and transitioning from pixel to object-level changes with morphological operations. SiROC achieves competitive performance for change detection with medium-resolution Sentinel-2 and high-resolution Planetscope imagery on four datasets. Besides accurate predictions without the need for training, SiROC also provides a well-calibrated uncertainty of its predictions. This makes the method especially useful in conjunction with deep-learning based methods for applications such as pseudo-labeling.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Detecting changes on the ground in multitemporal Earth observation data is one of the key problems in remote sensing. In this paper, we introduce Sibling Regression for Optical Change detection (SiROC), an unsupervised method for change detection in optical satellite images with medium and high resolution. SiROC is a spatial context-based method that models a pixel as a linear combination of its distant neighbors. It uses this model to analyze differences in the pixel and its spatial context-based predictions in subsequent time periods for change detection. We combine this spatial context-based change detection with ensembling over mutually exclusive neighborhoods and transitioning from pixel to object-level changes with morphological operations. SiROC achieves competitive performance for change detection with medium-resolution Sentinel-2 and high-resolution Planetscope imagery on four datasets. Besides accurate predictions without the need for training, SiROC also provides a well-calibrated uncertainty of its predictions. This makes the method especially useful in conjunction with deep-learning based methods for applications such as pseudo-labeling. | |
Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang Towards Out-of-Distribution Detection for Remote Sensing Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), Institute of Electrical and Electronics Engineers (IEEE), 2021, ISBN: 9781665403696. Abstract | Links | BibTeX | Tags: @conference{Gawlikowski2021b, title = {Towards Out-of-Distribution Detection for Remote Sensing}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553266}, isbn = {9781665403696}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, pages = {8676-8679}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, abstract = {In remote sensing, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data, differences in the geographic area, and multi-sensor differences. Deep learning based models may behave in unexpected manners when subjected to test data that has such distributional shifts from the training data, also called out-of-distribution (OOD) examples. Vulnerability to OOD data severely reduces the reliability of deep learning based models. In this work, we address this issue by proposing a model to quantify distributional uncertainty of deep learning based remote sensing models. In particular, we adopt a Dirichlet Prior Network for remote sensing data. The approach seeks to maximize the representation gap between the in-domain and OOD examples for a better identification of unknown examples at test time. Experimental results on three exemplary test scenarios show that the proposed model can detect OOD images in remote sensing.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } In remote sensing, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data, differences in the geographic area, and multi-sensor differences. Deep learning based models may behave in unexpected manners when subjected to test data that has such distributional shifts from the training data, also called out-of-distribution (OOD) examples. Vulnerability to OOD data severely reduces the reliability of deep learning based models. In this work, we address this issue by proposing a model to quantify distributional uncertainty of deep learning based remote sensing models. In particular, we adopt a Dirichlet Prior Network for remote sensing data. The approach seeks to maximize the representation gap between the in-domain and OOD examples for a better identification of unknown examples at test time. Experimental results on three exemplary test scenarios show that the proposed model can detect OOD images in remote sensing. | |
Saha, Sudipan; Banerjee, Biplab; Zhu, Xiao Xiang Trusting small training dataset for supervised change detection Conference IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted), Institute of Electrical and Electronics Engineers (IEEE), 2021, ISBN: 9781665403696. Abstract | Links | BibTeX | Tags: @conference{Saha2021g, title = {Trusting small training dataset for supervised change detection}, author = {Sudipan Saha and Biplab Banerjee and Xiao Xiang Zhu}, url = {https://arxiv.org/abs/2104.05443v1}, doi = {10.48550/arxiv.2104.05443}, isbn = {9781665403696}, year = {2021}, date = {2021-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2021 (Accepted)}, pages = {2031-2034}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, abstract = {Deep learning (DL) based supervised change detection (CD) models require large labeled training data. Due to the difficulty of collecting labeled multi-temporal data, unsupervised methods are preferred in the CD literature. However, unsupervised methods cannot fully exploit the potentials of data-driven deep learning and thus they are not absolute alternative to the supervised methods. This motivates us to look deeper into the supervised DL methods and investigate how they can be adopted intelligently for CD by minimizing the requirement of labeled training data. Towards this, in this work we show that geographically diverse training dataset can yield significant improvement over less diverse training datasets of the same size. We propose a simple confidence indicator for verifying the trustworthiness/confidence of supervised models trained with small labeled dataset. Moreover, we show that for the test cases where supervised CD model is found to be less confident/trustworthy, unsupervised methods often produce better result than the supervised ones.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Deep learning (DL) based supervised change detection (CD) models require large labeled training data. Due to the difficulty of collecting labeled multi-temporal data, unsupervised methods are preferred in the CD literature. However, unsupervised methods cannot fully exploit the potentials of data-driven deep learning and thus they are not absolute alternative to the supervised methods. This motivates us to look deeper into the supervised DL methods and investigate how they can be adopted intelligently for CD by minimizing the requirement of labeled training data. Towards this, in this work we show that geographically diverse training dataset can yield significant improvement over less diverse training datasets of the same size. We propose a simple confidence indicator for verifying the trustworthiness/confidence of supervised models trained with small labeled dataset. Moreover, we show that for the test cases where supervised CD model is found to be less confident/trustworthy, unsupervised methods often produce better result than the supervised ones. | |
Nandy, Jay; Saha, Sudipan; Hsu, Wynne; Lee, Mong; Zhu, Xiao Covariate Shift Adaptation for Adversarially Robust Classifier Conference ICLR 2021 Workshop on Security and Safety in Machine Learning Systems, 2021. BibTeX | Tags: @conference{Nandy2021, title = {Covariate Shift Adaptation for Adversarially Robust Classifier}, author = {Jay Nandy and Sudipan Saha and Wynne Hsu and Mong Lee and Xiao Zhu}, year = {2021}, date = {2021-01-01}, booktitle = {ICLR 2021 Workshop on Security and Safety in Machine Learning Systems}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Tuia, Xiao Xiang Zhu Markus Reichstein Gustau Camps-Valls Devis Wiley & Sons, 2021, ISBN: 978-1-119-64614-3. Abstract | Links | BibTeX | Tags: @book{EditorsCampsValls2021, title = {Deep learning for the earth sciences : a comprehensive approach to remote sensing, climate science and geosciences}, author = {Xiao Xiang Zhu Markus Reichstein Gustau Camps-Valls Devis Tuia}, editor = {Xiao Xiang Zhu Markus Reichstein Gustau Camps-Valls Devis Tuia}, url = {https://www.wiley-vch.de/de/fachgebiete/ingenieurwesen/elektrotechnik-und-elektronik-10ee/bildgebende-systeme-u-verfahren-10eec/fernerkundung-10eec1/deep-learning-for-the-earth-sciences-978-1-119-64614-3}, isbn = {978-1-119-64614-3}, year = {2021}, date = {2021-01-01}, publisher = {Wiley & Sons}, abstract = {"The research in deep learning for the geosciences and Earth observation is growing fast and goes beyond the mere application of algorithms to new data. This is a huge interdisciplinary field. Applying new algorithms to the data deluge is a hot topic in all these cross-sectorial fields. Academic research on this area is strongly involved, and many specialized conferences and special issues in journals are arising each year. The book will provide the reader with the landscape, skills, and principles to quickly become familiar with both fields? needs and applications and will give a principled status of where are we now. The practitioner will be ready to use the technology and principles in his/her own research field in a short period of time. The highlights on future research at the end of each chapter will provide new ideas, particularly for those people involved in advanced research education, who will find these highlights of special interest for PhD Thesis orientations"--}, keywords = {}, pubstate = {published}, tppubtype = {book} } "The research in deep learning for the geosciences and Earth observation is growing fast and goes beyond the mere application of algorithms to new data. This is a huge interdisciplinary field. Applying new algorithms to the data deluge is a hot topic in all these cross-sectorial fields. Academic research on this area is strongly involved, and many specialized conferences and special issues in journals are arising each year. The book will provide the reader with the landscape, skills, and principles to quickly become familiar with both fields? needs and applications and will give a principled status of where are we now. The practitioner will be ready to use the technology and principles in his/her own research field in a short period of time. The highlights on future research at the end of each chapter will provide new ideas, particularly for those people involved in advanced research education, who will find these highlights of special interest for PhD Thesis orientations"-- | |
Saha, Sudipan; Kondmann, Lukas; Zhu, Xiao Xiang Deep No Learning Approach for Unsupervised Change Detection in Hyperspectral Images Conference XXIV ISPRS Congress 2021, 2021. @conference{Saha2021bb, title = {Deep No Learning Approach for Unsupervised Change Detection in Hyperspectral Images}, author = {Sudipan Saha and Lukas Kondmann and Xiao Xiang Zhu}, url = {https://www.researchgate.net/publication/352887191_Deep_No_Learning_Approach_for_Unsupervised_Change_Detection_in_Hyperspectral_Images}, year = {2021}, date = {2021-01-01}, booktitle = {XXIV ISPRS Congress 2021}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Ebel, Patrick; Saha, Sudipan; Zhu, Xiao Xiang Fusing Multi-modal Data for Supervised Change Detection Conference XXIV ISPRS Congress 2021, 2021. Abstract | Links | BibTeX | Tags: @conference{Ebel2021, title = {Fusing Multi-modal Data for Supervised Change Detection}, author = {Patrick Ebel and Sudipan Saha and Xiao Xiang Zhu}, url = {https://doi.org/10.5194/isprs-archives-XLIII-B3-2021-243-2021}, doi = {10.5194/isprs-archives-XLIII-B3-2021-243-2021}, year = {2021}, date = {2021-01-01}, booktitle = {XXIV ISPRS Congress 2021}, abstract = {With the rapid development of remote sensing technology in the last decade, different modalities of remote sensing data recorded via a variety of sensors are now easily accessible. Different sensors often provide complementary information and thus a more detailed and accurate Earth observation is possible by integrating their joint information. While change detection methods have been traditionally proposed for homogeneous data, combining multi-sensor multi-temporal data with different characteristics and resolution may provide a more robust interpretation of spatio-temporal evolution. However, integration of multi-temporal information from disparate sensory sources is challenging. Moreover, research in this direction is often hindered by a lack of available multi-modal data sets. To resolve these current shortcomings we curate a novel data set for multi-modal change detection. We further propose a novel Siamese architecture for fusion of SAR and optical observations for multi-modal change detection, which underlines the value of our newly gathered data. An experimental validation on the aforementioned data set demonstrates the potentials of the proposed model, which outperforms common mono-modal methods compared against.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } With the rapid development of remote sensing technology in the last decade, different modalities of remote sensing data recorded via a variety of sensors are now easily accessible. Different sensors often provide complementary information and thus a more detailed and accurate Earth observation is possible by integrating their joint information. While change detection methods have been traditionally proposed for homogeneous data, combining multi-sensor multi-temporal data with different characteristics and resolution may provide a more robust interpretation of spatio-temporal evolution. However, integration of multi-temporal information from disparate sensory sources is challenging. Moreover, research in this direction is often hindered by a lack of available multi-modal data sets. To resolve these current shortcomings we curate a novel data set for multi-modal change detection. We further propose a novel Siamese architecture for fusion of SAR and optical observations for multi-modal change detection, which underlines the value of our newly gathered data. An experimental validation on the aforementioned data set demonstrates the potentials of the proposed model, which outperforms common mono-modal methods compared against. | |
Klemmer, Konstantin; Saha, Sudipan; Kahl, Matthias; Xu, Tianlin; Zhu, Xiao Xiang Generative modeling of spatio-temporal weather patterns with extreme event conditioning Inproceedings AI: Modeling Oceans and Climate Change (AIMOCC 2021) Workshop, ICLR 2021, 2021. Abstract | Links | BibTeX | Tags: @inproceedings{Klemmer2021b, title = {Generative modeling of spatio-temporal weather patterns with extreme event conditioning}, author = {Konstantin Klemmer and Sudipan Saha and Matthias Kahl and Tianlin Xu and Xiao Xiang Zhu}, url = {https://arxiv.org/abs/2104.12469v1}, doi = {10.48550/arxiv.2104.12469}, year = {2021}, date = {2021-01-01}, booktitle = {AI: Modeling Oceans and Climate Change (AIMOCC 2021) Workshop, ICLR 2021}, abstract = {Deep generative models are increasingly used to gain insights in the geospatial data domain, e.g., for climate data. However, most existing approaches work with temporal snapshots or assume 1D time-series; few are able to capture spatio-temporal processes simultaneously. Beyond this, Earth-systems data often exhibit highly irregular and complex patterns, for example caused by extreme weather events. Because of climate change, these phenomena are only increasing in frequency. Here, we proposed a novel GAN-based approach for generating spatio-temporal weather patterns conditioned on detected extreme events. Our approach augments GAN generator and discriminator with an encoded extreme weather event segmentation mask. These segmentation masks can be created from raw input using existing event detection frameworks. As such, our approach is highly modular and can be combined with custom GAN architectures. We highlight the applicability of our proposed approach in experiments with real-world surface radiation and zonal wind data.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Deep generative models are increasingly used to gain insights in the geospatial data domain, e.g., for climate data. However, most existing approaches work with temporal snapshots or assume 1D time-series; few are able to capture spatio-temporal processes simultaneously. Beyond this, Earth-systems data often exhibit highly irregular and complex patterns, for example caused by extreme weather events. Because of climate change, these phenomena are only increasing in frequency. Here, we proposed a novel GAN-based approach for generating spatio-temporal weather patterns conditioned on detected extreme events. Our approach augments GAN generator and discriminator with an encoded extreme weather event segmentation mask. These segmentation masks can be created from raw input using existing event detection frameworks. As such, our approach is highly modular and can be combined with custom GAN architectures. We highlight the applicability of our proposed approach in experiments with real-world surface radiation and zonal wind data. | |
Kochupillai, Mrinalini; Gallersdörfer, Ulrich; Köninger, Julia; Beck, Roman Incentivizing research & innovation with agrobiodiversity conserved in situ: Possibilities and limitations of a blockchain-based solution Journal Article Journal of Cleaner Production, 309 , pp. 127155, 2021, ISSN: 0959-6526. @article{Kochupillai2021a, title = {Incentivizing research & innovation with agrobiodiversity conserved in situ: Possibilities and limitations of a blockchain-based solution}, author = {Mrinalini Kochupillai and Ulrich Gallersdörfer and Julia Köninger and Roman Beck}, issn = {0959-6526}, year = {2021}, date = {2021-01-01}, journal = {Journal of Cleaner Production}, volume = {309}, pages = {127155}, abstract = {This article explores the use of blockchain for agrobiodiversity (B4A) with a specific focus on (i) providing an overview of the existing regulatory challenges when it comes to conserving agrobiodiversity, which results in a lack of research and innovation when it comes to agrobiodiversity conserved in situ, (ii) investigating how a blockchain-based solution may help overcome these challenges, and (iii) illustrating how incentive mechanisms can help to overcome existing intellectual property regimes that prevent effective conservation, research and innovation (CRI). Our research identifies (i) lack of incentives, (ii) lack of trust among stakeholders, and (iii) lack of traceability options as main hindering reasons for in situ CRI with agrobiodiversity. Further, We find that blockchain solutions may empower data providers, including small farmers, to collectively track, control and monetize the use of data and assets shared, while minimizing fraudulent activities. Transaction costs may also be lowered by removing complex and expensive interaction processes. However, further research and development is necessary to design an ethical and sustainable blockchain-based solution to incentivize in situ conservation, research and innovation with agrobiodiversity. Some future directions of research are recommended.}, keywords = {}, pubstate = {published}, tppubtype = {article} } This article explores the use of blockchain for agrobiodiversity (B4A) with a specific focus on (i) providing an overview of the existing regulatory challenges when it comes to conserving agrobiodiversity, which results in a lack of research and innovation when it comes to agrobiodiversity conserved in situ, (ii) investigating how a blockchain-based solution may help overcome these challenges, and (iii) illustrating how incentive mechanisms can help to overcome existing intellectual property regimes that prevent effective conservation, research and innovation (CRI). Our research identifies (i) lack of incentives, (ii) lack of trust among stakeholders, and (iii) lack of traceability options as main hindering reasons for in situ CRI with agrobiodiversity. Further, We find that blockchain solutions may empower data providers, including small farmers, to collectively track, control and monetize the use of data and assets shared, while minimizing fraudulent activities. Transaction costs may also be lowered by removing complex and expensive interaction processes. However, further research and development is necessary to design an ethical and sustainable blockchain-based solution to incentivize in situ conservation, research and innovation with agrobiodiversity. Some future directions of research are recommended. | |
Kochupillai, Mrinalini; Köninger, Julia 2021. @conference{Kochupillai2021c, title = {Creating a Digital Marketplace for Agrobiodiversity and Plant Genetic Sequence Data: Legal and Ethical Considerations of an AI and Blockchain Based Solution Sustainable Innovation in Plant Varieties View project TRIPs, Patent Law and Policy View project}, author = {Mrinalini Kochupillai and Julia Köninger}, url = {https://www.researchgate.net/publication/350546174}, year = {2021}, date = {2021-01-01}, keywords = {}, pubstate = {published}, tppubtype = {conference} } | |
Ahmed, Nouman; Saha, Sudipan; Mohsin, Maaz; Shahzad, Muhammad; Zhu, Xiao Xiang Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image Inproceedings International Conference on Computer Vision (ICCV), pp. 752–761, 2021. Abstract | Links | BibTeX | Tags: @inproceedings{Ahmed2021b, title = {Progressive Unsupervised Deep Transfer Learning for Forest Mapping in Satellite Image}, author = {Nouman Ahmed and Sudipan Saha and Maaz Mohsin and Muhammad Shahzad and Xiao Xiang Zhu}, url = {https://elib.dlr.de/145759/}, year = {2021}, date = {2021-01-01}, booktitle = {International Conference on Computer Vision (ICCV)}, journal = {ICCV 2021 workshop on Learning to Understand Aerial Images (LUAI)}, pages = {752--761}, abstract = {Automated forest mapping is important to understand our forests that play a key role in ecological system. However, efforts towards forest mapping is impeded by difficulty to collect labeled forest images that show large intraclass variation. Recently unsupervised learning has shown promising capability when exploiting limited labeled data. Motivated by this, we propose a progressive unsupervised deep transfer learning method for forest mapping. The proposed method exploits a pre-trained model that is subsequently fine-tuned over the target forest domain. We propose two different fine-tuning echanism, one works in a totally unsupervised setting by jointly learning the parameters of CNN and the k-means based cluster assignments of the resulting features and the other one works in a semi-supervised setting by exploiting the extracted k-nearest neighbor based pseudo labels. The proposed progressive scheme is evaluated on publicly available EuroSAT dataset using the relevant base model trained on BigEarth-Net labels. The results show that the proposed method greatly improves the forest regions classification accuracy as compared to the unsupervised baseline, nearly approaching the supervised classification approach.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Automated forest mapping is important to understand our forests that play a key role in ecological system. However, efforts towards forest mapping is impeded by difficulty to collect labeled forest images that show large intraclass variation. Recently unsupervised learning has shown promising capability when exploiting limited labeled data. Motivated by this, we propose a progressive unsupervised deep transfer learning method for forest mapping. The proposed method exploits a pre-trained model that is subsequently fine-tuned over the target forest domain. We propose two different fine-tuning echanism, one works in a totally unsupervised setting by jointly learning the parameters of CNN and the k-means based cluster assignments of the resulting features and the other one works in a semi-supervised setting by exploiting the extracted k-nearest neighbor based pseudo labels. The proposed progressive scheme is evaluated on publicly available EuroSAT dataset using the relevant base model trained on BigEarth-Net labels. The results show that the proposed method greatly improves the forest regions classification accuracy as compared to the unsupervised baseline, nearly approaching the supervised classification approach. | |
Thanthrige, Udaya Miriya S K P; Jung, Peter; Sezgin, Aydin Deep Unfolding of Iteratively Reweighted ADMM for Wireless RF Sensing Miscellaneous 2021. Abstract | Links | BibTeX | Tags: @misc{Thanthrige2021, title = {Deep Unfolding of Iteratively Reweighted ADMM for Wireless RF Sensing}, author = {Udaya Miriya S K P Thanthrige and Peter Jung and Aydin Sezgin}, url = {https://arxiv.org/abs/2106.03686v3}, doi = {10.48550/arxiv.2106.03686}, year = {2021}, date = {2021-01-01}, publisher = {arXiv}, abstract = {We address the detection of material defects, which are inside a layered material structure using compressive sensing based multiple-input and multiple-output (MIMO) wireless radar. Here, the strong clutter due to the reflection of the layered structure's surface often makes the detection of the defects challenging. Thus, sophisticated signal separation methods are required for improved defect detection. In many scenarios, the number of defects that we are interested in is limited and the signaling response of the layered structure can be modeled as a low-rank structure. Therefore, we propose joint rank and sparsity minimization for defect detection. In particular, we propose a non-convex approach based on the iteratively reweighted nuclear and $ell_1-$norm (a double-reweighted approach) to obtain a higher accuracy compared to the conventional nuclear norm and $ell_1-$norm minimization. To this end, an iterative algorithm is designed to estimate the low-rank and sparse contributions. Further, we propose deep learning to learn the parameters of the algorithm (i.e., algorithm unfolding) to improve the accuracy and the speed of convergence of the algorithm. Our numerical results show that the proposed approach outperforms the conventional approaches in terms of mean square errors of the recovered low-rank and sparse components and the speed of convergence.}, keywords = {}, pubstate = {published}, tppubtype = {misc} } We address the detection of material defects, which are inside a layered material structure using compressive sensing based multiple-input and multiple-output (MIMO) wireless radar. Here, the strong clutter due to the reflection of the layered structure's surface often makes the detection of the defects challenging. Thus, sophisticated signal separation methods are required for improved defect detection. In many scenarios, the number of defects that we are interested in is limited and the signaling response of the layered structure can be modeled as a low-rank structure. Therefore, we propose joint rank and sparsity minimization for defect detection. In particular, we propose a non-convex approach based on the iteratively reweighted nuclear and $ell_1-$norm (a double-reweighted approach) to obtain a higher accuracy compared to the conventional nuclear norm and $ell_1-$norm minimization. To this end, an iterative algorithm is designed to estimate the low-rank and sparse contributions. Further, we propose deep learning to learn the parameters of the algorithm (i.e., algorithm unfolding) to improve the accuracy and the speed of convergence of the algorithm. Our numerical results show that the proposed approach outperforms the conventional approaches in terms of mean square errors of the recovered low-rank and sparse components and the speed of convergence. | |
Basirat, Mina; Geiger, Bernhard; Roth, Peter M A Geometric Perspective on Information Plane Analysis Journal Article Entropy, 23 , pp. 711, 2021. @article{article, title = {A Geometric Perspective on Information Plane Analysis}, author = {Mina Basirat and Bernhard Geiger and Peter M Roth}, doi = {10.3390/e23060711}, year = {2021}, date = {2021-01-01}, journal = {Entropy}, volume = {23}, pages = {711}, keywords = {}, pubstate = {published}, tppubtype = {article} } |