Our project is done in close collaboration with the Technical University of Munich. In particular with the TUM Data Science in Earth Observation (Sipeo) group. The complete list of associated publications might be also interesting for you and is available here.
Publications
Publications
2022 |
|
Fang Xu Yilei Shi, Patrick Ebel Lei Yu Gui-Song Xia Wen Yang ; Zhu, Xiao Xiang GLF-CR: SAR-Enhanced Cloud Removal with Global-Local Fusion Journal Article Forthcoming ISPRS Journal of Photogrammetry and Remote Sensing, Forthcoming. @article{xu2022, title = {GLF-CR: SAR-Enhanced Cloud Removal with Global-Local Fusion}, author = {Fang Xu, Yilei Shi, Patrick Ebel,Lei Yu, Gui-Song Xia, Wen Yang, and Xiao Xiang Zhu}, year = {2022}, date = {2022-08-10}, journal = {ISPRS Journal of Photogrammetry and Remote Sensing}, keywords = {.}, pubstate = {forthcoming}, tppubtype = {article} } | |
Mrinalini Kochupillai Matthias Kahl, Michael Schmitt Hannes Taubenböck Xiao Xiang Zhu Artificial Intelligence for Earth Observation: Understanding Emerging Ethical Issues and Opportunities Journal Article Forthcoming IEEE Geoscience and Remote Sensing Magazine, Forthcoming. @article{kochupillai2022, title = {Artificial Intelligence for Earth Observation: Understanding Emerging Ethical Issues and Opportunities}, author = {Mrinalini Kochupillai, Matthias Kahl, Michael Schmitt, Hannes Taubenböck, Xiao Xiang Zhu}, year = {2022}, date = {2022-08-10}, journal = {IEEE Geoscience and Remote Sensing Magazine}, keywords = {.}, pubstate = {forthcoming}, tppubtype = {article} } | |
Qian, Kun; Wang, Yuanyuan; Shi, Yilei; Zhu, Xiao Xiang γ-Net: Superresolving SAR Tomographic Inversion via Deep Learning Journal Article Forthcoming IEEE Transactions on Geoscience and Remote Sensing, Forthcoming, (in press). @article{, title = {γ-Net: Superresolving SAR Tomographic Inversion via Deep Learning}, author = {Kun Qian and Yuanyuan Wang and Yilei Shi and Xiao Xiang Zhu}, year = {2022}, date = {2022-03-30}, booktitle = {IEEE Transactions on Geoscience and Remote Sensing}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, abstract = {(This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.) Synthetic aperture radar tomography (TomoSAR) has been extensively employed in 3-D reconstruction in dense urban areas using high-resolution SAR acquisitions. Compressive sensing (CS)-based algorithms are generally considered as the state of the art in super-resolving TomoSAR, in particular in the single look case. This superior performance comes at the cost of extra computational burdens, because of the sparse reconstruction, which cannot be solved analytically and we need to employ computationally expensive iterative solvers. In this paper, we propose a novel deep learning-based super-resolving TomoSAR inversion approach, γ-Net, to tackle this challenge. γ-Net adopts advanced complex-valued learned iterative shrinkage thresholding algorithm (CV-LISTA) to mimic the iterative optimization step in sparse reconstruction. Simulations show the height estimate from a well-trained γ-Net approaches the Cramér-Rao lower bound while improving the computational efficiency by 1 to 2 orders of magnitude comparing to the first-order CS-based methods. It also shows no degradation in the super-resolution power comparing to the state-of-the-art second-order TomoSAR solvers, which are much more computationally expensive than the first-order methods. Specifically, γ-Net reaches more than 90% detection rate in moderate super-resolving cases at 25 measurements at 6dB SNR. Moreover, simulation at limited baselines demonstrates that the proposed algorithm outperforms the second-order CS-based method by a fair margin. Test on real TerraSAR-X data with just 6 interferograms also shows high-quality 3-D reconstruction with high-density detected double scatterers.}, note = {in press}, keywords = {}, pubstate = {forthcoming}, tppubtype = {article} } (This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.) Synthetic aperture radar tomography (TomoSAR) has been extensively employed in 3-D reconstruction in dense urban areas using high-resolution SAR acquisitions. Compressive sensing (CS)-based algorithms are generally considered as the state of the art in super-resolving TomoSAR, in particular in the single look case. This superior performance comes at the cost of extra computational burdens, because of the sparse reconstruction, which cannot be solved analytically and we need to employ computationally expensive iterative solvers. In this paper, we propose a novel deep learning-based super-resolving TomoSAR inversion approach, γ-Net, to tackle this challenge. γ-Net adopts advanced complex-valued learned iterative shrinkage thresholding algorithm (CV-LISTA) to mimic the iterative optimization step in sparse reconstruction. Simulations show the height estimate from a well-trained γ-Net approaches the Cramér-Rao lower bound while improving the computational efficiency by 1 to 2 orders of magnitude comparing to the first-order CS-based methods. It also shows no degradation in the super-resolution power comparing to the state-of-the-art second-order TomoSAR solvers, which are much more computationally expensive than the first-order methods. Specifically, γ-Net reaches more than 90% detection rate in moderate super-resolving cases at 25 measurements at 6dB SNR. Moreover, simulation at limited baselines demonstrates that the proposed algorithm outperforms the second-order CS-based method by a fair margin. Test on real TerraSAR-X data with just 6 interferograms also shows high-quality 3-D reconstruction with high-density detected double scatterers. | |
Li, Tian; Dawson, Geoffrey J; Chuter, Stephen J; Bamber, Jonathan L A high-resolution Antarctic grounding zone product from ICESat-2 laser altimetry Journal Article Earth System Science Data, 14 , pp. 535-557, 2022, ISSN: 1866-3516. Abstract | Links | BibTeX | Tags: @article{Li2022, title = {A high-resolution Antarctic grounding zone product from ICESat-2 laser altimetry}, author = {Tian Li and Geoffrey J Dawson and Stephen J Chuter and Jonathan L Bamber}, url = {https://essd.copernicus.org/articles/14/535/2022/}, doi = {10.5194/essd-14-535-2022}, issn = {1866-3516}, year = {2022}, date = {2022-01-01}, journal = {Earth System Science Data}, volume = {14}, pages = {535-557}, abstract = { }, <p><![CDATA[Abstract. The Antarctic grounding zone, which is the transition between the fully grounded ice sheet to freely floating ice shelf, plays a critical role in ice sheet stability, mass budget calculations, and ice sheet model projections. It is therefore important to continuously monitor its location and migration over time. Here we present the first ICESat-2-derived high-resolution grounding zone product of the Antarctic Ice Sheet, including three important boundaries: the inland limit of tidal flexure (Point F), inshore limit of hydrostatic equilibrium (Point H), and the break in slope (Point Ib). This dataset was derived from automated techniques developed in this study, using ICESat-2 laser altimetry repeat tracks between 30 March 2019 and 30 September 2020. The new grounding zone product has a near-complete coverage of the Antarctic Ice Sheet with a total of 21 346 Point F, 18 149 Point H, and 36 765 Point Ib locations identified, including the difficult-to-survey grounding zones, such as the fast-flowing glaciers draining into the Amundsen Sea embayment. The locations of newly derived ICESat-2 landward limit of tidal flexure agree well with the most recent differential synthetic aperture radar interferometry (DInSAR) observations in 2018, with a mean absolute separation and standard deviation of 0.02 and 0.02 km, respectively. By comparing the ICESat-2-derived grounding zone with the previous grounding zone products, we find a grounding line retreat of up to 15 km on the Crary Ice Rise of Ross Ice Shelf and a pervasive landward grounding line migration along the Amundsen Sea embayment during the past 2 decades. We also identify the presence of ice plains on the Filchner–Ronne Ice Shelf and the influence of oscillating ocean tides on grounding zone migration. The product derived from this study is available at https://doi.org/10.5523/bris.bnqqyngt89eo26qk8keckglww (Li et al., 2021) and is archived and maintained at the National Snow and Ice Data Center.]]></p> | |
Lehmann, Fanny; Vishwakarma, Bramha Dutt; Bamber, Jonathan How well are we able to close the water budget at the global scale? Journal Article Hydrology and Earth System Sciences, 26 , pp. 35-54, 2022, ISSN: 1607-7938. Abstract | Links | BibTeX | Tags: @article{Lehmann2022, title = {How well are we able to close the water budget at the global scale?}, author = {Fanny Lehmann and Bramha Dutt Vishwakarma and Jonathan Bamber}, url = {https://hess.copernicus.org/articles/26/35/2022/}, doi = {10.5194/hess-26-35-2022}, issn = {1607-7938}, year = {2022}, date = {2022-01-01}, journal = {Hydrology and Earth System Sciences}, volume = {26}, pages = {35-54}, abstract = { Abstract. The water budget equation describes the exchange of water between the land, ocean, and atmosphere. Being able to adequately close the water budget gives confidence in our ability to model and/or observe the spatio-temporal variations in the water cycle and its components. Due to advances in observation techniques, satellite sensors, and modelling, a number of data products are available that represent the components of water budget in both space and time. Despite these advances, closure of the water budget at the global scale has been elusive. In this study, we attempt to close the global water budget using precipitation, evapotranspiration, and runoff data at the catchment scale. The large number of recent state-of-the-art datasets provides a new evaluation of well-used datasets. These estimates are compared to terrestrial water storage (TWS) changes as measured by the Gravity Recovery And Climate Experiment (GRACE) satellite mission. We investigated 189 river basins covering more than 90 % of the continental land area. TWS changes derived from the water balance equation were compared against GRACE data using two metrics: the Nash–Sutcliffe efficiency (NSE) and the cyclostationary NSE. These metrics were used to assess the performance of more than 1600 combinations of the various datasets considered. We found a positive NSE and cyclostationary NSE in 99 % and 62 % of the basins examined respectively. This means that TWS changes reconstructed from the water balance equation were more accurate than the long-term (NSE) and monthly (cyclostationary NSE) mean of GRACE time series in the corresponding basins. By analysing different combinations of the datasets that make up the water balance, we identified data products that performed well in certain regions based on, for example, climatic zone. We identified that some of the good results were obtained due to the cancellation of errors in poor estimates of water budget components. Therefore, we used coefficients of variation to determine the relative quality of a data product, which helped us to identify bad combinations giving us good results. In general, water budget components from ERA5-Land and the Catchment Land Surface Model (CLSM) performed better than other products for most climatic zones. Conversely, the latest version of CLSM, v2.2, performed poorly for evapotranspiration in snow-dominated catchments compared, for example, with its predecessor and other datasets available. Thus, the nature of the catchment dynamics and balance between components affects the optimum combination of datasets. For regional studies, the combination of datasets that provides the most realistic TWS for a basin will depend on its climatic conditions and factors that cannot be determined a priori. We believe that the results of this study provide a road map for studying the water budget at catchment scale. },keywords = {}, pubstate = {published}, tppubtype = {article} } <p>Abstract. The water budget equation describes the exchange of water between the land, ocean, and atmosphere. Being able to adequately close the water budget gives confidence in our ability to model and/or observe the spatio-temporal variations in the water cycle and its components. Due to advances in observation techniques, satellite sensors, and modelling, a number of data products are available that represent the components of water budget in both space and time. Despite these advances, closure of the water budget at the global scale has been elusive. In this study, we attempt to close the global water budget using precipitation, evapotranspiration, and runoff data at the catchment scale. The large number of recent state-of-the-art datasets provides a new evaluation of well-used datasets. These estimates are compared to terrestrial water storage (TWS) changes as measured by the Gravity Recovery And Climate Experiment (GRACE) satellite mission. We investigated 189 river basins covering more than 90 % of the continental land area. TWS changes derived from the water balance equation were compared against GRACE data using two metrics: the Nash–Sutcliffe efficiency (NSE) and the cyclostationary NSE. These metrics were used to assess the performance of more than 1600 combinations of the various datasets considered. We found a positive NSE and cyclostationary NSE in 99 % and 62 % of the basins examined respectively. This means that TWS changes reconstructed from the water balance equation were more accurate than the long-term (NSE) and monthly (cyclostationary NSE) mean of GRACE time series in the corresponding basins. By analysing different combinations of the datasets that make up the water balance, we identified data products that performed well in certain regions based on, for example, climatic zone. We identified that some of the good results were obtained due to the cancellation of errors in poor estimates of water budget components. Therefore, we used coefficients of variation to determine the relative quality of a data product, which helped us to identify bad combinations giving us good results. In general, water budget components from ERA5-Land and the Catchment Land Surface Model (CLSM) performed better than other products for most climatic zones. Conversely, the latest version of CLSM, v2.2, performed poorly for evapotranspiration in snow-dominated catchments compared, for example, with its predecessor and other datasets available. Thus, the nature of the catchment dynamics and balance between components affects the optimum combination of datasets. For regional studies, the combination of datasets that provides the most realistic TWS for a basin will depend on its climatic conditions and factors that cannot be determined a priori. We believe that the results of this study provide a road map for studying the water budget at catchment scale.</p> | |
Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang An Advanced Dirichlet Prior Network for Out-of-Distribution Detection in Remote Sensing Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2022. BibTeX | Tags: @article{Gawlikowski2022, title = {An Advanced Dirichlet Prior Network for Out-of-Distribution Detection in Remote Sensing}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Saha, Sudipan; Zhao, Shan; Zhu, Xiao Xiang Multi-target domain adaptation for remote sensing classification using graph neural network Journal Article IEEE Geoscience and Remote Sensing Letters, 2022. BibTeX | Tags: @article{Saha2022, title = {Multi-target domain adaptation for remote sensing classification using graph neural network}, author = {Sudipan Saha and Shan Zhao and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Roschlaub, Robert; Glock, Clemens; Möst, Karin; Hümmer, Frank; Li, Qingyu; Auer, Stefan; Kruspe, Anna; Zhu, Xiao Xiang Implementierung einer KI-Infrastruktur zur automatisierten Erkennung von landesweiten Gebäudeveränderungen aus Luftbildern Journal Article ZFV - Zeitschrift für Geodasie, Geoinformation und Landmanagement, 2022. Abstract | Links | BibTeX | Tags: @article{Roschlaub2022, title = {Implementierung einer KI-Infrastruktur zur automatisierten Erkennung von landesweiten Gebäudeveränderungen aus Luftbildern}, author = {Robert Roschlaub and Clemens Glock and Karin Möst and Frank Hümmer and Qingyu Li and Stefan Auer and Anna Kruspe and Xiao Xiang Zhu}, url = {https://elib.dlr.de/185326/}, year = {2022}, date = {2022-01-01}, journal = {ZFV - Zeitschrift für Geodasie, Geoinformation und Landmanagement}, publisher = {Wißner-Verlag}, abstract = {Aufbauend auf einer vorangegangenen Studie über den Einsatz von Künstlicher Intelligenz (KI) zur Detektion von Gebäudeveränderungen im amtlichen Liegenschaftskataster, wird in diesem Beitrag die Transformation von der Projektphase in einen Produktivbetrieb vorgestellt, es werden verschiedene KI-Architekturen angesprochen, miteinander verglichen und eine Auswahlentscheidung gegeben. Die erzielten Ergebnisse einer landesweiten Gebäudedetektion werden gemeinsam mit Untersuchungen zu den Trainingsläufen, zur Performance und den von einem Vermessungsamt validierten KI-Ergebnissen präsentiert.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Aufbauend auf einer vorangegangenen Studie über den Einsatz von Künstlicher Intelligenz (KI) zur Detektion von Gebäudeveränderungen im amtlichen Liegenschaftskataster, wird in diesem Beitrag die Transformation von der Projektphase in einen Produktivbetrieb vorgestellt, es werden verschiedene KI-Architekturen angesprochen, miteinander verglichen und eine Auswahlentscheidung gegeben. Die erzielten Ergebnisse einer landesweiten Gebäudedetektion werden gemeinsam mit Untersuchungen zu den Trainingsläufen, zur Performance und den von einem Vermessungsamt validierten KI-Ergebnissen präsentiert. | |
Stadtler, Scarlet; Betancourt, Clara; Roscher, Ribana Explainable Machine Learning Reveals Capabilities, Redundancy, and Limitations of a Geospatial Air Quality Benchmark Dataset Journal Article Machine Learning and Knowledge Extraction, 4 , pp. 150-171, 2022, ISSN: 2504-4990. Abstract | Links | BibTeX | Tags: @article{Stadtler2022, title = {Explainable Machine Learning Reveals Capabilities, Redundancy, and Limitations of a Geospatial Air Quality Benchmark Dataset}, author = {Scarlet Stadtler and Clara Betancourt and Ribana Roscher}, url = {https://www.mdpi.com/2504-4990/4/1/8}, doi = {10.3390/make4010008}, issn = {2504-4990}, year = {2022}, date = {2022-01-01}, journal = {Machine Learning and Knowledge Extraction}, volume = {4}, pages = {150-171}, publisher = {Multidisciplinary Digital Publishing Institute}, abstract = { Air quality is relevant to society because it poses environmental risks to humans and nature. We use explainable machine learning in air quality research by analyzing model predictions in relation to the underlying training data. The data originate from worldwide ozone observations, paired with geospatial data. We use two different architectures: a neural network and a random forest trained on various geospatial data to predict multi-year averages of the air pollutant ozone. To understand how both models function, we explain how they represent the training data and derive their predictions. By focusing on inaccurate predictions and explaining why these predictions fail, we can (i) identify underrepresented samples, (ii) flag unexpected inaccurate predictions, and (iii) point to training samples irrelevant for predictions on the test set. Based on the underrepresented samples, we suggest where to build new measurement stations. We also show which training samples do not substantially contribute to the model performance. This study demonstrates the application of explainable machine learning beyond simply explaining the trained model. },keywords = {}, pubstate = {published}, tppubtype = {article} } <p>Air quality is relevant to society because it poses environmental risks to humans and nature. We use explainable machine learning in air quality research by analyzing model predictions in relation to the underlying training data. The data originate from worldwide ozone observations, paired with geospatial data. We use two different architectures: a neural network and a random forest trained on various geospatial data to predict multi-year averages of the air pollutant ozone. To understand how both models function, we explain how they represent the training data and derive their predictions. By focusing on inaccurate predictions and explaining why these predictions fail, we can (i) identify underrepresented samples, (ii) flag unexpected inaccurate predictions, and (iii) point to training samples irrelevant for predictions on the test set. Based on the underrepresented samples, we suggest where to build new measurement stations. We also show which training samples do not substantially contribute to the model performance. This study demonstrates the application of explainable machine learning beyond simply explaining the trained model.</p> | |
Betancourt, C; Stomberg, T T; Edrich, A -K; Patnala, A; Schultz, M G; Roscher, R; Kowalski, J; Stadtler, S Global, high-resolution mapping of tropospheric ozone – explainable machine learning and impact of uncertainties Journal Article Geoscientific Model Development Discussions, 2022 , pp. 1-36, 2022. @article{Betancourt2022, title = {Global, high-resolution mapping of tropospheric ozone – explainable machine learning and impact of uncertainties}, author = {C Betancourt and T T Stomberg and A -K Edrich and A Patnala and M G Schultz and R Roscher and J Kowalski and S Stadtler}, url = {https://gmd.copernicus.org/preprints/gmd-2022-2/}, doi = {10.5194/gmd-2022-2}, year = {2022}, date = {2022-01-01}, journal = {Geoscientific Model Development Discussions}, volume = {2022}, pages = {1-36}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Li, Tian; Dawson, Geoffrey J; Chuter, Stephen J; Bamber, Jonathan L A high-resolution Antarctic grounding zone product from ICESat-2 laser altimetry Journal Article Earth System Science Data, 14 , pp. 535–557, 2022, ISSN: 1866-3516. Abstract | Links | BibTeX | Tags: @article{Li2022b, title = {A high-resolution Antarctic grounding zone product from ICESat-2 laser altimetry}, author = {Tian Li and Geoffrey J Dawson and Stephen J Chuter and Jonathan L Bamber}, url = {https://essd.copernicus.org/articles/14/535/2022/}, doi = {10.5194/essd-14-535-2022}, issn = {1866-3516}, year = {2022}, date = {2022-01-01}, journal = {Earth System Science Data}, volume = {14}, pages = {535--557}, abstract = { }, <p><![CDATA[Abstract. The Antarctic grounding zone, which is the transition between the fully grounded ice sheet to freely floating ice shelf, plays a critical role in ice sheet stability, mass budget calculations, and ice sheet model projections. It is therefore important to continuously monitor its location and migration over time. Here we present the first ICESat-2-derived high-resolution grounding zone product of the Antarctic Ice Sheet, including three important boundaries: the inland limit of tidal flexure (Point F), inshore limit of hydrostatic equilibrium (Point H), and the break in slope (Point Ib). This dataset was derived from automated techniques developed in this study, using ICESat-2 laser altimetry repeat tracks between 30 March 2019 and 30 September 2020. The new grounding zone product has a near-complete coverage of the Antarctic Ice Sheet with a total of 21 346 Point F, 18 149 Point H, and 36 765 Point Ib locations identified, including the difficult-to-survey grounding zones, such as the fast-flowing glaciers draining into the Amundsen Sea embayment. The locations of newly derived ICESat-2 landward limit of tidal flexure agree well with the most recent differential synthetic aperture radar interferometry (DInSAR) observations in 2018, with a mean absolute separation and standard deviation of 0.02 and 0.02 km, respectively. By comparing the ICESat-2-derived grounding zone with the previous grounding zone products, we find a grounding line retreat of up to 15 km on the Crary Ice Rise of Ross Ice Shelf and a pervasive landward grounding line migration along the Amundsen Sea embayment during the past 2 decades. We also identify the presence of ice plains on the Filchner–Ronne Ice Shelf and the influence of oscillating ocean tides on grounding zone migration. The product derived from this study is available at https://doi.org/10.5523/bris.bnqqyngt89eo26qk8keckglww (Li et al., 2021) and is archived and maintained at the National Snow and Ice Data Center.]]></p> | |
Gawlikowski, Jakob; Saha, Sudipan; Kruspe, Anna; Zhu, Xiao Xiang An Advanced Dirichlet Prior Network for Out-of-Distribution Detection in Remote Sensing Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2022. BibTeX | Tags: @article{Gawlikowski2022b, title = {An Advanced Dirichlet Prior Network for Out-of-Distribution Detection in Remote Sensing}, author = {Jakob Gawlikowski and Sudipan Saha and Anna Kruspe and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Roschlaub, Robert; Glock, Clemens; Möst, Karin; Hümmer, Frank; Li, Qingyu; Auer, Stefan; Kruspe, Anna; Zhu, Xiao Xiang Implementierung einer KI-Infrastruktur zur automatisierten Erkennung von landesweiten Gebäudeveränderungen aus Luftbildern Journal Article ZFV - Zeitschrift für Geodasie, Geoinformation und Landmanagement, 2022. Abstract | Links | BibTeX | Tags: @article{Roschlaub2022b, title = {Implementierung einer KI-Infrastruktur zur automatisierten Erkennung von landesweiten Gebäudeveränderungen aus Luftbildern}, author = {Robert Roschlaub and Clemens Glock and Karin Möst and Frank Hümmer and Qingyu Li and Stefan Auer and Anna Kruspe and Xiao Xiang Zhu}, url = {https://elib.dlr.de/185326/}, year = {2022}, date = {2022-01-01}, journal = {ZFV - Zeitschrift für Geodasie, Geoinformation und Landmanagement}, publisher = {Wißner-Verlag}, abstract = {Aufbauend auf einer vorangegangenen Studie über den Einsatz von Künstlicher Intelligenz (KI) zur Detektion von Gebäudeveränderungen im amtlichen Liegenschaftskataster, wird in diesem Beitrag die Transformation von der Projektphase in einen Produktivbetrieb vorgestellt, es werden verschiedene KI-Architekturen angesprochen, miteinander verglichen und eine Auswahlentscheidung gegeben. Die erzielten Ergebnisse einer landesweiten Gebäudedetektion werden gemeinsam mit Untersuchungen zu den Trainingsläufen, zur Performance und den von einem Vermessungsamt validierten KI-Ergebnissen präsentiert.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Aufbauend auf einer vorangegangenen Studie über den Einsatz von Künstlicher Intelligenz (KI) zur Detektion von Gebäudeveränderungen im amtlichen Liegenschaftskataster, wird in diesem Beitrag die Transformation von der Projektphase in einen Produktivbetrieb vorgestellt, es werden verschiedene KI-Architekturen angesprochen, miteinander verglichen und eine Auswahlentscheidung gegeben. Die erzielten Ergebnisse einer landesweiten Gebäudedetektion werden gemeinsam mit Untersuchungen zu den Trainingsläufen, zur Performance und den von einem Vermessungsamt validierten KI-Ergebnissen präsentiert. | |
Khan, Shfaqat A; Bamber, Jonathan L; Rignot, Eric; Helm, Veit; Aschwanden, Andy; Holland, David M; van den Broeke, Michiel; King, Michalea; Noël, Brice; Truffer, Martin; Humbert, Angelika; Colgan, William; Vijay, Saurabh; Munneke, Peter Kuipers Greenland Mass Trends From Airborne and Satellite Altimetry During 2011–2020 Journal Article Journal of Geophysical Research: Earth Surface, 127 , pp. e2021JF006505, 2022, ISSN: 2169-9011. @article{Khan2022, title = {Greenland Mass Trends From Airborne and Satellite Altimetry During 2011–2020}, author = {Shfaqat A Khan and Jonathan L Bamber and Eric Rignot and Veit Helm and Andy Aschwanden and David M Holland and Michiel van den Broeke and Michalea King and Brice Noël and Martin Truffer and Angelika Humbert and William Colgan and Saurabh Vijay and Peter Kuipers Munneke}, url = {https://onlinelibrary.wiley.com/doi/full/10.1029/2021JF006505 https://onlinelibrary.wiley.com/doi/abs/10.1029/2021JF006505 https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2021JF006505}, doi = {10.1029/2021JF006505}, issn = {2169-9011}, year = {2022}, date = {2022-01-01}, journal = {Journal of Geophysical Research: Earth Surface}, volume = {127}, pages = {e2021JF006505}, publisher = {John Wiley & Sons, Ltd}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Hua, Yuansheng; Mou, Lichao; Jin, Pu; Zhu, Xiao Xiang MultiScene: A Large-Scale Dataset and Benchmark for Multiscene Recognition in Single Aerial Images Journal Article IEEE Transactions on Geoscience and Remote Sensing, 60 , 2022, ISSN: 15580644. Abstract | Links | BibTeX | Tags: @article{Hua2022, title = {MultiScene: A Large-Scale Dataset and Benchmark for Multiscene Recognition in Single Aerial Images}, author = {Yuansheng Hua and Lichao Mou and Pu Jin and Xiao Xiang Zhu}, doi = {10.1109/TGRS.2021.3110314}, issn = {15580644}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, volume = {60}, publisher = {Institute of Electrical and Electronics Engineers Inc.}, abstract = {Aerial scene recognition is a fundamental research problem in interpreting high-resolution aerial imagery. Over the past few years, most studies focus on classifying an image into one scene category, while in real-world scenarios, it is more often that a single image contains multiple scenes. Therefore, in this article, we investigate a more practical yet underexplored task—multiscene recognition in single images. To this end, we create a large-scale dataset, called MultiScene, composed of 100 000 unconstrained high-resolution aerial images. Considering that manually labeling such images is extremely arduous, we resort to low-cost annotations from crowdsourcing platforms, e.g., OpenStreetMap (OSM). However, OSM data might suffer from incompleteness and incorrectness, which introduce noise into image labels. To address this issue, we visually inspect 14 000 images and correct their scene labels, yielding a subset of cleanly annotated images, named MultiScene-Clean. With it, we can develop and evaluate deep networks for multiscene recognition using clean data. Moreover, we provide crowdsourced annotations of all images for the purpose of studying network learning with noisy labels. We conduct experiments with extensive baseline models on both MultiScene-Clean and MultiScene to offer benchmarks for multiscene recognition in single images and learning from noisy labels for this task, respectively. To facilitate progress, we make our dataset and trained models available on https://gitlab.lrz.de/ai4eo/reasoning/multiscene.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Aerial scene recognition is a fundamental research problem in interpreting high-resolution aerial imagery. Over the past few years, most studies focus on classifying an image into one scene category, while in real-world scenarios, it is more often that a single image contains multiple scenes. Therefore, in this article, we investigate a more practical yet underexplored task—multiscene recognition in single images. To this end, we create a large-scale dataset, called MultiScene, composed of 100 000 unconstrained high-resolution aerial images. Considering that manually labeling such images is extremely arduous, we resort to low-cost annotations from crowdsourcing platforms, e.g., OpenStreetMap (OSM). However, OSM data might suffer from incompleteness and incorrectness, which introduce noise into image labels. To address this issue, we visually inspect 14 000 images and correct their scene labels, yielding a subset of cleanly annotated images, named MultiScene-Clean. With it, we can develop and evaluate deep networks for multiscene recognition using clean data. Moreover, we provide crowdsourced annotations of all images for the purpose of studying network learning with noisy labels. We conduct experiments with extensive baseline models on both MultiScene-Clean and MultiScene to offer benchmarks for multiscene recognition in single images and learning from noisy labels for this task, respectively. To facilitate progress, we make our dataset and trained models available on https://gitlab.lrz.de/ai4eo/reasoning/multiscene. | |
Cummings, Sol; Kondmann, Lukas; Zhu, Xiaoxiang Siamese Attention U-Net for Multi-Class Change Detection. Inproceedings 2022. BibTeX | Tags: @inproceedings{Cummings2022, title = {Siamese Attention U-Net for Multi-Class Change Detection.}, author = {Sol Cummings and Lukas Kondmann and Xiaoxiang Zhu}, year = {2022}, date = {2022-01-01}, journal = {2022 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
EPFL, Jonathan Sauder; Genzel, Martin; Berlin, Peter TU Jung Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery Miscellaneous 2022. Abstract | Links | BibTeX | Tags: @misc{JonathanSauder2022, title = {Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery}, author = {Jonathan Sauder EPFL and Martin Genzel and Peter TU Jung Berlin}, url = {https://arxiv.org/abs/2202.03391v1}, doi = {10.48550/arxiv.2202.03391}, year = {2022}, date = {2022-01-01}, abstract = {Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and often even discrete optimization task. While the potential of gradient-based learning via the unrolling of iterative recovery algorithms has been demonstrated, it has remained unclear how to leverage this technique when the set of admissible measurement operators is structured and discrete. We tackle this problem by combining unrolled optimization with Gumbel reparametrizations, which enable the computation of low-variance gradient estimates of categorical random variables. Our approach is formalized by GLODISMO (Gradient-based Learning of DIscrete Structured Measurement Operators). This novel method is easy-to-implement, computationally efficient, and extendable due to its compatibility with automatic differentiation. We empirically demonstrate the performance and flexibility of GLODISMO in several prototypical signal recovery applications, verifying that the learned measurement matrices outperform conventional designs based on randomization as well as discrete optimization baselines.}, keywords = {}, pubstate = {published}, tppubtype = {misc} } Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and often even discrete optimization task. While the potential of gradient-based learning via the unrolling of iterative recovery algorithms has been demonstrated, it has remained unclear how to leverage this technique when the set of admissible measurement operators is structured and discrete. We tackle this problem by combining unrolled optimization with Gumbel reparametrizations, which enable the computation of low-variance gradient estimates of categorical random variables. Our approach is formalized by GLODISMO (Gradient-based Learning of DIscrete Structured Measurement Operators). This novel method is easy-to-implement, computationally efficient, and extendable due to its compatibility with automatic differentiation. We empirically demonstrate the performance and flexibility of GLODISMO in several prototypical signal recovery applications, verifying that the learned measurement matrices outperform conventional designs based on randomization as well as discrete optimization baselines. | |
Lehmann, Fanny; Vishwakarma, Bramha Dutt; Bamber, Jonathan How well are we able to close the water budget at the global scale? Journal Article Hydrology and Earth System Sciences, 26 , pp. 35-54, 2022, ISSN: 1607-7938. Abstract | Links | BibTeX | Tags: @article{Lehmann2022b, title = {How well are we able to close the water budget at the global scale?}, author = {Fanny Lehmann and Bramha Dutt Vishwakarma and Jonathan Bamber}, url = {https://hess.copernicus.org/articles/26/35/2022/}, doi = {10.5194/hess-26-35-2022}, issn = {1607-7938}, year = {2022}, date = {2022-01-01}, journal = {Hydrology and Earth System Sciences}, volume = {26}, pages = {35-54}, publisher = {Copernicus GmbH}, abstract = { Abstract. The water budget equation describes the exchange of water between the land, ocean, and atmosphere. Being able to adequately close the water budget gives confidence in our ability to model and/or observe the spatio-temporal variations in the water cycle and its components. Due to advances in observation techniques, satellite sensors, and modelling, a number of data products are available that represent the components of water budget in both space and time. Despite these advances, closure of the water budget at the global scale has been elusive. In this study, we attempt to close the global water budget using precipitation, evapotranspiration, and runoff data at the catchment scale. The large number of recent state-of-the-art datasets provides a new evaluation of well-used datasets. These estimates are compared to terrestrial water storage (TWS) changes as measured by the Gravity Recovery And Climate Experiment (GRACE) satellite mission. We investigated 189 river basins covering more than 90 % of the continental land area. TWS changes derived from the water balance equation were compared against GRACE data using two metrics: the Nash–Sutcliffe efficiency (NSE) and the cyclostationary NSE. These metrics were used to assess the performance of more than 1600 combinations of the various datasets considered. We found a positive NSE and cyclostationary NSE in 99 % and 62 % of the basins examined respectively. This means that TWS changes reconstructed from the water balance equation were more accurate than the long-term (NSE) and monthly (cyclostationary NSE) mean of GRACE time series in the corresponding basins. By analysing different combinations of the datasets that make up the water balance, we identified data products that performed well in certain regions based on, for example, climatic zone. We identified that some of the good results were obtained due to the cancellation of errors in poor estimates of water budget components. Therefore, we used coefficients of variation to determine the relative quality of a data product, which helped us to identify bad combinations giving us good results. In general, water budget components from ERA5-Land and the Catchment Land Surface Model (CLSM) performed better than other products for most climatic zones. Conversely, the latest version of CLSM, v2.2, performed poorly for evapotranspiration in snow-dominated catchments compared, for example, with its predecessor and other datasets available. Thus, the nature of the catchment dynamics and balance between components affects the optimum combination of datasets. For regional studies, the combination of datasets that provides the most realistic TWS for a basin will depend on its climatic conditions and factors that cannot be determined a priori. We believe that the results of this study provide a road map for studying the water budget at catchment scale. },keywords = {}, pubstate = {published}, tppubtype = {article} } <p>Abstract. The water budget equation describes the exchange of water between the land, ocean, and atmosphere. Being able to adequately close the water budget gives confidence in our ability to model and/or observe the spatio-temporal variations in the water cycle and its components. Due to advances in observation techniques, satellite sensors, and modelling, a number of data products are available that represent the components of water budget in both space and time. Despite these advances, closure of the water budget at the global scale has been elusive. In this study, we attempt to close the global water budget using precipitation, evapotranspiration, and runoff data at the catchment scale. The large number of recent state-of-the-art datasets provides a new evaluation of well-used datasets. These estimates are compared to terrestrial water storage (TWS) changes as measured by the Gravity Recovery And Climate Experiment (GRACE) satellite mission. We investigated 189 river basins covering more than 90 % of the continental land area. TWS changes derived from the water balance equation were compared against GRACE data using two metrics: the Nash–Sutcliffe efficiency (NSE) and the cyclostationary NSE. These metrics were used to assess the performance of more than 1600 combinations of the various datasets considered. We found a positive NSE and cyclostationary NSE in 99 % and 62 % of the basins examined respectively. This means that TWS changes reconstructed from the water balance equation were more accurate than the long-term (NSE) and monthly (cyclostationary NSE) mean of GRACE time series in the corresponding basins. By analysing different combinations of the datasets that make up the water balance, we identified data products that performed well in certain regions based on, for example, climatic zone. We identified that some of the good results were obtained due to the cancellation of errors in poor estimates of water budget components. Therefore, we used coefficients of variation to determine the relative quality of a data product, which helped us to identify bad combinations giving us good results. In general, water budget components from ERA5-Land and the Catchment Land Surface Model (CLSM) performed better than other products for most climatic zones. Conversely, the latest version of CLSM, v2.2, performed poorly for evapotranspiration in snow-dominated catchments compared, for example, with its predecessor and other datasets available. Thus, the nature of the catchment dynamics and balance between components affects the optimum combination of datasets. For regional studies, the combination of datasets that provides the most realistic TWS for a basin will depend on its climatic conditions and factors that cannot be determined a priori. We believe that the results of this study provide a road map for studying the water budget at catchment scale.</p> | |
Saha, Sudipan; Zhao, Shan; Zhu, Xiao Xiang Multi-target domain adaptation for remote sensing classification using graph neural network Journal Article IEEE Geoscience and Remote Sensing Letters, 2022. BibTeX | Tags: @article{Saha2022b, title = {Multi-target domain adaptation for remote sensing classification using graph neural network}, author = {Sudipan Saha and Shan Zhao and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Stadtler, Scarlet; Betancourt, Clara; Roscher, Ribana Explainable Machine Learning Reveals Capabilities, Redundancy, and Limitations of a Geospatial Air Quality Benchmark Dataset Journal Article Machine Learning and Knowledge Extraction, 4 , pp. 150-171, 2022, ISSN: 2504-4990. Abstract | Links | BibTeX | Tags: @article{Stadtler2022b, title = {Explainable Machine Learning Reveals Capabilities, Redundancy, and Limitations of a Geospatial Air Quality Benchmark Dataset}, author = {Scarlet Stadtler and Clara Betancourt and Ribana Roscher}, url = {https://www.mdpi.com/2504-4990/4/1/8}, doi = {10.3390/make4010008}, issn = {2504-4990}, year = {2022}, date = {2022-01-01}, journal = {Machine Learning and Knowledge Extraction}, volume = {4}, pages = {150-171}, publisher = {Multidisciplinary Digital Publishing Institute}, abstract = { Air quality is relevant to society because it poses environmental risks to humans and nature. We use explainable machine learning in air quality research by analyzing model predictions in relation to the underlying training data. The data originate from worldwide ozone observations, paired with geospatial data. We use two different architectures: a neural network and a random forest trained on various geospatial data to predict multi-year averages of the air pollutant ozone. To understand how both models function, we explain how they represent the training data and derive their predictions. By focusing on inaccurate predictions and explaining why these predictions fail, we can (i) identify underrepresented samples, (ii) flag unexpected inaccurate predictions, and (iii) point to training samples irrelevant for predictions on the test set. Based on the underrepresented samples, we suggest where to build new measurement stations. We also show which training samples do not substantially contribute to the model performance. This study demonstrates the application of explainable machine learning beyond simply explaining the trained model. },keywords = {}, pubstate = {published}, tppubtype = {article} } <p>Air quality is relevant to society because it poses environmental risks to humans and nature. We use explainable machine learning in air quality research by analyzing model predictions in relation to the underlying training data. The data originate from worldwide ozone observations, paired with geospatial data. We use two different architectures: a neural network and a random forest trained on various geospatial data to predict multi-year averages of the air pollutant ozone. To understand how both models function, we explain how they represent the training data and derive their predictions. By focusing on inaccurate predictions and explaining why these predictions fail, we can (i) identify underrepresented samples, (ii) flag unexpected inaccurate predictions, and (iii) point to training samples irrelevant for predictions on the test set. Based on the underrepresented samples, we suggest where to build new measurement stations. We also show which training samples do not substantially contribute to the model performance. This study demonstrates the application of explainable machine learning beyond simply explaining the trained model.</p> | |
Mou, Lichao; Saha, Sudipan; Hua, Yuansheng; Bovolo, Francesca; Bruzzone, Lorenzo; Zhu, Xiao Xiang Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification Journal Article IEEE Transactions on Geoscience and Remote Sensing, 60 , 2022, ISSN: 15580644. Abstract | Links | BibTeX | Tags: @article{Mou2022, title = {Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification}, author = {Lichao Mou and Sudipan Saha and Yuansheng Hua and Francesca Bovolo and Lorenzo Bruzzone and Xiao Xiang Zhu}, doi = {10.1109/TGRS.2021.3067096}, issn = {15580644}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, volume = {60}, publisher = {Institute of Electrical and Electronics Engineers Inc.}, abstract = {Band selection refers to the process of choosing the most relevant bands in a hyperspectral image. By selecting a limited number of optimal bands, we aim at speeding up model training, improving accuracy, or both. It reduces redundancy among spectral bands while trying to preserve the original information of the image. By now, many efforts have been made to develop unsupervised band selection approaches, of which the majorities are heuristic algorithms devised by trial and error. In this article, we are interested in training an intelligent agent that, given a hyperspectral image, is capable of automatically learning policy to select an optimal band subset without any hand-engineered reasoning. To this end, we frame the problem of unsupervised band selection as a Markov decision process, propose an effective method to parameterize it, and finally solve the problem by deep reinforcement learning. Once the agent is trained, it learns a band-selection policy that guides the agent to sequentially select bands by fully exploiting the hyperspectral image and previously picked bands. Furthermore, we propose two different reward schemes for the environment simulation of deep reinforcement learning and compare them in experiments. This, to the best of our knowledge, is the first study that explores a deep reinforcement learning model for hyperspectral image analysis, thus opening a new door for future research and showcasing the great potential of deep reinforcement learning in remote sensing applications. Extensive experiments are carried out on four hyperspectral data sets, and experimental results demonstrate the effectiveness of the proposed method. The code is publicly available.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Band selection refers to the process of choosing the most relevant bands in a hyperspectral image. By selecting a limited number of optimal bands, we aim at speeding up model training, improving accuracy, or both. It reduces redundancy among spectral bands while trying to preserve the original information of the image. By now, many efforts have been made to develop unsupervised band selection approaches, of which the majorities are heuristic algorithms devised by trial and error. In this article, we are interested in training an intelligent agent that, given a hyperspectral image, is capable of automatically learning policy to select an optimal band subset without any hand-engineered reasoning. To this end, we frame the problem of unsupervised band selection as a Markov decision process, propose an effective method to parameterize it, and finally solve the problem by deep reinforcement learning. Once the agent is trained, it learns a band-selection policy that guides the agent to sequentially select bands by fully exploiting the hyperspectral image and previously picked bands. Furthermore, we propose two different reward schemes for the environment simulation of deep reinforcement learning and compare them in experiments. This, to the best of our knowledge, is the first study that explores a deep reinforcement learning model for hyperspectral image analysis, thus opening a new door for future research and showcasing the great potential of deep reinforcement learning in remote sensing applications. Extensive experiments are carried out on four hyperspectral data sets, and experimental results demonstrate the effectiveness of the proposed method. The code is publicly available. | |
Saha, Sudipan; Ebel, Patrick; Zhu, Xiao Xiang Self-Supervised Multisensor Change Detection Journal Article IEEE Transactions on Geoscience and Remote Sensing, 60 , 2022, ISSN: 15580644. Abstract | Links | BibTeX | Tags: @article{Saha2022c, title = {Self-Supervised Multisensor Change Detection}, author = {Sudipan Saha and Patrick Ebel and Xiao Xiang Zhu}, doi = {10.1109/TGRS.2021.3109957}, issn = {15580644}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, volume = {60}, publisher = {Institute of Electrical and Electronics Engineers Inc.}, abstract = {Most change detection (CD) methods assume that prechange and postchange images are acquired by the same sensor. However, in many real-life scenarios, e.g., natural disasters, it is more practical to use the latest available images before and after the occurrence of incidence, which may be acquired using different sensors. In particular, we are interested in the combination of the images acquired by optical and synthetic aperture radar (SAR) sensors. SAR images appear vastly different from the optical images even when capturing the same scene. Adding to this, CD methods are often constrained to use only target image-pair, no labeled data, and no additional unlabeled data. Such constraints limit the scope of traditional supervised machine learning and unsupervised generative approaches for multisensor CD. The recent rapid development of self-supervised learning methods has shown that some of them can even work with only few images. Motivated by this, in this work, we propose a method for multisensor CD using only the unlabeled target bitemporal images that are used for training a network in a self-supervised fashion by using deep clustering and contrastive learning. The proposed method is evaluated on four multimodal bitemporal scenes showing change, and the benefits of our self-supervised approach are demonstrated. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/sarOpticalMultisensorTgrs2021.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Most change detection (CD) methods assume that prechange and postchange images are acquired by the same sensor. However, in many real-life scenarios, e.g., natural disasters, it is more practical to use the latest available images before and after the occurrence of incidence, which may be acquired using different sensors. In particular, we are interested in the combination of the images acquired by optical and synthetic aperture radar (SAR) sensors. SAR images appear vastly different from the optical images even when capturing the same scene. Adding to this, CD methods are often constrained to use only target image-pair, no labeled data, and no additional unlabeled data. Such constraints limit the scope of traditional supervised machine learning and unsupervised generative approaches for multisensor CD. The recent rapid development of self-supervised learning methods has shown that some of them can even work with only few images. Motivated by this, in this work, we propose a method for multisensor CD using only the unlabeled target bitemporal images that are used for training a network in a self-supervised fashion by using deep clustering and contrastive learning. The proposed method is evaluated on four multimodal bitemporal scenes showing change, and the benefits of our self-supervised approach are demonstrated. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/sarOpticalMultisensorTgrs2021. | |
Betancourt, C; Stomberg, T T; Edrich, A -K; Patnala, A; Schultz, M G; Roscher, R; Kowalski, J; Stadtler, S GMDD - Global, high-resolution mapping of tropospheric ozone – explainable machine learning and impact of uncertainties Journal Article Geosci. Model Dev., pp. 1-36, 2022. @article{Betancourt2022b, title = {GMDD - Global, high-resolution mapping of tropospheric ozone – explainable machine learning and impact of uncertainties}, author = {C Betancourt and T T Stomberg and A -K Edrich and A Patnala and M G Schultz and R Roscher and J Kowalski and S Stadtler}, url = {https://gmd.copernicus.org/preprints/gmd-2022-2/}, doi = {10.5194/gmd-2022-2}, year = {2022}, date = {2022-01-01}, journal = {Geosci. Model Dev.}, pages = {1-36}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Song, Qian; Albrecht, Conrad M; Xiong, Zhitong; Zhu, Xiao Xiang Towards Global Forest Biomass estimators from tree height data Inproceedings 2022. BibTeX | Tags: @inproceedings{Song2022, title = {Towards Global Forest Biomass estimators from tree height data}, author = {Qian Song and Conrad M Albrecht and Zhitong Xiong and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
L., Wang Zhu Yuan Mou Q X Z From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2022, (in press). BibTeX | Tags: @article{Yuan2022, title = {From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data}, author = {Wang Zhu Q X Yuan Z. Mou L.}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, note = {in press}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Heidler, Konrad; Mou, Lichao; Loebel, Erik; Scheinert, Mirko; `e, Sébastien" Lef; Zhu, Xiao Xiang Deep Active Contour Models for Delineating Glacier Calving Fronts Inproceedings IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium, 2022. BibTeX | Tags: @inproceedings{igarss2022_3131, title = {Deep Active Contour Models for Delineating Glacier Calving Fronts}, author = {Konrad Heidler and Lichao Mou and Erik Loebel and Mirko Scheinert and Sébastien" Lef{`e}vre and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Gütter, Jonas; Niebling, Julia; Zhu, Xiao Xiang Analysing the interactions between training dataset size, label noise and model performance in remote sensing data Inproceedings 2022 IEEE International Geoscience and Remote Sensing Symposium IGARSS, IEEE 2022. BibTeX | Tags: @inproceedings{gütter2022, title = {Analysing the interactions between training dataset size, label noise and model performance in remote sensing data}, author = {Jonas Gütter and Julia Niebling and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {2022 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, organization = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Wang, Yi; Albrecht, Conrad M; Zhu, Xiao Xiang Self-supervised Vision Transformers for Joint SAR-optical Representation Learning Journal Article 2022. BibTeX | Tags: @article{wang2022self, title = {Self-supervised Vision Transformers for Joint SAR-optical Representation Learning}, author = {Yi Wang and Conrad M Albrecht and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {2022 IEEE International Geoscience and Remote Sensing Symposium}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Li, Jun; Wu, Zhaocong; Hu, Zhongwen; Jian, Canliang; Luo, Shaojie; Mou, Lichao; Zhu, Xiao Xiang; Molinier, Matthieu A Lightweight Deep Learning-Based Cloud Detection Method for Sentinel-2A Imagery Fusing Multiscale Spectral and Spatial Features Journal Article IEEE Transactions on Geoscience and Remote Sensing, 60 , pp. 1–19, 2022. @article{Li_2022, title = {A Lightweight Deep Learning-Based Cloud Detection Method for Sentinel-2A Imagery Fusing Multiscale Spectral and Spatial Features}, author = {Jun Li and Zhaocong Wu and Zhongwen Hu and Canliang Jian and Shaojie Luo and Lichao Mou and Xiao Xiang Zhu and Matthieu Molinier}, url = {https://doi.org/10.1109%2Ftgrs.2021.3069641}, doi = {10.1109/tgrs.2021.3069641}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, volume = {60}, pages = {1--19}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Heidler, Konrad; Mou, Lichao; Baumhoer, Celia; Dietz, Andreas; Zhu, Xiao Xiang HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline Journal Article IEEE Transactions on Geoscience and Remote Sensing, 60 , pp. 1-14, 2022. @article{9383809, title = {HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline}, author = {Konrad Heidler and Lichao Mou and Celia Baumhoer and Andreas Dietz and Xiao Xiang Zhu}, doi = {10.1109/TGRS.2021.3064606}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, volume = {60}, pages = {1-14}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Hua, Yuansheng; Marcos, Diego; Mou, Lichao; Zhu, Xiao Xiang; Tuia, Devis Semantic Segmentation of Remote Sensing Images With Sparse Annotations Journal Article IEEE Geoscience and Remote Sensing Letters, 19 , pp. 1-5, 2022. @article{9335495, title = {Semantic Segmentation of Remote Sensing Images With Sparse Annotations}, author = {Yuansheng Hua and Diego Marcos and Lichao Mou and Xiao Xiang Zhu and Devis Tuia}, doi = {10.1109/LGRS.2021.3051053}, year = {2022}, date = {2022-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, volume = {19}, pages = {1-5}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Sun, Yao; Hua, Yuansheng; Mou, Lichao; Zhu, Xiao Xiang CG-Net: Conditional GIS-Aware Network for Individual Building Segmentation in VHR SAR Images Journal Article IEEE Transactions on Geoscience and Remote Sensing, 60 , pp. 1-15, 2022. @article{9321533, title = {CG-Net: Conditional GIS-Aware Network for Individual Building Segmentation in VHR SAR Images}, author = {Yao Sun and Yuansheng Hua and Lichao Mou and Xiao Xiang Zhu}, doi = {10.1109/TGRS.2020.3043089}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, volume = {60}, pages = {1-15}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Geiß, Christian; Zhu, Yue; Qiu, Chunping; Mou, Lichao; Zhu, Xiao Xiang; Taubenböck, Hannes Deep Relearning in the Geospatial Domain for Semantic Remote Sensing Image Segmentation Journal Article IEEE Geoscience and Remote Sensing Letters, 19 , pp. 1-5, 2022. @article{9247397, title = {Deep Relearning in the Geospatial Domain for Semantic Remote Sensing Image Segmentation}, author = {Christian Geiß and Yue Zhu and Chunping Qiu and Lichao Mou and Xiao Xiang Zhu and Hannes Taubenböck}, doi = {10.1109/LGRS.2020.3031339}, year = {2022}, date = {2022-01-01}, journal = {IEEE Geoscience and Remote Sensing Letters}, volume = {19}, pages = {1-5}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Zaidi, Syed Ali S; Fraz, Muhammad Moazam; Shahzad, Muhammad; Khan, Sharifullah A multiapproach generalized framework for automated solution suggestion of support tickets Journal Article International Journal of Intelligent Systems, 37 (6), pp. 3654-3681, 2022. Abstract | Links | BibTeX | Tags: @article{https://doi.org/10.1002/int.22701, title = {A multiapproach generalized framework for automated solution suggestion of support tickets}, author = {Syed S Ali Zaidi and Muhammad Moazam Fraz and Muhammad Shahzad and Sharifullah Khan}, url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/int.22701}, doi = {https://doi.org/10.1002/int.22701}, year = {2022}, date = {2022-01-01}, journal = {International Journal of Intelligent Systems}, volume = {37}, number = {6}, pages = {3654-3681}, abstract = {Abstract Nowadays, customer support systems are one of the key factors in maintaining any big company's reputation and success. These systems are capable of handling a large number of tickets systemically and provides a mechanism to track/logs the communication between customer and support agents. Companies invest huge amounts of money in training support agents and deploying customer care services for their products and services. Support agents are responsible for handling different customer queries and implementing required actions to solve a particular issue or problem raised by the service/product user. In a bigger picture, customer support systems could receive a large amount of ticket raised depending upon the number of users and services being offered. Customer care service gets directly affected due to the high volume of tickets and a limited number of support agents. Therefore, providing support agents with the recommendations about the possible resolution actions for a new ticket would be helpful and can save a lot of time. This study is focused on the development of an end-to-end framework for suggesting resolution actions rather than recommending free form resolution text against a newly raised ticket. To develop such a system, the pipeline is broadly divided into four components that are data preprocessing, actions extractor, resolution predictor, and evaluation. In actions extractor module, we have proposed a technique to identify and extract actionable phrases from resolution text. For resolution predictor, we have proposed two different pipelines that are referred as “Similarity Search Model” and “End-to-End Model.” The similarity search method is based on a ticket similarity search to find the most relevant historical tickets which then leads to corresponding resolution actions. On the other hand, end-to-end model make use of actions extractor module directly and implemented in a way to directly predict resolution actions. To compare and evaluate the mentioned methods on the same ground, we also proposed an actions evaluation criterion which uses BertScore and METEOR score jointly to compute the score against actual and predicted actions for a particular test ticket. The analysis and experiments are performed on the real-world IBM ticket data set. Overall, we observed that end-to-end model outperformed similarity search-based methods and achieved better performance and scores comparatively. The trained models and code are available at https://bit.ly/2GbUBVk.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Abstract Nowadays, customer support systems are one of the key factors in maintaining any big company's reputation and success. These systems are capable of handling a large number of tickets systemically and provides a mechanism to track/logs the communication between customer and support agents. Companies invest huge amounts of money in training support agents and deploying customer care services for their products and services. Support agents are responsible for handling different customer queries and implementing required actions to solve a particular issue or problem raised by the service/product user. In a bigger picture, customer support systems could receive a large amount of ticket raised depending upon the number of users and services being offered. Customer care service gets directly affected due to the high volume of tickets and a limited number of support agents. Therefore, providing support agents with the recommendations about the possible resolution actions for a new ticket would be helpful and can save a lot of time. This study is focused on the development of an end-to-end framework for suggesting resolution actions rather than recommending free form resolution text against a newly raised ticket. To develop such a system, the pipeline is broadly divided into four components that are data preprocessing, actions extractor, resolution predictor, and evaluation. In actions extractor module, we have proposed a technique to identify and extract actionable phrases from resolution text. For resolution predictor, we have proposed two different pipelines that are referred as “Similarity Search Model” and “End-to-End Model.” The similarity search method is based on a ticket similarity search to find the most relevant historical tickets which then leads to corresponding resolution actions. On the other hand, end-to-end model make use of actions extractor module directly and implemented in a way to directly predict resolution actions. To compare and evaluate the mentioned methods on the same ground, we also proposed an actions evaluation criterion which uses BertScore and METEOR score jointly to compute the score against actual and predicted actions for a particular test ticket. The analysis and experiments are performed on the real-world IBM ticket data set. Overall, we observed that end-to-end model outperformed similarity search-based methods and achieved better performance and scores comparatively. The trained models and code are available at https://bit.ly/2GbUBVk. | |
L., Xia Zhu Jin Mou G X P Anomaly Detection in Aerial Videos with Transformers Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2022, (in press). BibTeX | Tags: @article{, title = {Anomaly Detection in Aerial Videos with Transformers}, author = {Xia Zhu G X Jin P. Mou L.}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, note = {in press}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Guo, Jianhua; Xu, Qingsong; Zeng, Yue; Liu, Zhiheng; Zhu, Xiaoxiang Semi-Supervised Cloud Detection in Satellite Images by Considering the Domain Shift Problem Journal Article Remote Sensing, 14 (11), 2022, ISSN: 2072-4292. Abstract | Links | BibTeX | Tags: @article{rs14112641, title = {Semi-Supervised Cloud Detection in Satellite Images by Considering the Domain Shift Problem}, author = {Jianhua Guo and Qingsong Xu and Yue Zeng and Zhiheng Liu and Xiaoxiang Zhu}, url = {https://www.mdpi.com/2072-4292/14/11/2641}, doi = {10.3390/rs14112641}, issn = {2072-4292}, year = {2022}, date = {2022-01-01}, journal = {Remote Sensing}, volume = {14}, number = {11}, abstract = {In terms of semi-supervised cloud detection work, efforts are being made to learn a promising cloud detection model via a limited number of pixel-wise labeled images and a large number of unlabeled ones. However, remote sensing images obtained from the same satellite sensor often show a data distribution drift problem due to the different cloud shapes and land-cover types on the Earth’s surface. Therefore, there are domain distribution gaps between labeled and unlabeled satellite images. To solve this problem, we take the domain shift problem into account for the semi-supervised learning (SSL) network. Feature-level and output-level domain adaptations are applied to reduce the domain distribution gaps between labeled and unlabeled images, thus improving predicted results accuracy of the SSL network. Experimental results on Landsat-8 OLI and GF-1 WFV multispectral images demonstrate that the proposed semi-supervised cloud detection network (SSCDnet) is able to achieve promising cloud detection performance when using a limited number of labeled samples and outperforms several state-of-the-art SSL methods.}, keywords = {}, pubstate = {published}, tppubtype = {article} } In terms of semi-supervised cloud detection work, efforts are being made to learn a promising cloud detection model via a limited number of pixel-wise labeled images and a large number of unlabeled ones. However, remote sensing images obtained from the same satellite sensor often show a data distribution drift problem due to the different cloud shapes and land-cover types on the Earth’s surface. Therefore, there are domain distribution gaps between labeled and unlabeled satellite images. To solve this problem, we take the domain shift problem into account for the semi-supervised learning (SSL) network. Feature-level and output-level domain adaptations are applied to reduce the domain distribution gaps between labeled and unlabeled images, thus improving predicted results accuracy of the SSL network. Experimental results on Landsat-8 OLI and GF-1 WFV multispectral images demonstrate that the proposed semi-supervised cloud detection network (SSCDnet) is able to achieve promising cloud detection performance when using a limited number of labeled samples and outperforms several state-of-the-art SSL methods. | |
Saha, Sudipan; Gawlikowski, Jakob; Zhu, Xiao Xiang Fusing multiple untrained networks for hyperspectral change detection Inproceedings XXIV ISPRS Congress 2022, 2022. BibTeX | Tags: @inproceedings{Saha2022a, title = {Fusing multiple untrained networks for hyperspectral change detection}, author = {Sudipan Saha and Jakob Gawlikowski and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {XXIV ISPRS Congress 2022}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Zhao, Shan; Saha, Sudipan; Zhu, Xiao Xiang Graph neural network based open-set domain adaptation Inproceedings XXIV ISPRS Congress 2022, 2022. BibTeX | Tags: @inproceedings{Zhao2022, title = {Graph neural network based open-set domain adaptation}, author = {Shan Zhao and Sudipan Saha and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {XXIV ISPRS Congress 2022}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Saha, Sudipan; Zhan, Shao; Shahzad, Muhammad; Zhu, Xiao Xiang Mitigating distribution shift for multi-sensor classification Inproceedings IEEE Geoscience and Remote Sensing Symposium 2022, IEEE, 2022. BibTeX | Tags: @inproceedings{Saha2022bb, title = {Mitigating distribution shift for multi-sensor classification}, author = {Sudipan Saha and Shao Zhan and Muhammad Shahzad and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2022}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Gawlikowski, Jakob; Saha, Sudipan; Niebling, Julia; Zhu, Xiao Xiang Robust Distribution-Shift Aware SAR-Optical Data Fusion for Multi-Label Scene Classification Inproceedings IEEE Geoscience and Remote Sensing Symposium 2022, IEEE, 2022. BibTeX | Tags: @inproceedings{Gawlikowski2022a, title = {Robust Distribution-Shift Aware SAR-Optical Data Fusion for Multi-Label Scene Classification}, author = {Jakob Gawlikowski and Sudipan Saha and Julia Niebling and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {IEEE Geoscience and Remote Sensing Symposium 2022}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Hermann, Martin; Saha, Sudipan; Zhu, Xiao Xiang Few-Shot Filtering for the Detection of Specialized Change in Remote Sensing Inproceedings ICLR 2022 Workshop on Practical ML for Developing Countries, 2022. BibTeX | Tags: @inproceedings{Hermann2022, title = {Few-Shot Filtering for the Detection of Specialized Change in Remote Sensing}, author = {Martin Hermann and Sudipan Saha and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {ICLR 2022 Workshop on Practical ML for Developing Countries}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Li, Qingyu; Shi, Yilei; Zhu, Xiao Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training 2022. @unknown{unknown, title = {Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training}, author = {Qingyu Li and Yilei Shi and Xiao Zhu}, doi = {10.48550/arXiv.2205.08416}, year = {2022}, date = {2022-01-01}, keywords = {}, pubstate = {published}, tppubtype = {unknown} } | |
Diaconu, Codruț-Andrei; Saha, Sudipan; Günnemann, Stephan; Zhu, Xiao Xiang Understanding the Role of Weather Data for Earth Surface Forecasting using a ConvLSTM-based Model Inproceedings CVPR 2022 Workshop Earthvision, 2022. BibTeX | Tags: @inproceedings{Diaconu2022, title = {Understanding the Role of Weather Data for Earth Surface Forecasting using a ConvLSTM-based Model}, author = {Codruț-Andrei Diaconu and Sudipan Saha and Stephan Günnemann and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {CVPR 2022 Workshop Earthvision}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Gu, Ziqi; Ebel, Patrick; Yuan, Qiangqiang; Schmitt, Michael; Zhu, Xiao Xiang Explicit Haze & Cloud Removal for Global Land Cover Classification Inproceedings CVPR Workshops, pp. 1–6, 2022. BibTeX | Tags: @inproceedings{gu2022explicithcr, title = {Explicit Haze & Cloud Removal for Global Land Cover Classification}, author = {Ziqi Gu and Patrick Ebel and Qiangqiang Yuan and Michael Schmitt and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {CVPR Workshops}, pages = {1--6}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
Saha, Sudipan; Zhao, Shan; Sheikh, Nasrullah; Zhu, Xiao Xiang Reiterative Domain Aware Multi-Target Adaptation Inproceedings German Conference on Pattern Recognition (GCPR), 2022. BibTeX | Tags: @inproceedings{Saha2022d, title = {Reiterative Domain Aware Multi-Target Adaptation}, author = {Sudipan Saha and Shan Zhao and Nasrullah Sheikh and Xiao Xiang Zhu}, year = {2022}, date = {2022-01-01}, booktitle = {German Conference on Pattern Recognition (GCPR)}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } | |
M., Mou Song Zhu Saha Shahzad L Q X S Unsupervised Single-Scene Semantic Segmentation for Earth Observation Journal Article IEEE Transactions on Geoscience and Remote Sensing, 2022. BibTeX | Tags: @article{Saha2022cb, title = {Unsupervised Single-Scene Semantic Segmentation for Earth Observation}, author = {Mou Song Zhu L Q X Saha S. Shahzad M.}, year = {2022}, date = {2022-01-01}, journal = {IEEE Transactions on Geoscience and Remote Sensing}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
2021 |
|
Usmani, Fehmida; Khan, Ihtesham; Siddiqui, Mehek; Khan, Mahnoor; Bilal, Muhammad; Masood, M; Ahmad, Arsalan; Shahzad, Muhammad; Curri, Vittorio Cross-feature trained machine learning models for QoT-estimation in optical networks Journal Article Optical Engineering, 60 (12), pp. 125106, 2021. @article{Usmani2021, title = {Cross-feature trained machine learning models for QoT-estimation in optical networks}, author = {Fehmida Usmani and Ihtesham Khan and Mehek Siddiqui and Mahnoor Khan and Muhammad Bilal and M Masood and Arsalan Ahmad and Muhammad Shahzad and Vittorio Curri}, doi = {10.1117/1.oe.60.12.125106}, year = {2021}, date = {2021-12-03}, journal = {Optical Engineering}, volume = {60}, number = {12}, pages = {125106}, publisher = {SPIE-Intl Soc Optical Eng}, keywords = {}, pubstate = {published}, tppubtype = {article} } | |
Bloice, Marcus D; Roth, Peter M; Holzinger, Andreas Performing Arithmetic Using a Neural Network Trained on Images of Digit Permutation Pairs Journal Article J. Intell. Inf. Syst., 57 (3), pp. 547–562, 2021, ISSN: 0925-9902. Abstract | Links | BibTeX | Tags: @article{10.1007/s10844-021-00662-9, title = {Performing Arithmetic Using a Neural Network Trained on Images of Digit Permutation Pairs}, author = {Marcus D Bloice and Peter M Roth and Andreas Holzinger}, url = {https://doi.org/10.1007/s10844-021-00662-9}, doi = {10.1007/s10844-021-00662-9}, issn = {0925-9902}, year = {2021}, date = {2021-12-01}, journal = {J. Intell. Inf. Syst.}, volume = {57}, number = {3}, pages = {547–562}, publisher = {Kluwer Academic Publishers}, address = {USA}, abstract = {In this paper, a neural network is trained to perform simple arithmetic using images of concatenated handwritten digit pairs. A convolutional neural network was trained with images consisting of two side-by-side handwritten digits, where the image’s label is the summation of the two digits contained in the combined image. Crucially, the network was tested on permutation pairs that were not present during training in an effort to see if the network could learn the task of addition, as opposed to simply mapping images to labels. A dataset was generated for all possible permutation pairs of length 2 for the digits 0–9 using MNIST as a basis for the images, with one thousand samples generated for each permutation pair. For testing the network, samples generated from previously unseen permutation pairs were fed into the trained network, and its predictions measured. Results were encouraging, with the network achieving an accuracy of over 90% on some permutation train/test splits. This suggests that the network learned at first digit recognition, and subsequently the further task of addition based on the two recognised digits. As far as the authors are aware, no previous work has concentrated on learning a mathematical operation in this way. This paper is an attempt to demonstrate that a network can learn more than a direct mapping from image to label, but is learning to analyse two separate regions of an image and combining what was recognised to produce the final output label.}, keywords = {}, pubstate = {published}, tppubtype = {article} } In this paper, a neural network is trained to perform simple arithmetic using images of concatenated handwritten digit pairs. A convolutional neural network was trained with images consisting of two side-by-side handwritten digits, where the image’s label is the summation of the two digits contained in the combined image. Crucially, the network was tested on permutation pairs that were not present during training in an effort to see if the network could learn the task of addition, as opposed to simply mapping images to labels. A dataset was generated for all possible permutation pairs of length 2 for the digits 0–9 using MNIST as a basis for the images, with one thousand samples generated for each permutation pair. For testing the network, samples generated from previously unseen permutation pairs were fed into the trained network, and its predictions measured. Results were encouraging, with the network achieving an accuracy of over 90% on some permutation train/test splits. This suggests that the network learned at first digit recognition, and subsequently the further task of addition based on the two recognised digits. As far as the authors are aware, no previous work has concentrated on learning a mathematical operation in this way. This paper is an attempt to demonstrate that a network can learn more than a direct mapping from image to label, but is learning to analyse two separate regions of an image and combining what was recognised to produce the final output label. | |
Yuan, Zhenghang; Mou, Lichao; Zhu, Xiao Xiang Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data Conference IEEE International Symposium on Geoscience and Remote Sensing (IGARSS), Institute of Electrical and Electronics Engineers (IEEE), 2021, ISBN: 9781665403696. Abstract | Links | BibTeX | Tags: @conference{Yuan2021b, title = {Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data}, author = {Zhenghang Yuan and Lichao Mou and Xiao Xiang Zhu}, doi = {10.1109/IGARSS47720.2021.9553624}, isbn = {9781665403696}, year = {2021}, date = {2021-10-01}, booktitle = {IEEE International Symposium on Geoscience and Remote Sensing (IGARSS)}, journal = {IEEE International Symposium on Geoscience and Remote Sensing (IGARSS)}, pages = {2999-3002}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, abstract = {Answering questions with natural language by extracting information from image has great potential in various applications. Although visual question answering (VQA) for natural image has been broadly studied, VQA for remote sensing data is still in the early research stage. For the same remote sensing image, there exist questions with dramatically different difficulty-levels. Treating these questions equally may mislead the model and limit the VQA model performance. Considering this problem, in this work, we propose a self-paced curriculum learning (SPCL) based VQA model with hard and soft weighting strategies for remote sensing data. Like human learning process, the model is trained from easy to hard question samples gradually. Extensive experimental results on two datasets demonstrate that the proposed training method can achieve promising performance.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Answering questions with natural language by extracting information from image has great potential in various applications. Although visual question answering (VQA) for natural image has been broadly studied, VQA for remote sensing data is still in the early research stage. For the same remote sensing image, there exist questions with dramatically different difficulty-levels. Treating these questions equally may mislead the model and limit the VQA model performance. Considering this problem, in this work, we propose a self-paced curriculum learning (SPCL) based VQA model with hard and soft weighting strategies for remote sensing data. Like human learning process, the model is trained from easy to hard question samples gradually. Extensive experimental results on two datasets demonstrate that the proposed training method can achieve promising performance. | |
Perko, Roland; Almer, Alexander; Theuermann, Mario; Klopschitz, Manfred; Schnsbel, Thomas; Roth, Peter Protocol Design Issues for Object Density Estimation and Counting in Remote Sensing Book Chapter 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, IEEE, 2021. @inbook{Perko2021b, title = {Protocol Design Issues for Object Density Estimation and Counting in Remote Sensing}, author = {Roland Perko and Alexander Almer and Mario Theuermann and Manfred Klopschitz and Thomas Schnsbel and Peter Roth}, doi = {10.1109/igarss47720.2021.9553934}, year = {2021}, date = {2021-07-11}, booktitle = {2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } |