PUBLICATIONS
On Measures Of Uncertainty In Classification
S. Chlaily, D. Ratha, P. Lozou and A. Marinoni
IEEE Transactions on Signal Processing, vol. 71, pp. 3710-3725, 2023, doi: 10.1109/TSP.2023.3322843.
Uncertainty is unavoidable in classification tasks and might originate from data (e.g., due to noise or wrong labeling), or the model (e.g., due to erroneous assumptions, etc). Providing an assessment of uncertainty associated with each outcome is of paramount importance in assessing the reliability of classification algorithms, especially on unseen data. In this work, we propose two measures of uncertainty in classification. One of the measures is developed from a geometrical perspective and quantifies a classifier’s distance from a random guess. In contrast, the second proposed uncertainty measure is homophily-based since it takes into account the similarity between the classes. Accordingly, it reflects the type of mistaken classes. The proposed measures are not aggregated, i.e., they provide an uncertainty assessment to each data point. Moreover, they do not require label information. Using several datasets, we demonstrate the proposed measures’ differences and merit in assessing uncertainty in classification. The source code is available at github.com/pioui/uncertainty.
SAR and Passive Microwave Fusion Scheme: A Test Case on Sentinel‐1/AMSR‐2 for Sea Ice Classification
E. Khachatrian, W. Dierking, S. Chlaily, T. Eltoft, F. Dinessen, N. Hughes, A. Marinoni
Geophysical Research Letters, vol. 50, 2023
The most common source of information about sea ice conditions is remote sensing data, especially images obtained from synthetic aperture radar (SAR) and passive microwave radiometers (PMR). Here we introduce an adaptive fusion scheme based on Graph Laplacians that allows us to retrieve the most relevant information from satellite images. In a first test case, we explore the potential of sea ice classification employing SAR and PMR separately and simultaneously, in order to evaluate the complementarity of both sensors and to assess the result of a combined use. Our test case illustrates the flexibility and efficiency of the proposed scheme and indicates an advantage of combining AMSR‐2 89 GHz and Sentinel‐1 data for sea ice mapping.
A multimodal feature selection method for remote sensing data analysis based on double graph Laplacian diagonalization
E. Khachatrian, S. Chlaily, T. Eltoft, A. Marinoni
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 11546-11566, 2021
When dealing with multivariate remotely sensed records collected by multiple sensors, an accurate selection of information at the data, feature, or decision level is instrumental in improving the scenes’ characterization. This will also enhance the system’s efficiency and provide more details on modeling the physical phenomena occurring on the Earth’s surface. In this article, we introduce a flexible and efficient method based on graph Laplacians for information selection at different levels of data fusion. The proposed approach combines data structure and information content to address the limitations of existing graph-Laplacian-based methods in dealing with heterogeneous datasets. Moreover, it adapts the selection to each homogenous area of the considered images according to their underlying properties. Experimental tests carried out on several multivariate remote sensing datasets show the consistency of the proposed approach.
Automatic selection of relevant attributes for multi-sensor remote sensing analysis: a case study on sea ice classificationn
E. Khachatrian, S. Chlaily, T. Eltoft, W. Dierking, F. Dinessen, A. Marinoni
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 9025-9037, 2021
It is of considerable benefit to combine information obtained from different satellite sensors to achieve advanced and improved characterization of sea ice conditions. However, it is also true that not all the information is relevant. It may be redundant, corrupted, or unnecessary for the given task, hence decreasing the performance of the algorithms. Therefore, it is crucial to select an optimal set of image attributes which provides the relevant information content to enhance the efficiency and accuracy of the image interpretation and retrieval of geophysical parameters. Comprehensive studies have been focused on the analysis of relevant features for sea ice analysis obtained from different sensors, especially synthetic aperture radar. However, the outcomes of these studies are mostly data and application-dependent and can, therefore, rarely be generalized. In this article, we employ a feature selection method based on graph Laplacians, which is fully automatic and easy to implement. The proposed approach assesses relevant information on a global and local level using two metrics and selects relevant features for different regions of an image according to their physical characteristics and observation conditions. In the recent study, we investigate the effectiveness of this approach for sea ice classification, using different multi-sensor data combinations. Experiments show the advantage of applying multi-sensor data sets and demonstrate that the attributes selected by our method result in high classification accuracies. We demonstrate that our approach automatically considers varying technical, sensor-specific, environmental, and sea ice conditions by employing flexible and adaptive feature selection method as a pre-processing step.
On Enhanced Ensemble Learning for Multimodal Remote Sensing Data Analysis by Capacity Optimization
S. Chlaily, D. Ienco, C. Jutten, A. Marinoni
IEEE Statistical Signal Processing Workshop (SSP), Rio de Janeiro, Brazil, 2021, pp. 151-155
Multimodal remote sensing data analysis can strongly improve the characterization of physical phenomena on Earth’s surface. Nonetheless, nonidealities and estimation imperfections between records and investigation models can limit its information extraction ability. Ensemble learning could be used to tackle these issues. Combining the information acquired by multiple weak classifiers can prevent the analysis of large scale heterogeneous datasets from being affected by overfitting and biasing. In this paper, we introduce an enhanced ensemble learning scheme where the information acquired by the weak classifiers is combined to optimize the maximum information extraction for the given system at a decision level. Using an asymptotic information theory-based approach, we define the capacity index associated with the maximum accuracy that can be achieved under optimal conditions for multimodal analysis. By selecting the decisions delivered by the different classifiers according to the capacity optimization, the performance of the ensemble learning scheme will be maximized.
Unsupervised Band Selection for Hyperspectral Datasets by Double Graph Laplacian Diagonalization
E. Khachatrian,S. Chlaily, T. Eltoft, P. Gamba, A. Marinoni
IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium, 2021, pp. 4007-4010
The vast amount of spectral information provided by hyperspectral images can be useful for different applications. However, the presence of redundant bands will negatively affect application performance. Therefore, it is crucial to select a relevant subset that preserves the information of the original set. In this paper, we present an automatic and accurate band selection method based on Graph Laplacians. Unlike existing band selection methods, this method exploits two similarity measures simultaneously. Furthermore, it is performed on a superpixel level, so it allows us to preserve not only global but contemporaneously local particularities of original data. Experiments show the importance of measuring the relevance of the bands at local and global scales and the ability of the method to minimize intercorrelation among selected bands, hence improving the selection of the most informative spectral channels.
Enhancing ensemble learning and transfer learning in multimodal data analysis by adaptive dimensionality reduction
A. Marinoni,S. Chlaily, E. Khachatrian, T. Eltoft, S. Selvakumaran, , M. Girolami, C. Jutten
arXiv preprint arXiv:2105.03682
Modern data analytics take advantage of ensemble learning and transfer learning approaches to tackle some of the most relevant issues in data analysis, such as lack of labeled data to use to train the analysis models, sparsity of the information, and unbalanced distributions of the records. Nonetheless, when applied to multimodal datasets (i.e., datasets acquired by means of multiple sensing techniques or strategies), the state-of-theart methods for ensemble learning and transfer learning might show some limitations. In fact, in multimodal data analysis, not all observations would show the same level of reliability or information quality, nor an homogeneous distribution of errors and uncertainties. This condition might undermine the classic assumptions ensemble learning and transfer learning methods rely on. In this work, we propose an adaptive approach for dimensionality reduction to overcome this issue. By means of a graph theory-based approach, the most relevant features across variable size subsets of the considered datasets are identified. This information is then used to set-up ensemble learning and transfer learning architectures. We test our approach on multimodal datasets acquired in diverse research fields (remote sensing, brain-computer interfaces, photovoltaic energy). Experimental results show the validity and the robustness of our approach, able to outperform state-of-the-art techniques.
A wavelet-based thermal noise removal approach for Sentinel-1 records on polar areas
S. Chlaily, T. Kramer, T. Eltoft, A. Marinoni
EUSAR 2021; 13th European Conference on Synthetic Aperture Radar, online, 2021, pp. 1-5
The potential of Sentinel-1 recordings, mainly the most extensive spatial coverage and the highest temporal resolution, is not fully exploited in polar areas because of thermal noise. The strong intensity of the noise in low backscattering areas makes it resemble ice, which deteriorates the sea-ice analysis. Moreover, in multi-looking modes, the thermal noise varies across the sub-swaths and introduces large stripes at their intersection. In this paper, we propose a noise removal approach based on wavelets. The experimental analysis conducted on several images shows the aptitude of the proposed method to effectively and efficiently eliminate the noise.
Selecting principal attributes in multimodal remote sensing for sea ice characterization
E. Khachatrian, S. Chlaily, T. Eltoft, A. Marinoni
EUSAR 2021; 13th European Conference on Synthetic Aperture Radar, online, 2021, pp. 1-6
Automatic ice charting can not be achieved using only SAR modalities. It is fundamental to combine information from other remote sensors with different characteristics for more reliable sea ice characterization. In this paper, we employ principal feature analysis (PFA) to select significant information from multimodal remote sensing data. PFA is a simple yet very effective approach that can be applied to several types of data without loss of physical interpretability. Considering that different homogeneous regions require different types of information, we perform the selection patch-wise. Accordingly, by exploiting the spatial information, we increase the robustness and accuracy of PFA.
On the Exploitation of Heterophily in Graph-Based Multimodal Remote Sensing Data Analysis.
C. Taelman, S. Chlaily, E. Khachatrian, F. van der Sommen, A. Marinoni
CIKM Workshops
The field of Earth observation is dealing with increasingly large, multimodal data sets. An important processing step consists of providing these data sets with labels. However, standard label propagation algorithms cannot be applied to multimodal remote sensing data for two reasons. First, multimodal data is heterogeneous while classic label propagation algorithms assume a homogeneous network. Second, real-world data can show both homophily (’birds of a feather flock together’) and heterophily (’opposites attract’) during propagation, while standard algorithms only consider homophily. Both shortcomings are addressed in this work and the result is a graph-based label propagation algorithm for multimodal data that includes homophily and/or heterophily. Furthermore, the method is also able to transfer information between uni- and multimodal data. Experiments on the remote sensing data set of Houston, which contains a LiDAR and a hyperspectral image, show that our approach ties state-of-the-art methods for classification with an OA of 91.4%, while being more flexible and not constrained to a specific data set or a specific combination of modalities.
Addressing Reliability of Multimodal Remote Sensing to Enhance Multisensor Data Fusion and Transfer Learning
A. Marinoni, S. Chlaily, C. Jutten
IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA, 2020, pp. 3896-3899
In literature, we find many examples showing that, contrary to what one might think, multimodal remote sensing analysis might be suboptimal. Given the high computational complexity typically required by multimodal investigation in order to properly extract information from multiple sources, there is a need to assess its actual benefit during image preprocessing. This urgency becomes indeed crucial when targeting transfer learning in remote sensing, as understanding the actual relationship between diverse sensors is fundamental to accurately characterize the considered scenes. In this work, we derive a reliability metric by means of an information theory-based approach. The proposed metric is able to estimate how confident one can be of the considered datasets when characterizing each pixel in the considered region of interest. Experimental results on real datasets show how this quantity can be used to improve the understanding of the scenes, and to enhance multisensor transfer learning.
Capacity and limits of multimodal remote sensing: theoretical aspects and automatic information theory-based image selection
S. Chlaily, M. Dalla Mura, J. Chanussot, C. Jutten, P. Gamba, A. Marinoni
IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 7, pp. 5598-5618, July 2021
Although multimodal remote sensing data analysis can strongly improve the characterization of physical phenomena on Earth’s surface, nonidealities and estimation imperfections between records and investigation models can limit its actual information extraction ability. In this article, we aim at predicting the maximum information extraction that can be reached when analyzing a given data set. By means of an asymptotic information theory-based approach, we investigate the reliability and accuracy that can be achieved under optimal conditions for multimodal analysis as a function of data statistics and parameters that characterize the multimodal scenario to be addressed. Our approach leads to the definition of two indices that can be easily computed before the actual processing takes place. Moreover, we report in this article how they can be used for operational use in terms of image selection in order to maximize the robustness of the multimodal analysis, as well as to properly design data collection campaigns for understanding and quantifying physical phenomena. Experimental results show the consistency of our approach.
Modèle d'interaction et performances du traitement du signal multimodal
S. Chlaily
Université Grenoble Alpes (ComUE)
The joint processing of multimodal measurements is supposed to lead to better performances than those obtained using a single modality or several modalities independently. However, in literature, there are examples that show that is not always true. In this thesis, we analyze, in terms of mutual information and estimation error, the different situations of multimodal analysis in order to determine the conditions to achieve the optimal performances.In the first part, we consider the simple case of two or three modalities, each associated with noisy measurement of a signal. These modalities are linked through the correlations between the useful parts of the signal and the correlations between the noises. We show that the performances are improved if the links between the modalities are exploited. In the second part, we study the impact on performance of wrong links between modalities. We show that these false assumptions decline the performance, which can become lower than the performance achieved using a single modality.In the general case, we model the multiple modalities as a noisy Gaussian channel. We then extend literature results by considering the impact of the errors on signal and noise probability densities on the information transmitted by the channel. We then analyze this relationship in the case of a simple model of two modalities. Our results show in particular the unexpected fact that a double mismatch of the noise and the signal can sometimes compensate for each other, and thus lead to very good performances.
L'interaction entre deux modalités complémentaires
S Chlaily, PO Amblard, OJJ Michel, C Jutten
GRETSI 2017-XXVIème Colloque francophone de traitement du signal et des images
It is generally assumed that the joint processing of multimodal measurements leads to better performance than that obtained by exploiting a single modality, or all modalities independently. Under what conditions is this statement true or false? To answer this, we consider a simple example of two modalities correlated via information-bearing signals and noise. In this very simple framework, we can analyze meticulously the impact of these correlations on performance in terms of information and estimation error. We also show that mismatched correlations has effects, sometimes quite surprising, on performance.
Information–estimation relationship in mismatched Gaussian channels
S. Chlaily, C. Ren, P. -O. Amblard, O. Michel, P. Comon, C. Jutten
IEEE Signal Processing Letters, vol. 24, no. 5, pp. 688-692, May 2017
In this letter, we investigated the connection between information and estimation measures for mismatched Gaussian models. In addition to the input prior mismatch, we take into account the noise mismatch and establish a new relation between relative entropy and excess mean square error. The derived formula shows that the input prior mismatch may be canceled by the noise mismatch. Finally, an example illustrates the impact of model mismatches on estimation accuracy.
Impact of noise correlation on multimodality
S. Chlaily, P. -O. Amblard, O. Michel, C. Jutten
24th European Signal Processing Conference (EUSIPCO), Budapest, Hungary, 2016, pp. 195-199
In this paper, we consider the problem of estimating an unknown random scalar observed by two modalities. We study two scenarios using mutual information and mean square error. In the first scenario, we consider that the noise correlation is known and examine its impact on the information content of two modalities. In the second scenario we quantify the information loss when the considered value of the noise correlation is wrong. It is shown that the noise correlation usually enhances the estimation accuracy and increases information. However, the performance declines if the noise correlation is misdefined, and the two modalities may jointly convey less information than one single modality.