Tackling distribution shift for reliable and data-efficient deep learning

Abstract:

The talk presents robust connectionist techniques for the empirical estimation of multivariate probability density functions (pdf) from unlabeled data samples (still an open yet crucial issue in pattern recognition and machine learning). Data may either be samples of random feature vectors or generalized random graphs. First, a soft-constrained unsupervised algorithm for training a (possibly deep) feed-forward neural net is discussed. A variant of the Metropolis-Hastings algorithm (exploiting the very probabilistic nature of the present deep network) is used to guarantee a model that satisfies numerically Kolmogorov's second axiom of probability. The approach overcomes the major limitations of the established statistical estimators. Graphical and quantitative experimental results show that the proposed technique can offer estimates that improve significantly over parametric and nonparametric approaches, regardless of (1) the complexity of the underlying pdf, (2) the dimensionality of the feature space, and (3) the amount of data available for training.  Then, a hybrid machine (combining a graph neural network with an RBF-like network) is presented that can be trained via maximum likelihood to estimate pdfs over structured (i.e., graphical) patterns.

Biography:

Edmondo Trentin, PhD, is a Professore Associato (tenured) in Artificial Intelligence at DIISM, Univ. di Siena (Italy), and a member of the SAIlab. ET taught more than 70 courses at the undergraduate and graduate levels over 4 Countries. Author of more than 90 peer-reviewed publications and a monograph (Hybrid Random Fields, with A. Freno. Springer, 2011). his interests are in neural networks, statistical pattern recognition, and machine learning over graphs. Dr. Trentin is an Associate Editor of the journal Neural Processing Letters (Springer). He was the Chair of the IAPR-TC3 (International Association for Pattern Recognition Technical Committee 03 - Neural Networks & Computational Intelligence) in the years 2017-2020, and he held the Secretariat of the IEEE Computational Intelligence Society in the years 2005-2006. He served as a General Chair of several international Workshops sponsored by the IAPR. Website: http://www.dii.unisi.it/~trentin/HomePage.html