How to Sparsify the Singular Value Decomposition With Orthogonality Constraints: Theory and Applications to Image Analysis and Psychometry

The Singular Value Decomposition (SVD) constitutes the core of most popular multivariate methods. To analyze a data table, these methods use the SVD to generate orthogonal components (for the rows) and orthogonal loadings (for the columns) that, together, extract the important information of a data table. The loadings are used to interpret the corresponding components and this interpretation is greatly facilitated when only few variables have large loadings (particularly when the number of variables is large). When this “sparse” pattern does not naturally hold, several techniques can generate sparse components and loadings; but, in most methods, this sparsification is obtained at the cost of orthogonality. Here we propose a new approach for the SVD—called constrained SVD (CSVD)—that includes sparsity constraints on the columns and rows of a rectangular matrix while keeping the singular vectors orthogonal. We illustrate the CSVD and compare it to alternatives (e.g., rotation and other sparsification) with simulated data and applications on a few real data sets such as psychometric data and face-image analysis.

Authors: Vincent Guillemot, Aida Eslami, Arnaud Gloaguen, Arthur Tenenhaus, & Hervé Abdi.

Speaker Bio:

Hervé is currently a full professor in the School of Behavioral and Brain Sciences at the University of Texas at Dallas. He was twice a Fulbright scholar. He has been also a visiting scientist or professor in numerous institutions in Europe and North America. His recent work is concerned with face and person perception, odor perception, and with computational modeling of these processes. He is also developing statistical techniques to analyze the structure of large data sets as found, for example, in Genomics, brain imaging, and sensory evaluation (e.g., principal component analysis, correspondence analysis, PLS-Regression, STATIS, DISTATIS, discriminant correspondence analysis, multiple factor analysis, multi-table analysis, additive tree representations,...). He has has more than 250 publications on these topics. He teaches or has taught classes in cognition, computational modeling, experimental design, multivariate statistics, and the analysis of brain imaging data.