Title: Independent Subspace Analysis

We are often presented with data in a high-dimensional measurement
space, which we would like to reduce to a lower dimensional space for
further analysis, ideally retaining as much relevant information as
possible. A common approach is to employ a linear generative model in
the measurement space, with potentially relevant features existing in
subspaces. Extraction of the relevant information can then be
performed by linear projection (using a left inverse of the generating
subspace.) Usually, the information-containing subspace is not known
exactly and must be either approximately constructed using prior
knowledge, or estimated in an unsupervised manner, based on
statistical properties of the extracted signal. In this talk we will
be concerned with unsupervised learning of subspaces based on the
criterion that the generating source signal vector be independent of
the other sources contributing to the full data measurement.

In the case of one-dimensional subspaces, this problem is referred to
as Independent Component Analysis (ICA). A remarkable property of ICA,
and the ICA model, is that independent sources can be estimated
sequentially, one at a time, using only fairly weak conditions and
properties of the source probability density, e.g. sub- or
super-Gaussianity. In other words, it turns out to be remarkably easy
to extract large families of signals from independent linear
mixtures. This process is enabled by deep properties of linear
mixtures of random variables leading to the possibility of defining
"contrast functions", which are able to measure, essentially, how
mixed a signal is, or how much extraneous information is present from
independent linearly superimposed sources. In this talk, we will show
that the basic approach to ICA, using contrast functions, can be
extended directly to the case of independent subspaces, retaining the
possibility of estimating one dimension at a time. Essentially, the
contrast function approach allows us to guarantee that no extracted
dimension contains information from two independent subspaces.

We will begin by introducing non-Gaussian probability densities, and
the categorizations of sub- and super-Gaussianity, log-concavity type
properties, cumulant properties, and Gaussian Scale Mixtures. We will
then introduce contrast functions, and show how these can be used to
extract non-Gaussian sources from linear mixtures. Finally we will
consider the problem of Independent Subspace Analysis (ISA), and how
the techniques considered can be used to perform ISA.

Bio: Jason Palmer is an Asst. Project Scientist at the Institute for
Neural Computation, Swartz Center for Computational Neuroscience,
UCSD.  Dr. Palmer received his PhD in Electrical and Computer
Engineering from UCSD in 2006, and has since worked in theoretical
foundations of ICA, and the application of statistical signal
processing to analysis of EEG signals. Email: japalmer@ucsd.edu. Web:
http://sccn.ucsd.edu/~jason/.