Next: Single and Multiple ICA
Up: tr01dl1
Previous: Variance criterion
With PCA one obtains a decomposition like 1 with variance criterion and the constraints
on the components is no correlation. Generalisations of these criterion and constraints has been
a continuous endeavour and the literature is abondant about this subject. Recently the concept of
Independent Components Analysis has been widely used in fMRI analysis. Initially the purpose was
to look for Independent Components at a higher order than two (no correlation). The motivation was
to recover non-gaussian latent variables (called sources) who generated the data by linear mixing
of them, and supposed to be mutually independent. The link between this two concepts can be
understood as follows.
A measure of mutual independence [5] of the variables is the Kullback-Liebler divergence
between the joint density and the product of the marginal densities :
and is often termed mutual information. Negentropy itself can be defined as the Kullback-Liebler
divergence between the density (single or vector variable) and the normal equivalent i.e.
with the same mean and variance, or difference of the corresponding entropies 2:
|
(7) |
The Negentropy can be related to the mutual information:
|
(8) |
then as the Negentropy is invariant by invertible transformation minimising the mutual information
is equivalent to maximising negentropies of components which then are the least Gaussian variables
obtained by linear transformation of the original vector variable. This assertion is in fact true
is the third term is null as it is not invariant by invertible transformation. This term which
would correspond to a PCA criterion is null when the data is spherised. One can see the analogy
with Projection-Pursuit where projections of the data who are the least Gaussian are looked for
often using also the negentropy as non-Gaussianity measure. The difference is that in
Projection-Pursuit [8] the projections are not necessarily fixed to unidimensional ones.
Next: Single and Multiple ICA
Up: tr01dl1
Previous: Variance criterion
Didier Leibovici
2001-09-06