next up previous
Next: Accuracy and dimensionality Up: Results Previous: Artificial signal in synthetic

Spatio-temporal accuracy of PICA

We analysed the data set with artificial activation introduced into baseline FMRI data using PICA and GLM. The exact activation time courses were used within the GLM as regressors of interest. This will introduce a small bias in favour of the GLM analysis which we prefer compared to the alternative where we would have to artificially encode the more plausible ignorance that normally exists over the exact shape of the signal within the data. In order to estimate the consistency of the probabilistic ICA approach, the analysis was repeated 150 times in order to evaluate the repeatability between runs5. Table 1 summarises the mean correlation between the estimated and true time courses over all 150 runs while figure 6 shows the ROC curves for both GLM and PICA Naturally, the estimation accuracy improves with increased signal level. Note that the ROC curves for the GLM are not monotonically increasing. This is a direct consequence of the ambiguity built into the statistical thresholding steps based on Gaussian Random Field Theory6, where a $ Z$-threshold level is combined with a significance level for cluster heights or size. In the present case, we evaluated different sets with $ Z$ ranging from $ 1.1$ to $ 7.0$ and $ p$ ranging from $ 0.0005$ to $ 0.1$. For fixed $ Z$ or fixed $ p$, a monotonically increasing ROC curve can be plotted but for reasons of simplicity, we ordered all results by increasing false positive rate and in the case of multiple true positive outcomes only used the best one. In almost all cases, the PICA estimates show an improved ROC characteristics compared to the GLM results despite the fact that GLM analysis was carried out under the ideal condition of perfect knowledge of the regressors of interest. This is due to the fact that a standard GLM analysis is adversely affected by the presence of un-modelled structured noise in this data. The PICA decomposition, on the other hand, estimates sufficiently strong structured noise as separate components resulting in increased spatial accuracy for the activation component. Note that in the case of the GLM a difference in the $ Z$ threshold even within a commonly-used range of values leads to substantially different quality of estimation.

Table 1: Temporal accuracy at different activation levels: correlation between the extracted time courses and the true signal time courses over 150 runs.
  $ 0.5\% $ $ 1\% $ $ 3\% $ $ 5\% $
vis. $ 0.33\pm 0.03 $ $ 0.62\pm 0.01 $ $ 0.9\pm 0 $ $ 0.95\pm 0 $
aud. $ 0.29\pm 0.01 $ $ 0.5\pm 0.01$ $ 0.87\pm 0 $ $ 0.94\pm 0 $
       



Figure 6: Spatial accuracy at different activation levels: ROC curves for PICA (solid lines - mean over 150 runs) vs. ROC curves for $ Z$ statistical maps thresholded using Gaussian Random Field theory at different $ Z$ and $ p$ levels. Markers indicate typical threshold levels: 0.33,0.5 and 0.66 (PICA alternative hypothesis test); $ Z> 1.6, 2.3$ and $ 3.1$ (GLM null-hypothesis test).


next up previous
Next: Accuracy and dimensionality Up: Results Previous: Artificial signal in synthetic
Christian F. Beckmann 2003-08-05