Hi Alfonso,

This is turning into a very interesting conversation!

Apologies for the error in the code, you are indeed correct on all counts. However, I should clarify what I meant as I was simply saying that partitioned-errors are more sensitive than pooled errors and that the two-stage approach, although claiming to implement partitioned errors, does not agree with the partitioned error approaches in other software. My point was just that we need to get the partitioned-errors right.

Even with the changes to the code, the approaches do not agree. I think we should be aiming for better than "close enough" with these methods, and the two-stage approach (based on Will Penny's recommendations) is still not estimating the F-statistic correctly nor the degrees of freedom for the within-subject models with > 2 levels. This can't be due to differences in the treatment of sphericity because the SPSS results I've reported have not adjusted for non-sphericity and nor have the SPM results. As such, the results should be identical.

My thinking is that the non-sphericity correction should not be necessary in order to achieve the "correct" F-statistic when sphericity is assumed. This speaks to a larger issue in terms of the GLM implementation in other packages that do assume sphericity (such as FSL) where the model we use should allow us to achieve the classical results, under assumptions of sphericity. If one wished to assume sphericity then the two-level models (given in the script I sent, not the ones you have indicated) do not actually achieve the classical results under assumptions of sphericity. My aim would be to get the models to agree under assumptions of sphericity and then apply the non-sphericity correction, as I don't think the correction should be seen as an integral part of the model, but rather an adjunct control for liberal/conservative conclusions when the assumptions are not adequately met.

Additionally, although the code you provided does give the same F-statistic it does not appear to give the same degrees of freedom (SPSS-like = 1.9781,55.3860 [which agrees with the Greenhouse-Geisser correction], CONN-like [2-dimensions] = 1.5814, 44.2780, CONN-like [3-dimensions] = 1.9781,55.3860 [again, Greenhouse-Geisser]). And so the 2-dimension model will give a different p-value to the SPSS and 3-dimension model. I suppose the question in then whether this is to be expected or not as to my mind all the models are testing the same hypothesis with the same data and so the conclusions should be invariant to this sort of choice? This is the claim made by the two-stage approach, but as far as I can tell it still does not agree.

Furthermore, even if the degrees of freedom agree with the Greenhouse-Geisser correction, this correction should be applied to the original F-statistic of 1.167, rather than the F-statistic of 1.1006 produced by the code. As such, the SPSS Greenhouse-Geisser corrected results of F(1.98,55.39) = 1.167, p = 0.319 is still not matched by the non-sphericity-corrected results of F(1.98,55.39) = 1.1009, p = 0.340.

I know this seems like splitting hairs, but we need to be convinced that the models we are using are appropriate and I still think the best way to do so is make sure we can achieve equivalency with the classical results in the textbooks (and implemented in other software) before adding on additional corrections (particularly when these corrections are not available in all software). This is easy enough when dealing with 2-levels, but > 2-levels is still a sticking point for me.

Looking forward to hearing your thoughts!

Best wishes,
- Martyn