help
help > RE: calculating cohen's d from rZ values
Dec 27, 2014 03:12 AM | Alfonso Nieto-Castanon - Boston University
RE: calculating cohen's d from rZ values
Hi Patrick,
Thanks for the clarification. The only thing I am not sure yet I am interpreting correctly is whether in step (3) you looked at the main effect across both subject groups (i.e. select both groups and enter a [.5 .5] contrast) or at the between-group differences (i.e. select both groups and enter a [-1 1] contrast) when defining the significant target ROIs. I am assuming you probably meant the latter.
In that case the Cohen's d values computed in step (5) will show some amount of bias due to the original selection of ROIs based on a non-orthogonal contrast (the ROIs created in each iteration of step (3) are based on the same between-subject contrast [-1 1] as that effectively looked at when computing the Cohen's d between-group difference measures). The bias will be smaller but still there if only the first iteration of step (3) was based on the same [-1 1] contrast while the second iteration of step (3) used the orthogonal contrast [.5 .5] instead, and there would be no bias if both of the iterations of step (3) were based on the orthogonal contrast [.5 .5] instead. Note also that these biases will translate to invalid statistics for your analyses in step (5) and (6), but not in the directionality tests in step (7) since the amount of inflation is not expected to have any preferred directionality. So overall I would say that (up to the interpretation of which second-level between-subjects contrast you used to define the different ROIs) you are likely incurring in some amount of "double-dipping". Of course this does not mean that your analyses are incorrect, I do believe the analyses are perfectly fine as a way to further explore your found effects, you should just phrase them as post-hoc exploratory analyses instead of confirmatory hypothesis testing and let the readers/reviewers know of the associated limitations (e.g. require independent replication with a-priori/independent ROI selection procedure)
Let me know if you would like me to further clarify any of this
Best
Alfonso
Originally posted by Patrick McConnell:
Thanks for the clarification. The only thing I am not sure yet I am interpreting correctly is whether in step (3) you looked at the main effect across both subject groups (i.e. select both groups and enter a [.5 .5] contrast) or at the between-group differences (i.e. select both groups and enter a [-1 1] contrast) when defining the significant target ROIs. I am assuming you probably meant the latter.
In that case the Cohen's d values computed in step (5) will show some amount of bias due to the original selection of ROIs based on a non-orthogonal contrast (the ROIs created in each iteration of step (3) are based on the same between-subject contrast [-1 1] as that effectively looked at when computing the Cohen's d between-group difference measures). The bias will be smaller but still there if only the first iteration of step (3) was based on the same [-1 1] contrast while the second iteration of step (3) used the orthogonal contrast [.5 .5] instead, and there would be no bias if both of the iterations of step (3) were based on the orthogonal contrast [.5 .5] instead. Note also that these biases will translate to invalid statistics for your analyses in step (5) and (6), but not in the directionality tests in step (7) since the amount of inflation is not expected to have any preferred directionality. So overall I would say that (up to the interpretation of which second-level between-subjects contrast you used to define the different ROIs) you are likely incurring in some amount of "double-dipping". Of course this does not mean that your analyses are incorrect, I do believe the analyses are perfectly fine as a way to further explore your found effects, you should just phrase them as post-hoc exploratory analyses instead of confirmatory hypothesis testing and let the readers/reviewers know of the associated limitations (e.g. require independent replication with a-priori/independent ROI selection procedure)
Let me know if you would like me to further clarify any of this
Best
Alfonso
Originally posted by Patrick McConnell:
Alfonso,
Thank you very much for the thoughtful and detailed reply. I'm sorry that I wasn't clearer in my original post. From c) your understanding/interpretation is spot-on. Just to rephrase again:
1) seed --> voxel bivariate regression with 5-mm spheres and a priori non-directional hypotheses.
2) marsbar --> write functional clusters from SPM.mat in SPM; extract eigenvariates from these clusters to explore directionality of group mean correlation (positive v inverse / strength).
3) enter new rois back into seed--> voxel bivariate correlation analysis; repeat step 2) for any significant target rois.
4) repeat step 3) one more time.
5) calculate cohen's d for each group using group mean eigenvariate for each significant roi.
6) bivariate regression ROI-ROI analysis within-groups using previously determined ROIs.
7) where both directions (e.g., roi1 --> roi2, roi2 --> roi1) were significant, extract single-subject regression coefficients and run paired-sample t-tests on roi1 --> roi2 vs. roi --> roi2 to determine if there is any indication of effective connectivity.
8) we also extracted denoised time series for each roi to explore visually.
Thanks again for the help!!!
-Patrick
Thank you very much for the thoughtful and detailed reply. I'm sorry that I wasn't clearer in my original post. From c) your understanding/interpretation is spot-on. Just to rephrase again:
1) seed --> voxel bivariate regression with 5-mm spheres and a priori non-directional hypotheses.
2) marsbar --> write functional clusters from SPM.mat in SPM; extract eigenvariates from these clusters to explore directionality of group mean correlation (positive v inverse / strength).
3) enter new rois back into seed--> voxel bivariate correlation analysis; repeat step 2) for any significant target rois.
4) repeat step 3) one more time.
5) calculate cohen's d for each group using group mean eigenvariate for each significant roi.
6) bivariate regression ROI-ROI analysis within-groups using previously determined ROIs.
7) where both directions (e.g., roi1 --> roi2, roi2 --> roi1) were significant, extract single-subject regression coefficients and run paired-sample t-tests on roi1 --> roi2 vs. roi --> roi2 to determine if there is any indication of effective connectivity.
8) we also extracted denoised time series for each roi to explore visually.
Thanks again for the help!!!
-Patrick
Threaded View
| Title | Author | Date |
|---|---|---|
| Crystal Goh | Jun 2, 2012 | |
| Alfonso Nieto-Castanon | Jul 8, 2012 | |
| Patrick McConnell | Dec 20, 2014 | |
| Alfonso Nieto-Castanon | Dec 21, 2014 | |
| Patrick McConnell | Dec 21, 2014 | |
| Alfonso Nieto-Castanon | Dec 24, 2014 | |
| Patrick McConnell | Dec 24, 2014 | |
| Patrick McConnell | Jan 8, 2015 | |
| Alfonso Nieto-Castanon | Dec 27, 2014 | |
| Patrick McConnell | Jan 7, 2015 | |
| Alfonso Nieto-Castanon | Jan 15, 2015 | |
| Patrick McConnell | Jan 22, 2015 | |
| Michael King | Dec 16, 2014 | |
| Alfonso Nieto-Castanon | Dec 17, 2014 | |
| Michael King | Dec 17, 2014 | |
| Michael King | Dec 17, 2014 | |