Hi,

I have questions about the FDR methods analysis. I apologize in advance if they are trivials:

1- can I use the fdr with both the t-test and f-test?

2- how do I interpret the results? I mean, I launched an analysis with FDR and 500 000 permutations and I got singnificant results. So I looked at the matrice nbs.NBS.con_mat to know the significant edges and usually (when I am using the network based statistics) I am plotting them (with BrainNet) with a weight corresponding to the matrix nbs.NBS.test_stat. However with the FDR method, I am getting value very closed to 0. Should I threshold my results (around 3) after getting the significant edges?

Thanks a lot for your help!

Cecile

Please see below: 

Originally posted by bordiercecile:

Hi,

I have questions about the FDR methods analysis. I apologize in advance if they are trivials:

1- can I use the fdr with both the t-test and f-test?

Yes. But it is important to use a very large number of permutations. With NBS, 5000 permutations is fine, but FDR may require 1 million or more permutations to infer accurate p-values. 

2- how do I interpret the results? I mean, I launched an analysis with FDR and 500 000 permutations and I got singnificant results. So I looked at the matrice nbs.NBS.con_mat to know the significant edges and usually (when I am using the network based statistics) I am plotting them (with BrainNet) with a weight corresponding to the matrix nbs.NBS.test_stat. However with the FDR method, I am getting value very closed to 0. Should I threshold my results (around 3) after getting the significant edges?

The interepretation is no different between NBS and FDR. I'm not sure what you mean by a value close to zero. No thresholding should be required. 

 

Thanks a lot for your help!

Cecile

Good morning!

Thanks a lot for your answer. 

I am running the FDR with 500 000 permutations, but you are right to be in the safe side I will increase to 1000 000.

My question about the thresholding is about the matrix NBS.test_stat. If I understood well this matrix represent the t (or f) values before the permutations. However, some of my FDR significant edges have very small value (<1) in the NBS.test_stat. So I was wondering if I should discard them afterwards when the t- or f-values are too "weak".

Thanks a lot for your help

Cecile

Yes - you could always threshold based on an effect size. An effect may be statistically significant but in the era of huge samples, the effect sizes becomes important. Tiny effects may be significant in a statistical sense but practically meaningnless. 

Note that t and F are not really effects sizes, but t can be converted to an effect size by normalizing by degrees of freedom. 

In short, yes it seems fine to threshold. 

Originally posted by bordiercecile:

Good morning!

 

Thanks a lot for your answer. 

I am running the FDR with 500 000 permutations, but you are right to be in the safe side I will increase to 1000 000.

My question about the thresholding is about the matrix NBS.test_stat. If I understood well this matrix represent the t (or f) values before the permutations. However, some of my FDR significant edges have very small value (<1) in the NBS.test_stat. So I was wondering if I should discard them afterwards when the t- or f-values are too "weak".

Thanks a lot for your help

Cecile