Hi,

Thank you very much for this great tool. I have a question, as follows. 

I performed two analyses after obtaining subject-specific directed functional connectivity: 1) used dNBS with primary threshold 1.8, 10K permuations, and size based on extent, to obtain a subnetwork with lower strength in diseased group compared to healthy group at significance level 0.05. 2) obtained edge-wise p-values for lower strength in diseased group compared to healthy group using Welch's t-test for each edge in the directed functional connectivities. 

However, the subnetwork obtained in analysis 1 is not a subset of edges having edge-wise p-value less than 0.05 in analysis 2. Isn't this wierd? I would really appreciate an explanation.

Some more detail: Only two edges in the subnetwork in analysis 1 have edge-wise p-value less than 0.05 in edge-wise Welch's t-test (analysis 2).

Thank you very much for explaining this.

Best regards,
Rahul

Hi Rahul, 

this is most likely due to your use of a lenient primary threshold (t=1.8). 

If the sample size is not sufficiently large, t=1.8 will not ensure that p<0.05 and thus it is unsurpising that several connections do not have p<0.05.

I recommend increasing the primary threshold. Perhaps try t=3.  

Andrew

Originally posted by Rahul Biswas:

Hi,

Thank you very much for this great tool. I have a question, as follows. 

I performed two analyses after obtaining subject-specific directed functional connectivity: 1) used dNBS with primary threshold 1.8, 10K permuations, and size based on extent, to obtain a subnetwork with lower strength in diseased group compared to healthy group at significance level 0.05. 2) obtained edge-wise p-values for lower strength in diseased group compared to healthy group using Welch's t-test for each edge in the directed functional connectivities. 

However, the subnetwork obtained in analysis 1 is not a subset of edges having edge-wise p-value less than 0.05 in analysis 2. Isn't this wierd? I would really appreciate an explanation.

Some more detail: Only two edges in the subnetwork in analysis 1 have edge-wise p-value less than 0.05 in edge-wise Welch's t-test (analysis 2).

Thank you very much for explaining this.

Best regards,
Rahul

Hi Andrew,

Thanks for your answer. Just a quick follow-up with details on the sample size.

We have a sample size of 34 subjects in the diseased group and 41 subjects in the healthy group. May I double check, if you still think this is the same issue of sample size that you mentioned? By the way, I checked that t = 1.8 would have a right-sided p-value < 0.05 for t-distribution with df = 73. Perhaps the distribution of the t statistic computed by dNBS is different.

Indeed, for t = 2.5, the subnetwork has three edges, for which the edge-wise p-value of two edges are less than 0.05 and that of the third edge is 0.056. t = 3 did not yield any significant outcomes.

Thank you!

Best regards,
Rahul

Originally posted by Andrew Zalesky:

Hi Rahul, 

this is most likely due to your use of a lenient primary threshold (t=1.8). 

If the sample size is not sufficiently large, t=1.8 will not ensure that p<0.05 and thus it is unsurpising that several connections do not have p<0.05.

I recommend increasing the primary threshold. Perhaps try t=3.  

Andrew

 

Originally posted by Rahul Biswas:

Hi,

Thank you very much for this great tool. I have a question, as follows. 

I performed two analyses after obtaining subject-specific directed functional connectivity: 1) used dNBS with primary threshold 1.8, 10K permuations, and size based on extent, to obtain a subnetwork with lower strength in diseased group compared to healthy group at significance level 0.05. 2) obtained edge-wise p-values for lower strength in diseased group compared to healthy group using Welch's t-test for each edge in the directed functional connectivities. 

However, the subnetwork obtained in analysis 1 is not a subset of edges having edge-wise p-value less than 0.05 in analysis 2. Isn't this wierd? I would really appreciate an explanation.

Some more detail: Only two edges in the subnetwork in analysis 1 have edge-wise p-value less than 0.05 in edge-wise Welch's t-test (analysis 2).

Thank you very much for explaining this.

Best regards,
Rahul

 

Hi Rahul, 

I suspect that this may be caused by slight differences in how you and dNBS are computing the t-test. The computation of the denominator (standard deviations) varies between t-tests. I'm not familiar with how the t-test is computed in dNBS - perhaps others can comment. Of course it could also be a bug in the dNBS code! Let us know if you look into this issue in more detail. 

However, your intuition is correct. If you set a primary threshold of p<0.05 (or corresponding t-stat) all edges within a signifciant component should be p<0.05. 

Best wishes, 

Andrew

Originally posted by Rahul Biswas:

Hi Andrew,

Thanks for your answer. Just a quick follow-up with details on the sample size.

We have a sample size of 34 subjects in the diseased group and 41 subjects in the healthy group. May I double check, if you still think this is the same issue of sample size that you mentioned? By the way, I checked that t = 1.8 would have a right-sided p-value < 0.05 for t-distribution with df = 73. Perhaps the distribution of the t statistic computed by dNBS is different.

Indeed, for t = 2.5, the subnetwork has three edges, for which the edge-wise p-value of two edges are less than 0.05 and that of the third edge is 0.056. t = 3 did not yield any significant outcomes.

Thank you!

Best regards,
Rahul

Originally posted by Andrew Zalesky:

Hi Rahul, 

this is most likely due to your use of a lenient primary threshold (t=1.8). 

If the sample size is not sufficiently large, t=1.8 will not ensure that p<0.05 and thus it is unsurpising that several connections do not have p<0.05.

I recommend increasing the primary threshold. Perhaps try t=3.  

Andrew

 

Originally posted by Rahul Biswas:

Hi,

Thank you very much for this great tool. I have a question, as follows. 

I performed two analyses after obtaining subject-specific directed functional connectivity: 1) used dNBS with primary threshold 1.8, 10K permuations, and size based on extent, to obtain a subnetwork with lower strength in diseased group compared to healthy group at significance level 0.05. 2) obtained edge-wise p-values for lower strength in diseased group compared to healthy group using Welch's t-test for each edge in the directed functional connectivities. 

However, the subnetwork obtained in analysis 1 is not a subset of edges having edge-wise p-value less than 0.05 in analysis 2. Isn't this wierd? I would really appreciate an explanation.

Some more detail: Only two edges in the subnetwork in analysis 1 have edge-wise p-value less than 0.05 in edge-wise Welch's t-test (analysis 2).

Thank you very much for explaining this.

Best regards,
Rahul

 

 

Thank you very much Andrew for your kind and helpful answers!

Originally posted by Andrew Zalesky:

Hi Rahul, 

I suspect that this may be caused by slight differences in how you and dNBS are computing the t-test. The computation of the denominator (standard deviations) varies between t-tests. I'm not familiar with how the t-test is computed in dNBS - perhaps others can comment. Of course it could also be a bug in the dNBS code! Let us know if you look into this issue in more detail. 

However, your intuition is correct. If you set a primary threshold of p<0.05 (or corresponding t-stat) all edges within a signifciant component should be p<0.05. 

Best wishes, 

Andrew

 

Originally posted by Rahul Biswas:

Hi Andrew,

Thanks for your answer. Just a quick follow-up with details on the sample size.

We have a sample size of 34 subjects in the diseased group and 41 subjects in the healthy group. May I double check, if you still think this is the same issue of sample size that you mentioned? By the way, I checked that t = 1.8 would have a right-sided p-value < 0.05 for t-distribution with df = 73. Perhaps the distribution of the t statistic computed by dNBS is different.

Indeed, for t = 2.5, the subnetwork has three edges, for which the edge-wise p-value of two edges are less than 0.05 and that of the third edge is 0.056. t = 3 did not yield any significant outcomes.

Thank you!

Best regards,
Rahul

Originally posted by Andrew Zalesky:

Hi Rahul, 

this is most likely due to your use of a lenient primary threshold (t=1.8). 

If the sample size is not sufficiently large, t=1.8 will not ensure that p<0.05 and thus it is unsurpising that several connections do not have p<0.05.

I recommend increasing the primary threshold. Perhaps try t=3.  

Andrew

 

Originally posted by Rahul Biswas:

Hi,

Thank you very much for this great tool. I have a question, as follows. 

I performed two analyses after obtaining subject-specific directed functional connectivity: 1) used dNBS with primary threshold 1.8, 10K permuations, and size based on extent, to obtain a subnetwork with lower strength in diseased group compared to healthy group at significance level 0.05. 2) obtained edge-wise p-values for lower strength in diseased group compared to healthy group using Welch's t-test for each edge in the directed functional connectivities. 

However, the subnetwork obtained in analysis 1 is not a subset of edges having edge-wise p-value less than 0.05 in analysis 2. Isn't this wierd? I would really appreciate an explanation.

Some more detail: Only two edges in the subnetwork in analysis 1 have edge-wise p-value less than 0.05 in edge-wise Welch's t-test (analysis 2).

Thank you very much for explaining this.

Best regards,
Rahul