1
$\begingroup$

I encountered a problem while performing analysis using the PANTHER website. I run the statistical overrepresentation test for a genelist of 99 genes (GO-Slim BP, Binomial, Bonferroni) in February 2020. My result was Bonferroni count = 1 and a list of several enriched GO terms. Next we run the same analysis with the same settings in July 2020 (two days ago) and the result is dramatically different, the Bonferroni count is 1770 and no enriched terms are found.

I am confused - is the result Bonferroni count = 1 possible? Why these results are so different?

This is how the correction is described on PANTHER website:

The expression data analysis statistics now include a Bonferroni correction for multiple testing. The Bonferroni correction is important because we are performing many statistical tests (one for each pathway, or each ontology term) at the same time. This correction multiplies the single-test P-value by the number of independent tests to obtain an expected error rate.

For pathways, we now correct the reported P-values by multiplying by the number of associated pathways with two or more genes. Some proteins participate in multiple pathways, so the tests are not completely independent of each other and the Bonferroni correction is conservative. For ontology terms, the simple Bonferroni correction becomes extremely conservative because parent (more general) and child (more specific) terms are not independent at all: any gene or protein associated with a child term is also associated with the parent (and grandparent, etc.) terms as well.

To estimate the number of independent tests for an ontology, we count the number of classes with at least two genes in the reference list that are annotated directly to that class (i.e. not indirectly via an annotation to a more specific subclass).

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

GO-Slim BP, Binomial, Bonferroni

This seems easy you because I suspect have run too many tests at 5%.

If you run 10 tests at 5% rejection threshold, the Bonferroni correction threshold is 0.5% - you divide the number of tests by the rejection threshold.

One possibility is the rejection threshold is given in number of genems: in one instance you've run a pairwise test with several thousand genes so the rejection threshold has been set accordingly. The other test you ran I suspect was a multi-variate test, thus the shift in rejection threshold measured in number of genes.

The altnative is that Bonf = 1 is in fact P = 1, in other words there is no significant difference at all. If P = 1 then this could be no genes show deviation hence 1770, because 1770/1770 =1

Looking at it again the above seems a better explanation.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I am not sure if understand what you mean. I run binomial test with Bonferroni correction in both cases using the same gene set. Once in April, next in July. I was using the PANTHER website so I have no access to the code but the settings and the input were exactly the same: nr of genes: 99 statistics: Binomial correction: Bonferroni $\endgroup$
    – Anna Maria Bugaj
    Aug 3, 2020 at 10:53
  • $\begingroup$ I've put an alternative explanation above, without knowing the basis of the test its the best guess. $\endgroup$
    – M__ ♦
    Aug 3, 2020 at 11:00
  • $\begingroup$ The p-value is not 1, there are several terms with p< 0.05 in the first case (with Bonferroni count = 1)<br/> Analysis Type: PANTHER Overrepresentation Test (Released 20200407)<br/> Annotation Version and Release Date: PANTHER version 15.0 Released 2020-02-14<br/> Analyzed List: Dupl_N1_WT_0503_list.txt (Mus musculus)<br/> Reference List: Mus musculus (all genes in database) <br/> Test Type: BINOMIAL <br/> Correction: BONFERRONI <br/> Bonferroni count: 1 <br/> (P-value)<br/> 1.19E-02<br/> 1.19E-02<br/> 1.19E-02<br/> $\endgroup$
    – Anna Maria Bugaj
    Aug 3, 2020 at 11:03
  • $\begingroup$ I am sorry for the formatting $\endgroup$
    – Anna Maria Bugaj
    Aug 3, 2020 at 11:09
  • $\begingroup$ No you ran the Binomial, Bonferroni. The red alert is the Bonferroni correction, because this is classically when things fall-over. $\endgroup$
    – M__ ♦
    Aug 3, 2020 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.