5
$\begingroup$

I am looking at the supplemental data from the paper "An allelic series of miR-17 ∼ 92-mutant mice uncovers functional specialization and cooperation among members of a microRNA polycistron" which lists the genes that are differentially expressed between a particular knock-out mouse and the wild-type control. In addition to showing the logFC changes for each of the genes, the table includes their FDR value. In many cases, the FDR value is 1. What does this mean?

$\endgroup$
3
  • $\begingroup$ A link/doi to the paper would be helpful. $\endgroup$
    – Bryan Krause
    Dec 1, 2018 at 3:07
  • $\begingroup$ Sure! Here is the link: ncbi.nlm.nih.gov/pubmed/26029871 Thank you very much, Leah $\endgroup$
    – leah
    Dec 1, 2018 at 18:20
  • 3
    $\begingroup$ @leah I don't see where they are reporting FDR values of 1 in that paper, but Mowgli's answer is correct for what FDR is and likely correct that the numbers you are seeing are meant to be FDR corrected p-values. They shouldn't be called "FDR values" though so the authors might have made a mistake or just an unfortunate choice of abbreviation (or you did). FDR is a threshold, sort of like when you choose an alpha value as a criterion for significance. $\endgroup$
    – Bryan Krause
    Dec 2, 2018 at 0:30

2 Answers 2

9
$\begingroup$

FDR stands for False Discovery Rate. It is a statistic tool used in multiple hypothesis testing.

As you may know, when you use a p-value cutoff (usually 0.05) for your experiments, it means strictly speaking that "is there was actually no signal, there would be a probability of 0.05 to observe this kind of extreme values". This can be understood as "there's a 5% chance that what you call a hit is actually not signal, just some extreme value that happened by chance, and really there's nothing to see here".

This definition has deep consequences if you run many experiments: imagine that you run 100 experiments (say, you test the effect of 100 different molecules), if none of them has any effect, on average you could still get 5% of false positives. So if you run these 100 experiments, and you get, say, 10 hits with a p-value below your thresold of 0.05; how can you tell which ones are real and which ones are just chance in the absence of biological signal?

This is where you can use FDR.

If you control for a false discovery rate of, say, 0.2 (20%), it means that after a few computations (the Benjamini-Hochberg procedure described on wikipedia is simple and commonly used) you will lower your p-value cutoff so as to be sure that, among the N hits (probably less than the 10 you initially selected above) that you got above, a maximum of 20% are actually false positives (i.e. no biological effect, but surprisingly high/low signal values by pure chance).

In the article you mention they have a different FDR value for each sample, which is unusual. The values are not round numbers, so it seems likely that they are numbers obtained after applying the FDR correction (i.e. they are "transformed p-values") on which they based their decision to include the samples or not. The FDR cutoff that they chose should be written somewhere in the main text.

$\endgroup$
2
$\begingroup$

In practice, you can interpret it just like a p-value.

An FDR value is a p-value adjusted for multiple tests (by the Benjamini-Hochberg procedure). It stands for the “false discovery rate” it corrects for multiple testing by giving the proportion of tests above threshold alpha that will be false positives (i.e., detected when the null hypothesis is true). Note this is less stringent than the Holm-Bonferroni FWER adjusted p-value (which considers the chance of any false positives among significant results).

So for a FDR p-value of 0.05, up to 5% of these tests will be false positives (note this is not the case for uncorrected raw p-values).

For an FDR p-value of 1, up to 100% of these tests will be false positives. This makes sense since if you take test with a p-value if 1, then it should include all of the negative results, where the null hypothesis cannot be rejected. Only tests with p-values lower than this, have any chance of being rejected under the null hypothesis as true positives. If you are getting FDR adjusted p-values, you have done so many tests that you do not have the power to detect true positives and cannot rule out these being false positives due to noise of multiple testing. So an FDR p-value of 1 is definitely not significant under any circumstances.

$\endgroup$
5
  • $\begingroup$ I think you cannot interpret it exactly like a p-value in a Benjamini Hochberg procedure. It requires 1) ranking the p-values, 2) computing adjusted values based on rank and 3) finding the largest index satisfying the FDR inequality computation and 4) selecting all the values below this index. There may be values that end up being selected (ie. their index is sufficiently small) even though the computation leads to a value above your selection threshold. $\endgroup$
    – Mowgli
    Dec 3, 2018 at 2:56
  • $\begingroup$ Additionally, an FDR just tells you about the maximum percentage of false positives that you are willing to accept - so even correcting for an FDR of 1 can lead to 100% of correct hits, it just means that you are willing to accept up to 100% of type I errors in a worst-case scenario (and you just don't know if it happens or not) $\endgroup$
    – Mowgli
    Dec 3, 2018 at 2:58
  • $\begingroup$ For the case of p = 1 or in terms of interpreting the results of an experiment, it’s not much different. Any algorithm to compute FDR correction will compute them in order for you. It’s not necessary for a biologist to know the details, although this may be more appropriate for Cross Validated. $\endgroup$
    – Tom Kelly
    Dec 3, 2018 at 2:59
  • $\begingroup$ I don't quite agree; this nuance can add tens of hits to your results in a large scale experiment like high-throughput RNAseq. And yes, I think it's necessary for a biologist - or any other scientist, for that matter - to understand precisely the statistical tools they use for research :) my personal opinion. $\endgroup$
    – Mowgli
    Dec 3, 2018 at 3:07
  • 2
    $\begingroup$ For this specific case, it is a question about FDR of 1. This should not be used as a threshold for defining hits under any circumstances. This will make every result a hit by definition. It’s the most false positives you will allow and you are allowing all them. Researchers should understand the tools they’re using but they should seek further information on them, this is not the right forum for that level of detail. $\endgroup$
    – Tom Kelly
    Dec 3, 2018 at 3:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.