How to calculate p-values for fold changes?

Question

I'm currently working with data from a Luminex multiplex assay. In this assay, the concentrations of 17 different analyte proteins were identified in 12 groups in triplicate. One of these 17 groups was used as the control, and the log2 fold changes were calculated for the analyte concentration of each sample in each group using the average control concentration for that analyte.

However, now I would like to calculate a p-value for the identified fold changes if possible.

My current preliminary idea is to perform the t test for each group compared with the control group for each analyte, but I don't think that my group sizes are large enough for this. My statistical knowledge is lacking, so if there is a better way of calculating these p-values I don't know about it.

Is there a better way of obtaining these p-values?

Answer 1

Your null hypothesis would be that the fold-changes are 0, so you can either do the T-test accordingly or simply do away with the fold changes and perform the T-test between the raw values in the group (this is preferable to performing a T-test of one group vs. 0, since it allows you to assess the expected variability around 0). Note, however, that 3 samples is pretty much the bare minimum needed for any statistics, so your power will be terrible and your estimation of variance will likely be pretty inaccurate. Consequently, take any p-values with an appropriate grain of salt.

To my knowledge there aren't any good alternatives for your situation without having a good expected background distribution.

Answer 2

The key point here is whether or not the values are approximately normally distributed and whether any transformations can be applied to make them so.

For a t-test, the most important thing is that there is no relationship between the mean and the variance.

Its also correct that you don't want to divide by the mean of the controls. You want to retain all the information.

So here is how i'd start, first calculate the mean and standard deviation of each of your 18 groups (17 treatments + control), and plot this on a scatter plot (i.e. mean on X, SD on Y).

Is there a relationship between the two?

Repeat the exercise with log transformed values. Is the relationship more or less.

Pick the set of values that has the weakest mean-SD relationship. I'm guessing the log transformed might well be better.

If $X = (x_1, x_2, x_3)$ is your controls and $Y_i = (y_{i,1}, y_{i,2}, y_{i,3})$ are the treated samples from treatment i, to test the difference between treatment and control, do a t-test on X and Y (not T/mean(c))

for example in R

X = c(x_1, x_2, x_3)
Y_1 = c(y_11, y_12, y_13)
t.test (X, Y_1)

or in excel is if the three values for X are in column A and the three values for $Y_1$ in column B

=T.TEST(A1:A3,B1:B3,2,3)

If you are going to do 17 of these, remember to do a multiple testing correction by multiplying the resulting p-values by 17.

If you want to compare treatment 1 ($Y_1$) vs treatment 2 ($Y_2$) do the comparison directly, not taking the control into account:

t.test(Y_1, Y_2)

Remember that if you use log transformed values, you calculate log fold changes using mean(Y)-mean(X) rather than mean(Y)/mean(X)

Answer 3

You can't calculate a p-value on the fold-change values, you need to use the concentrations in triplicate thus giving a measure of the variance for the t-test to use.

t-test assumes your data are normally distributed, if they aren't you're going to get spurious p-values. If you aren't sure a non-parametric test like Wilcoxon is better. It will be less sensitive although with only 3 replicates your experiment is low powered anyway.

Answer 4

So to get a p value you must have one null hypothesis like the fold changes are significant or not. Then you test the hypothesis against the data and calculate the p value (likeliness or unlikeliness) for a given threshold.

In your case, t test is good enough to calculate the p-value.

http://willett.ece.wisc.edu/wp-uploads/2016/01/05b-TandP.pdf