How to fix the Error in Kruskal Wallis rank sum test for comparing the guides in python? Also to check whether using the right test for my data?

Question

df

gene_name   guide_1 guide_1_new Correlation
MMP-1   A   A   1
MMP-1   A   B   0.426115
MMP-1   A   C   0.522499
MMP-1   A   D   0.431587
MMP-1   B   A   0.426115
MMP-1   B   B   1
MMP-1   B   C   0.60113
MMP-1   B   D   0.534858
MMP-1   C   A   0.522499
MMP-1   C   B   0.60113
MMP-1   C   C   1
MMP-1   C   D   0.622206
MMP-1   D   A   0.431587
MMP-1   D   B   0.534858
MMP-1   D   C   0.622206
MMP-1   D   D   1

I used four guides in an experiment (data = df). I calculated correlation between them based on the their log fold changes. I want to check which guide is outperforming or underperforming in comparison to other three guides in my experiment. I tried Kruskal and Wallis test to do this analysis with the script given below. However, I got an error ValueError: Samples must be one-dimensional. Also is this the right test to do this analysis?

My script

from scipy import stats

x = df[['guide_1']]
y = df[['guide_1_new']]
z = df[['Correlation']]
stats.kruskal(x, y, z)

Answer 1

Reformatting the data for the KW and doing the test is easy.

Using "guide_1" alone and pivot() to get A = [1, 0.426115 ...

df2 = df.pivot(columns='guide_1', values='Correlation')

This will now present A, B, C, D etc ... into columns...

x = df2[['A']].dropna() 
y = df2[['B']].dropna() 
z = df2[['C']].dropna() 
a = df2[['D']].dropna() 

stats.kruskal(x, y, z, a)

That will work ... whether that is a good test is a complicated question because this is log transformed data. Transformation is used in parametric tests. Removing the log transformation is strongly recommended because KW works with raw data.

You can switch the calculation to guide_1_new instead. It will give a different answer ... but I don't know the experiment. The original guide looks cool.

Looking at the data - the test should work because I believe KW is 'unpaired' (you need to check) and looking at the value the distribution looks pretty similar between A, B, C and D. However, just because the KW says 'yes' doesn't mean thats a robust answer (log data). If the untransformed data says 'yes' ... thats a robust answer for KW. Just check the 'unpaired' stuff - thats important for the result.

---

Personal note, I originally used a third df but was not needed,

df3 = pd.concat([df2[col].sort_values().reset_index(drop=True) for col in df2], axis=1, ignore_index=True)