# What could be the reason for the samples not clustering?

I'm performing RNA-seq analysis. I have used Hisat2 for aligning reads to the genome and stringtie for quantification and extracted read count information directly from the files generated by StringTie using "prepDE.py" mentioned in Stringtie manual. With the counts data I have used edgeR for Differential analysis.

Before that I have seen MDS plot showing in the following way:

I see that three samples are not clustered and thought that they are not sequenced well. So I removed the three samples and again made an MDS plot.

Can anyone tell me what could be the reason for this type of clustering? Is that because of sequencing? or using count data directly from stringtie or anything else?

Samples will only cluster by experimental group if the experimental effect is large enough that it's the primary source of variance between your samples. If that's not the case then you'll get results like you're seeing. Whether that's a problem or not ends up depending on how large of an effect you expect to see. If your groups are different stages in differentiation or another large effect, then you expect that samples will cluster into groups. If you're looking at something like some large unaffected tissue in a disease vs. normal state then seeing a cloud of points is quit reasonable.

• Yes, you are right. If some thing went wrong what all should I check anything like mapped reads percentage or anything else? Apr 18 '18 at 13:41
• Mapped percentages, size factors, and perhaps sparsity (percentage of genes with no counts). Basically you want to ensure that they're all similar. Also, try redoing the counts with featureCounts. Then you can eliminate an oddity in the counting as an issue. Apr 18 '18 at 13:42
• Which is the mapping percentage in the above comments? It gives overall alignment rate. Apr 18 '18 at 18:28
• The alignment rate is the mapping percentage. Apr 18 '18 at 18:45
• Yes, the last column is the counts (I wish there were an option to simplify that output to just geneid and count). You probably don't need the -P option, not that it's likely hurting anything. BTW, that could be its own post here. Apr 19 '18 at 9:42

The largest variation differences in total expression will be seen in cell population differences. It's a good thing that your conditions aren't clustering on the first and second principal components when all genes are considered, because it means that cell population is probably not a factor in the variation of expression.

There may be some other principal component that does separate the conditions, but you might need to drill down fairly deep into the data. This will depend on how much the treatment conditions contribute to the total expression variation. We got lucky with our Th2 RNASeq study in that cell population was mostly on the first and second principal components, and treatment condition appeared on the third principal component.

If you want to force condition clustering, identify the genes that are differentially expressed, and only use those genes for the principal component analysis.

• I'm not sure how useful doing a PCA on DE genes is. You already know your samples differ by group in those genes, after all. Apr 19 '18 at 7:08