# RNA-Seq: clustering/treatment of genes with low expression

I have some RNA-Seq data from leukaemia patients. I want to do unsupervised clustering on them with some other published leukaemia RNA-Seq data and see how they cluster. There are a few problems I encountered while doing this.

1. I read mix messages of whether using log2(TPM) or rlog(counts) (e.g. Deseq2 rlog or limma voom transformation) for clustering. Which one should I choose?

2. If I am filtering out genes with low counts, should I do it prior normalisation to library size or after?

3. I tried using filterByExpr from edgeR for filtering but it removed many of the genes that are only expressed by subtypes with small sample number. If I am to filter the counts "manually", is there a recommendation of how the threshold should be set?

4. If I should use log2(TPM) for unsupervised clustering, can I treat low TPM (e.g. <2) as 0? As this paper suggested TPM < 2 basically means no expression, is it alright to treat them as 0?

1. rlog(normalized counts) is going to be more robust than log2(TPM), so use it instead.
3. This is very much more of an art than a science, though have a look at the kOverA() function from the genefilter package. That will give you a bit more fine-grained control on filtering.