How many percents explained variance by the first 50 principal components of PCA analysis is suitable for downstream `KNN` and `umap`?

Question

In single-cell RNA sequencing analysis, I am wondering how many percents explained variance by the first 50 principal components of PCA analysis is suitable for downstream KNN and umap?

If there are lots of samples (>40) and cells (>100,000) to analyze together, there are a lot of highly_variable_genes (>4000) can be found

import scanpy as sc
sc.pp.highly_variable_genes(adata, flavor='cell_ranger', n_top_genes=None)

The explained variance by the first 50 principal components of PCA analysis sc.pp.pca(adata, n_comps=50, use_highly_variable=True, svd_solver='arpack') is very low (<12%), increase the number of principal components does not help a lot. I assume that the low explained variance of PCs will make KNN and umap difficult to separate the cells well. Am I correct?

sc.pp.neighbors(adata, n_neighbors=15)
sc.tl.umap(adata, min_dist=0.3, spread=2.0)

I can reduce the number of highly_variable_genes to get more explained variance by the first 50 principal components, but KNN and umap still can not separate the cells well.

I checked the first 5 PCs when I reduce the number of highly_variable_genes, and it doesn't look very high:

>>> np.cumsum(adata.uns['pca']['variance_ratio'])

array([0.08419679, 0.12635055, 0.13610176, 0.14490107, 0.15326007])

Any suggestions would be greatly appreciated.

Answer 1

Edit To answer the question, you're looking for 80% variance .... from the comments the actual command is np.cumsum which changes things a lot.

Previous response ... Firstly it's still good work. It wouldn't be sensible to proceed with KNN if 12% explained 50 PCs, now there are 15% in 5 PCs.

You wrote,

0.08419679, 0.12635055, 0.13610176, 0.14490107, 0.15326007

PC1 ~ 0.085, PC2 ~ 0.068

I think that you've cross checked the analysis and that is a good sign of good analysis.

The only easy possibility is to completely tighten down on all "variable genes" parameter. The total variance for the analysis think 50% total variance rather than an idealistic 80%.

The emergency measures ...

• Simply use say 10 samples
• Reducing the total number of genes, e.g. removing more variable genes looks good, anything else
• Check the input internally to make sure there is no formatting bug
• Check via Seraut

The thing I would cross check is reducing the sample size a lot ... is there a stray sample(s) here that is throwing everything out? That would be thing I'd really look into. What happens to the total variance when say 20 samples, or 10 samples are used?

Just to finalise, are there any natural population divisions in the sample that could be used? If this results in a huge jump in variance that would be an answer. The only thing that is unusual is that there are a lot of samples for this type of study (from my perspective).