Converting your data.frame to a matrix (and then removing the data.frame) will often free up enough memory that you won't run into this. Note that a matrix is more memory efficient than a data.frame and you're requiring Rtsne() to hold both in memory at the same time (many math-centric functions will end up converting things to a matrix at some point for ...
A SC3, single-cell consensus clustering, approach could be helpful here. It aims at achieving "high accuracy and robustness by combining multiple clustering solutions through a consensus approach" https://www.nature.com/nmeth/journal/v14/n5/full/nmeth.4236.html
Clustering like this is typically done with scipy. Here's the code we use in deepTools (original context here):
from scipy.cluster.hierarchy import fcluster, linkage
Z = linkage(some_matrix) # You might want to set `method` and `metric`
groups = fcluster(Z, nGroups) # You might want to set `criterion`
groups is then a vector containing the group ...
I would be very hesitant to blindly exclude those samples based on the clustering. Check to see if the clusters actually denote some sort of batch effect, since it's not like all of the TCGA datasets were processed at the same place or time. Check for clinical covariates too. Also, do a PCA and check the genes with high loading on the relevant PC. If it ...
heatmap.2 is very configurable, and has options to adjust the things you want to fix:
cexRow: changes the size of the row label font.
keysize: numeric value indicating the size of the key.
The size of the key is also affected by the layout of the plot. heatmap.2 splits your plotting device into 4 panes (see the picture below), and you can control the ...
The goal of a phylogeny is to estimate the "expected" number of mutations between all taxa in the analysis and their hypothetical common ancestors. A cluster-analysis will only identify the "observed" mutations and "expected" and "observed" mutations can be majorly different due to the major artefact of reversion mutation. This is particularly true of ...
A great question, though a little ambiguous. I don't know what "general clustering algorithms" refer to. For biological sequences, building a tree can be thought as a way of clustering. Anyway...
There are different tree building algorithms. Max parsimony (MP), max likelihood (ML) and bayesian algorithms directly take sequences as input. They are distinct ...
This article uses the freely available R package dropbead for filtering and then Seurat to perform a principal component analysis that groups together affine transcriptomes. It could be what you are looking for.
There is a way to do this, and even better--there is documentation for how to do it! No surprise coming from the Satija Lab. In the vignette they perform multidimensional scaling, but the idea is the same. cmdscale() returns the cell embeddings. SetDimReduction() is the Seurat function you are looking for. No manual editing using @ required.
The authors use ...
Make the image itself bigger (e.g., png("S7.png", width=1000, height=1000)).
Having said that, rethink the utility of having the labels there. Are you really going to go through and look at them if you have hundreds of lines? Probably not.
rlog(normalized counts) is going to be more robust than log2(TPM), so use it instead.
Do it afterward, keeping the low counts can be helpful for library-size normalization.
This is very much more of an art than a science, though have a look at the kOverA() function from the genefilter package. That will give you a bit more fine-grained control on filtering.
The dendrogram summarize the information of a group of values and sort them according to the similarity they have. It can be applied to both, samples and features.
The dendrogram allows to visualize features that are more similar together, usually revealing patterns that wouldn't have been seen otherwise.
In an article usually it is used something like: "...
I do not think there is a simple "yes" or "no" answer here.
A good starting point would be, as you suggest, use all the genes and assess the results in the light of the marker genes and expected results. This could both serve as as good quality control as well as give you overview of all the processes happening in the cells.
Depending on the marker genes ...
While better methods of evaluating your clusters would be to use an external dataset or a dataset with known truth, there are a variety of internal validation metrics that can be used to compare clustering solutions without another dataset.
Here are a few metrics:
Root-Mean-Square Standard Deviation
Many more ...
Based on your description I think you should have a look at a technique called 'biclustering'.
The example on this page defines the goal of this technique as 'Finding subgroups of rows and columns which are as similar as possible to each other and as different as possible to the rest.'
Since your examples are python-based, you could check out scikit-learn'...
Your cluster labels come from graph clustering implemented in the FindClusters() function. The resulting clusters are then visualised with a 2D tSNE plot (via RunTSNE() and TSNEPlot()). So, your cluster 13 is not split into three sub-clusters but cells within cluster 13 look somewhat distant from each other on the tSNE plot. Having mentioned distances on a ...
Samples will only cluster by experimental group if the experimental effect is large enough that it's the primary source of variance between your samples. If that's not the case then you'll get results like you're seeing. Whether that's a problem or not ends up depending on how large of an effect you expect to see. If your groups are different stages in ...
You can calculate allele frequencies for each cluster you have to further verify if they belong to similar population, however if size of your dataset is rather small this may not work for you.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3813878/, here is an article about differentiating populations based on sequencing data.
I actually found an answer just accidentally. It is very unfortunate that factoextra documentation does not explicitly say that data parameter should be of data.frame type. In my case rtsne$Y was a matrix. The error that factoextra is giving is very uninformative.
Here is the code that ultimately worked:
-----> data = as.data.frame(...
I'm not sure Seurat is the best tool for this as it was developed for single cell RNA seq data and there are a few intricacies of that type of data that are very different from bulk RNA seq.
For bulk there are really good packages available and corresponding workflows, e.g. limma, edgeR and DESeq2.
The main problems with Seurat for bulk RNA-seq:
You can't download the tSNE coordinates for cells directly from the Analysis tab of the fancy, polished .html document that Cell Ranger produces. If you have access to the machine on which the pipeline was run, you can grab the intermediate files. Everything is well documented by 10X. They explain defaults etc. on that page. The tSNE coordinates ...
Some of the first MSA analysis, eg of codon usage of bacteria were performed using matrix factorization (see old papers from Des Higgins). His group also published discriminative methods for finding functional residues using supervised correspondence analysis
Wallace IM1, Higgins DG BMC Bioinformatics. 2007 Apr 23;8:135.
Supervised multivariate ...
For outlier identification I suggest using the sample network approach developed by Oldham et al. It basically amounts to constructing a inter-sample connectivity (assuming normalizedData contains the log-normalized data with genes in colunms and samples in rows)
k = colSums(cor(t(normalizedData))),
scaling the connectivities
Z.k = scale(k)
and then ...
The direct answer is yes. If you have never worked with hdf5 you can start here.
But, cellranger outputs several hdf5 files. You can extract the count matrix from the "/outs/filtered_gene_bc_matrices_h5.h5" file.
They provide a lot of information on how to use the hdf5 format so their their documentation is a good place to start.
In addition: you can ...
Typically some sort of variance stabilizing transform is used before clustering. Popular options are regularized log transformation or a vst transform, which are available in DESeq2. Note that these are NOT used for performing differential expression, just things like clustering and PCA. For differential expression one would use TMM, RLE or similar.
M <- SetIdent(M, value = "status")
or more explicitly
M <- SetIdent(M, value = M@meta.data$status)
You can also use the group.by argument of UMAPPlot() or other plotting functions from Seurat for that matter.
You can extract the necessary values and add them directly the plot as a second layer using plot + geom_text(). This is very similar to the inner workings of the DimPlot function with label = TRUE but allows you to use anything as label.
# I use dplyr v1.0.2 for all data frame manipulations
# First the cluster annotation and the tsne embeddings are merged
Download a release (Hammock_v_220.127.116.11z) from https://github.com/krejciadam/hammock/releases
To run the first example:
java -jar ../../dist/Hammock.jar full -i musi.fa
As mentioned in the comments, it's in the manual and any further issues need more information (errors etc).
I think one of the reasons you struggle is that clustering DNA sequences is not a clearly defined task. In general intention of clustering is to reconstruct, or approximate the relatidness of the DNA seqeunces.
If all these sequences are homologous you can do a multiple sequence alignment (using MAFFT or Clustal) and build a phylogenetic tree (using ...