Reading in external single cell data

Question

I'm trying to read in an external single cell dataset from https://www.nature.com/articles/s41467-020-16164-1, but I am having trouble reading in the count matrix.

counts found here: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE154778

This is how I've been reading in the raw counts.

cts = read.table("GSE154778_RAW.tar", sep=',', header=T, stringsAsFactors=F, check.names = FALSE)

The output is this:

> head(test)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          GSM4679532_K16733_barcodes.tsv.gz
1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     xVZ\xbe\xc6pܧj\x95I\xed\030\xee\xfe\xf8ޔ\x93\xa7\xbc\xf2\023\xed{\x8e\x98)w\\\xef:ܣN\xdfrr\xc7\xf5\f\xaa\xcb\001\xfa[\002\021@|a\x91\024\xab\xc5^\017\xc0O\xbd`7\x97&m\xb9\x81\xe8R\x97{\x9f\023@\x9a\xf2\x81I\0368\x80N\027\xfa\xf8\x89fǾ6V\x98h\x9aF\023\x80\xef\x8f\xf2;f\xb1%@\xffif\004(4\xf4\037\020\xaf\x8d\032\037Y\021 1\xdd\xd9V\xc0\xde{\xf0U\017<?\xd6\xe0D\x9b\x96\x8e\xc1\xd4s\xdb\024 
2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               *\x9b\xee5\xbb\xb3g\xab%\xa5r\xd7?\x82\x89V\xe5\177\xd5L4g\xe7?\xa2\x96\x8b\xaa\xf0\0215D\xd4\xe1\x87~D\xd9ou\xda\xe7?~\xab
3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         e\xb1\xe9ky\xb0w0Qy\xd7\xdd\xc6uaZ\xcb8\025-N\x96\xb6Y\xac\0015h\xeb\xe5\nL\xeb\xc4\177\x9cX\xbdU\016\xde\xcc\xdeAE\xa7\\w\xfcE_\001\xbb\026\xba\xe7\xf7ܳS'\x96\u038bI\xfe\x9a\xef\xfdm\xa2\xbf_瘺a\030\037\xec\xa3\032\x97\xa2\xea\x9e\xf3\xf40\x8d\xfb#\n\xa9\xb8vE\x94\x9b:\xef\xcb\xe5\xd5\xc69n\xe1\xbd\xcab\025\xc7\xfb|\fT\xb5\xfa\xea4lK\xb2\037)\xdfL\xfc\x8f\xf5yz\xa6D}\001/\xaak?̑\xe9\003\xf1\031\x94\xed~\xfdӳ\xc7B]\xca/\\\xfb\xa3%n
4 \xef\xaa\xc4B\xe0]\xe5\xac6\x9e\xea\xbbe\xf9\x8a\xecch/nV5\xd1p_O\xb6N\034\xa9\xf3x\034\x98&\025\xd75\xa7b\021\xb3v\xd6P\xf7D4~u\xb3\xab(\xf9\x9d\xb29^,\xd8\xf3v`B)\xc3c|>'\xc36Z\016\xb4\0257x8\u07b5fb\xd1(\xb9\xbe\xe7WT.x\xaa5\xe4\xc1\xd6\xe6ny\f֟\fYST.\\\xd0\xf2h\xf6\x8d\xc5Q\x90o\xbc\xadG\004\xb1\x9el\xcb\xd8\037\xef\xd4\024Y\xf1Q.\xe3\x81FGZ\016\026t\xa4\xcaY\xf3\xb7\x95\017,\au\xe8\037\xcf\a\xb3\ab\001\x8d.\aճ\x8f\xe6\xc4h\xe5n8\xf7Sdg\xc5Z\xef\xaa\xcb!\xd6\xd0ӳ\x92\xa2\xc4Wݥ?\xcc\xc4\016\031\023\xaa\xe7\xb6-H^X\005\xb1ӡ\x9b\xe6\xfb\xf0\x9b\x9d\027q\xcbq\x9aEm%\036\xb3\xe9$f\xc8\xd6\005\xdb\xee\x8b\xec\xf5\xb9\xef'\xaa~\xad\xb3\xf9\x8fg\xb3\025\xe5h\xc9\xe9_\xee7v\x87\xa3\xf5\177,nj\x9c\016Z\f\xe6\xf353\027\0367=\xc0\002{\x89\037,\017\xd3Œ\030\034D/\xeeb\x8ct\033\xe5UD\xedϧ\xb9\xf1\xbe\xe2\034L\xa7o\xea\xff\xb9\xecŭ\x9b\036Õ\xdbM+>Su$\\\xe0\xda2\xdbl?\xa2\xcc\v\xf4rS\xe4\027\x8aEdR9f\xb9N\022\xb4И@,e\xfe<\x95gN\xb4Ć\x9b\xc7\xeb\032\xc7P~\xa2\034\xd8]\x98\032\te\xff\xd8!vr\xb0\xa6\x92K\xf0\xfd`*\xb3\xe8\aq\x90\x9e'\xe6FI\x84\xa3<B\xeddE\xefpg\001V[\xbc\xb7u\xb04\003\xe3K\xb6h\xc0ild`\x92\025W\xe5#J\177\xa6\x95\xabZ.\x96\xec\016w\x93d÷+]Di\xbaJ\xa2\003\xf5\021m\xa4\x80J\022\f\027QqE\021r\x94\xf0\xbd\037\xd6ye\x970Fm\xa1ˎO\035\xfe\xe5\xb8\xf2/\x89\x9bX$\xa9b\026\037B~\xe2\023k\033\xbaB\xbcɪ\xa7\xe4f\xf2\027\x8f\xa2t\xa1\xee\xbd\xec\x80\xe5;\037\xbc\xfeC\x94\xed\x91UQ\x81~\xbe^N<\003&\xceDJE\xab\031\xa7\xf9\x9a\xa2\xf2%\024}\x9e.g\x9a0J&lj\xd2\032\xc55~\022\xa3ޭ\xd3|\xd1,k#\017ֶ\xfbg\026CL\016\x9bx\023Y\x89\xffI\x85\x8c\x89>U=\024|í\xb4\xf2\006\xfa#ʎ\032v\xc6Hps\034\fI\xef`E\xb1\xec\xcb\xf3\xc6\xf2E\xe5ӹ\xee1σc\x8b\x89\xe5\xf2\xd0\xe3\xf2\xeb\xac\xe5\xfb,&N\002\x9b\xe14i\xc5\024y\xc8^Ď\xe73\n\006\022\x94TM\xc5p\xbf(\xb8w~ES\x8d&N\xafr\xefL\f\xbf\x816~$\x99\021A\xfe7m\tƘhX\xe0\xa2\xc2>&\xe5s\xa0\x99\t\004\x88\u07bd\xa34\xf6q\xe4\xf1ٕ\x9b\xea\003\xd7frq(~\xe7\024\035\xcf
5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 \x91\xb7U
6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               \x9f-\xcf\025\xa7\xeb6<\x89\xf55\xc6+\xb9K\xb5\xbaa-[.\xda \xca]<M\xea\xbd%

Any suggestions?

Update:

I unzipped the file in HPC but now I'm having trouble reading in the counts as it's all the counts per sample.

Data_Processing.nb.html            GSM4679535_T3_genes.tsv.gz     GSM4679540_T9_barcodes.tsv.gz      GSM4679544_Y00014_matrix.mtx.gz
Data_Processing.Rmd                GSM4679535_T3_matrix.mtx.gz    GSM4679540_T9_genes.tsv.gz         GSM4679545_Y00016_barcodes.tsv.gz
GSE154778_dgeMtx.csv.gz            GSM4679536_T4_barcodes.tsv.gz  GSM4679540_T9_matrix.mtx.gz        GSM4679545_Y00016_genes.tsv.gz
GSE154778_RAW.tar                  GSM4679536_T4_genes.tsv.gz     GSM4679541_T10_barcodes.tsv.gz     GSM4679545_Y00016_matrix.mtx.gz
GSM4679532_K16733_barcodes.tsv.gz  GSM4679536_T4_matrix.mtx.gz    GSM4679541_T10_genes.tsv.gz        GSM4679546_Y00019_barcodes.tsv.gz
GSM4679532_K16733_features.tsv.gz  GSM4679537_T5_barcodes.tsv.gz  GSM4679541_T10_matrix.mtx.gz       GSM4679546_Y00019_genes.tsv.gz
GSM4679532_K16733_matrix.mtx.gz    GSM4679537_T5_genes.tsv.gz     GSM4679542_Y00008_barcodes.tsv.gz  GSM4679546_Y00019_matrix.mtx.gz
GSM4679533_Y00006_barcodes.tsv.gz  GSM4679537_T5_matrix.mtx.gz    GSM4679542_Y00008_genes.tsv.gz     GSM4679547_Y00027_barcodes.tsv.gz
GSM4679533_Y00006_genes.tsv.gz     GSM4679538_T6_barcodes.tsv.gz  GSM4679542_Y00008_matrix.mtx.gz    GSM4679547_Y00027_genes.tsv.gz
GSM4679533_Y00006_matrix.mtx.gz    GSM4679538_T6_genes.tsv.gz     GSM4679543_Y00013_barcodes.tsv.gz  GSM4679547_Y00027_matrix.mtx.gz
GSM4679534_T2_barcodes.tsv.gz      GSM4679538_T6_matrix.mtx.gz    GSM4679543_Y00013_genes.tsv.gz     Lin_et_al_GMedicine_metadata.csv
GSM4679534_T2_genes.tsv.gz         GSM4679539_T8_barcodes.tsv.gz  GSM4679543_Y00013_matrix.mtx.gz
GSM4679534_T2_matrix.mtx.gz        GSM4679539_T8_genes.tsv.gz     GSM4679544_Y00014_barcodes.tsv.gz
GSM4679535_T3_barcodes.tsv.gz      GSM4679539_T8_matrix.mtx.gz    GSM4679544_Y00014_genes.tsv.gz

I've created this loop to read through all the samples.

counts_list <- list()
for (sample in samples){
    print(sample)
    filename_header <- sample
    barcode_file <- file.path(wd3, paste(filename_header, "_barcodes.tsv", sep=""))
    gene_file <- file.path(wd3, paste(filename_header, "_genes.tsv", sep=""))
    matrix_file <- file.path(wd3, paste(filename_header, "_matrix.mtx", sep=""))
    counts <- Matrix::readMM(matrix_file)
    barcodes <- read.csv(barcode_file, sep="\t", header=F)
    barcodes <- barcodes$V1
    colnames(counts) <- barcodes
    genes <- read.csv(gene_file, sep="\t", header=F)
    genes <- genes[,-3]
    colnames(genes) <- c("ENSG", "Name")
    rownames(counts) <- genes$ENSG
    genes <- genes[!duplicated(genes$Name),]
    counts <- counts[genes$ENSG,]
    rownames(counts) <- genes$Name
    counts_list[[sample]] <- counts
}

Receive this error when I run:

counts <- do.call(cbind, counts_list)
Error: Matrices must have same number of rows in cbind2(argl[[i]], r)

I added:

genes <- genes[,-3]

to ensure that all the gene file has only 2 columns as I noticed that one sample had 3 variables but I'm still erroring.

Answer 1

Tar is an archive format so as you figured out by now you have to unpack it.

The files you now see are as follows:

• mtx.gz are compressed count matrices which you load into R using the Matrix::readMM() function
• genes.tsv.gz are the associated rowwise data so the "genes" which you can load with read.delim and then use rownames() to assign them to the matrix generated by readMM()
• barcodes.tsv.gz are the associated column data so the samples or "cells", use colnames() to assign them to the matrix

I would use list.files to create a vector of files and then either a loop or lapply to iterate over them for the loading. Does that make sense to you?

Edit after OP received this dimensions error: This then means that these files come from different experiments and have different upstream processing. THere is nothing you can do about it but figuring out which files come from which toplevel experiment and then only combine those of one experiment, respectively. The procedure towards reading data is as I lined out above.

Answer 2

It looks like it is 10X data and I strongly suggest a dedicated single cell analysis package such as Seurat. You will most likely end up using one of these anyway for data analysis.

Read10X() can be a good start. I don't remember whether it requires dedicated folders per sample though. Even if this is the case, you can create individual sample folders with a simple bash script, can be done within R as well.