7
$\begingroup$

I'm trying to use the t-SNE algorithm on some microarrays data. More specifically my data frame has 18600 columns with genes (features) and 72 rows with conditions with replicates ( 10xWt , 10xTg , etc ). The expression values are in log2 scale.

Here is the code that I'm trying to run.

# t-SNE implementation
library(Rtsne)

set.seed(1)

for(i in 1:15){

  tsne = Rtsne(data.T[,-18601], dims = 2, perplexity=i, verbose=TRUE, max_iter = 1000, pca=T)

  colors = rainbow(length(unique(data.T$classes)))
  names(colors) = unique(data.T$classes)
  plot(tsne$Y, t='n', main="tsne")
  text(tsne$Y, labels=data.T$classes, col=colors[data.T$classes])

  readline(prompt="Press [enter] to continue")
}

Please note that I'm not counting the column 18601 because this colums contains the labels/classes for each condition.

The think here is that when I execute this script, R returns me this error:

Error: protect(): protection stack overflow

Should I change the --max-pp-size or it's a bug in Rtsne package?

Also I was wondering if it is more meaningful to run the tSNE algorithm using not the log2 values of the expression level but the log fold change values in respect to the Wt (wild type) condition. I'm asking because I couldn't find a such other implementation of the tSNE on microarray data.

For the configuration of the Rtsne function I read this article

Any other suggestion on the implementation is welcomed.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
13
$\begingroup$

Converting your data.frame to a matrix (and then removing the data.frame) will often free up enough memory that you won't run into this. Note that a matrix is more memory efficient than a data.frame and you're requiring Rtsne() to hold both in memory at the same time (many math-centric functions will end up converting things to a matrix at some point for efficiency).

For what it's worth, it's never been entirely clear to me what the interaction is between a data.frame and the pointer protection stack, but it's often the case that this solves this sort of error.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ data.frames are lists of column vectors, and every one of these columns has a pointer that may need protecting. With a huge number of columns, I can easily see how this might blow the protect stack. Dense matrices are contiguous memory, they only need a single pointer. $\endgroup$
    – Konrad Rudolph
    Sep 12, 2017 at 15:02
  • $\begingroup$ I'd considered that, but my reading never clarified if it's really due to pointers for the columns or something else. It's unfortunate how little is mentioned about that in the documentation. $\endgroup$
    – Devon Ryan
    Sep 12, 2017 at 16:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.