Getting a "system is computationally singular" error in sleuth

Question

I am analysing 142 samples belonging to 6 batches. Additionally, those samples belong to 72 strains, which means that for most of the strains there are two samples.

I could fit simple models (for strain and batches for instance), but when I get to the "full" model (~batch+strain), I get the following error:

so <- sleuth_fit(so, ~strain+batch, 'full')
Error in solve.default(t(X) %*% X) :
  system is computationally singular: reciprocal condition number = 5.2412e-19

I should point out that of the 72 strains, only 15 have samples in distinct batches. This means that most strains (57) have both samples in the same batch.

Is the error due to an unknown bug or rather to the experimental design? Does it mean that the information on batches cannot be used?

Thanks

EDIT I've posted the experimental design in a gist

batch   strain  replica
batch_1     strain_41   1
batch_4     strain_41   2
batch_1     strain_28   1
batch_4     strain_28   2
batch_1     strain_26   1
[...]

Answer 1

You should be able to remove any one of the following strains to end up with a rank-sufficient model matrix: 5, 10, 12, 13, 14, 15, 19, 26, 28, 3, 30, 32, 36, 39, 41, 45, 46, 49, 5, 50, 52, 53, 58, 59, 60, 69, 8. As an aside you can figure this sort of thing out as follows (I read your dataframe in a the d object):

> m = model.matrix(~batch+strain, d)
> dim(m) # 142 row, 77 columns, so minimum rank is 77
> qr(m)$rank # 76, so just barely rank insufficient
> #see if we can remove a single column and still get rank 76
> colnames(m)[which(sapply(1:77, function(x) qr(m[,-x])$rank) == 76)]

You obviously don't want to remove the batch columns or the intercept. The normal tricks that you can sometimes use to get around this issue with case-control studies don't appear to help here, which is why I would just drop a strain and call it done. Keep in mind that your power is still likely terrible. I generally recommend at least 6 replicates per group (scale down the number of groups to fit your budget).

EDIT:

Once the desired strain is removed from the model, it can be fit directly into the sleuth_fit function to obtain the full model:

> m = m[, -9] # whatever column to drop to get the appropriate rank
> so <- sleuth_fit(so, m, 'full')

Answer 2

This happen when the variables (strain +batch) create a design matrix like this:

batch strain
1 1 #
1 1 #
1 2
2 2
3 3
4 3
...
16 72

Which means that some of the covariates are not linearly independent (ie batch 1 and strain 1), all the strain 1 is in batch 1.

You can correct for batch effects, but not in this design of the linear model (if you want to take into account the strain). You could do one batch more with those 15 strains that are in a single batch (if they are in different batch between them) that way you would get an independent design.

Many strains in a single batch are in the same batch. You need to increase the number of samples (recommended anyway due to the low number of samples per strain) to avoid this problem.

---

_{There are a lot of related question in Bioconductor support forum, from where I expanded an answer.}