3
$\begingroup$

The problem

If I have a sample sheet that contains both single-indexed and dual-indexed samples, I can split it up into two sample sheets and then run bcl2fastq on each one. However, when doing this, large Undetermined fastq files are generated. E.g., when processing the single-indexed samples, all the dual-indexed samples go to Undetermined. And when processing the dual-indexed samples, the single-indexed samples go to Undetermined.

Additionally, because they are being processed separately, if any single index A is part of a dual index A+B, then when processing the single indexes, an A+B may be mistaken for an A, so it would seem that they need to processed simultaneously to avoid this mis-assignment.

The question

Given a sample sheet and directories of BCL files, how can such a set of sequencing data be demultiplexed correctly, either using bcl2fastq or the Picard tools?

To put it another way, I want to demultiplex a single sequencing run that contains both single-indexed and dual-indexed samples. It can be assumed that the indexes are sufficiently distinct such that any sample's index configuration is different from any other. But assuming that the different index configurations are not segregated to particular lanes of the sequencer, the question is how to demultiplex the files correctly such that both the single-indexed and dual-indexed samples are recognized.

Attempts at a solution

Using bcl2fastq directly

If a sample sheet of a mix of indexes is given to bcl2fastq (v2.17) directly, it produces the error

ERROR: bcl2fastq::common::Exception: Success (0): .../bcl2fastq2/src/cxx/lib/layout/BarcodeCollisionDetector.cpp(127): 
Throw in function void bcl2fastq::layout::BarcodeCollisionDetector::validateNewBarcodeSizesAgainstExisting(const std::vector<long unsigned int>&) const
Dynamic exception type: boost::exception_detail::clone_impl<bcl2fastq::layout::BarcodeCollisionError>
std::exception::what: Barcodes have an unequal number of components.

Barcodes have an unequal number of components.

Using picard

It seems that it should be possible using the Picard tools, but I have not found a way to set up the inputs to ExtractIlluminaBarcodes and IlluminaBasecallsToFastq that processes this configuration correctly.

The syntax for supporting multiple index configurations is not entirely clear. But when using various combinations of N and '*' on multiplex_params.tsv and barcodes.txt required by the Picard tools, a large Undetermined file is still produced and actual sample fastq files are tiny, indicating it is not processing them correctly.

Using multiple calls to bcl2fastq

As indicated in the discussion above, this suffers from the problem of having all the dual indexed samples going to the "Undetermined" file when processing the single-indexed samples, or vice versa, and creates two output directories of reports and stats, which must be merged.

Padding empty indexes with N

By padding, I mean converting ACGTACGT to ACGTACGT+NNNNNNNN so that the single-index samples in the sample sheet are "dual" as well. This strategy seems that it would work except there is a bug in bcl2fastq that it treats "N" literally instead of as a wildcard. See the release notes for details.

Improve this question
$\endgroup$
3
  • $\begingroup$ Welcome to the site. Sorry but I don't understand which files do you have. Could you clarify what are your files? Or do you have a mix of single-indexed and dual-indexed and you can't differentiate between them? $\endgroup$
    – llrs
    Jul 30, 2019 at 15:27
  • $\begingroup$ @llrs Hi, thanks for the question. In this case, I'm trying to demultiplex a single sequencing run that contains both single-indexed and dual-indexed samples. You can assume that the indexes are sufficiently distinct such that any sample's index configuration is different from any other. But assuming that the different index configurations are not segregated to particular lanes of the sequencer, the question is how to demultiplex the files correctly such that both the single-indexed and dual-indexed samples are recognized. $\endgroup$
    – AmadeusDrZaius
    Jul 30, 2019 at 15:33
  • 2
    $\begingroup$ Ok, now I understood the question, but I'm sorry I can't help with it. Perhaps you could edit your last comment in the question for other users. $\endgroup$
    – llrs
    Jul 30, 2019 at 15:34

2 Answers 2

Reset to default
1
$\begingroup$

I'm late to the party but simply run cellranger mkfastq twice, with different arguments for the mask and with --filter-dual-index for the double-indexed samples. In a second step you then have to disentangle the dual A+B reads from single-index A reads based on the readID's that are found in both of the demultiplexings. This is cumbersome and time+diskspace consuming, but it can be done. I would use some Unix tools for that but there are probably smarter ways.

EDIT: I mistakenly said bcl2fastq, this must be cellranger mkfastq. (thanks AmadeusDrZaius)

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your suggestion. I only see that "--filter-dual-index" option available for cellranger mkfastq, rather than bcl2fastq itself. Is that where you saw the option? $\endgroup$
    – AmadeusDrZaius
    May 25, 2021 at 18:46
  • 1
    $\begingroup$ ah yes, that is true, sorry. I guess you would have to do an additional filtering step on the FASTQ file, the second word on the ID lines is 'index1+index2' for properly demultiplexed reads. $\endgroup$
    – plijnzaad
    May 26, 2021 at 19:54
0
$\begingroup$

Have you just tried giving bcl2fastq one sample sheet with a mix of single and dual indices? I don't think what you are trying to do is a problem.

if any single index A is part of a dual index A+B,

Well, it might be a problem if you did that. That was poor planning.

Improve this answer
$\endgroup$
6
  • $\begingroup$ That was the first thing tried but unfortunately bcl2fastq will give the error "Barcodes have an unequal number of components" when you try to do that (using v2.17). I'll add a note to my question. $\endgroup$
    – AmadeusDrZaius
    Jul 30, 2019 at 18:01
  • $\begingroup$ Are you sure that the single index samples don't have a site for the second index to sit down and read? My lab does some samples with single indicies and some with double, but they always prep with libraries with kits meant for dual indexing, so the second index is always there, even if I ignore it. $\endgroup$
    – swbarnes2
    Jul 30, 2019 at 19:19
  • $\begingroup$ That's a good idea that might be a good workaround in some cases, but I think the general answer is that they are not always physically dual-indexed. The use of "N" as described in the bcl2fastq documentation would work if not for the fact that a bug in bcl2fastq causes it to treat "N" as a literal base "N" instead of as a wildcard. Given that, I'm guessing that the Picard tools are more likely to have a full solution for this particular situation. $\endgroup$
    – AmadeusDrZaius
    Jul 30, 2019 at 19:33
  • $\begingroup$ I don't think it's going to be trivial for software to figure out the difference between "second index doesn't match what I expect" and "second index isn't really here". You might have to have bcl2fastq export the indices as fastqs, so you can use the quality score to parse yourself. $\endgroup$
    – swbarnes2
    Jul 30, 2019 at 19:51
  • $\begingroup$ That's an interesting point. I expected that it would just make a "best guess", but I can see why it wouldn't. It sounds like that would primarily only be an issue in the case of "A" and "A+B" that I described above. If the case of "A" and "A+B" is avoided, it should be able to tell the difference. Thanks for your thoughts. $\endgroup$
    – AmadeusDrZaius
    Jul 30, 2019 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.