What is the index fastq file (sample_I*.fastq.gz) generated when demultiplexing Illumina paired-end runs?

Question

What is the index fastq file that comes with some Illumina sequencing datasets? (The samplename_I*.fastq.gz file.)

For example, I recently received some 10X Chromium reads for two libraries sequenced on the same lane. This was a 2x150 sequencing run, so there should be two fastq files. The sequencing center demultiplexed the libraries and generated two separate directories - one for each library.

• In each directory there are three fastq files:
  • Mysample_I1_001.fastq.gz
  • Mysample_R1_001.fastq.gz
  • Mysample_R2_001.fastq.gz

I know that the R1 fastq file contains the forward reads and the R2 fastq file contains the reverse reads, but what is the index file? The first few sequences do not match the 10X indices for this library (ACATTACT, CAGCCCAC, GGCAATGG, TTTGGGTA).

>bioawk -cfastx '{print($seq)}' Mysample_I1_001.fastq.gz | head
NTTGGGTA
NGCAATGG
NAGCCCAC
NAGCCCAC
NCATTACT
NCATTACT
NGCAATGG
NAGCCCAC
NTTGGGTA
NAGCCCAC

Answer 1

tldr - The I*.fastq.gz file contains the read index sequences.

long explanation

Illumina uses a program called bcl2fastq to demultiplex sequencing runs.

This software takes a list of samples and their associated indices and uses those sequences to make one or more fastq files per sample, binned by one or two index sequences on either end of the sequencing molecule (i5 and i7 indices, see page 6 for HiSeq).

Illumina sequencing isn't perfect though, and there are sometimes errors in reading the index sequence. For example this index, CAGCCCAC can easily have read errors in the A sandwiched between many Cs: CAGCCCAC -> CAGCCCCC.

Instead of throwing out all of the reads with indexing sequencing errors, the bcl2fastq program includes reads that are discernibly derived from a true index for that sample as long as there is no overlap with another sample.

In the example provided above, the four indices for mysample were: ACATTACT, CAGCCCAC, GGCAATGG, TTTGGGTA. If we look at all of the index sequences in the Mysample_I1_001.fastq.gz file, we will see that sequences with the four correct indices are indeed the most prevalent, however there are also reads that have indices with sequencing errors derived from the correct indices.

bioawk -cfastx '{print($seq)}' Mysample_I1_001.fastq.gz | \
    sort | uniq -c | sort -k1 -nr | head

count      index       source
41362311   CAGCCCAC    True index
37209190   GGCAATGG    True index
36863213   ACATTACT    True index 
33674467   TTTGGGTA    True index
 1140358   NAGCCCAC    CAGCCCAC
 1026099   NGCAATGG    GGCAATGG
 1016754   NCATTACT    ACATTACT
  933342   NTTGGGTA    TTTGGGTA
  119626   TTTGGGGA    TTTGGGTA
   98657   GTTGGGTA    TTTGGGTA
   96625   GGCAATGA    GGCAATGG

Answer 2

It took me a while to figure out that the "index" is the same thing as the "barcode" that says which sample each sequence is from on a multiplexed run.

If your data is not demultiplexed (single R1.fastq and R2.fastq files contain the information for multiple samples), then this I1.fastq file is what you use to assign each sequence to a sample (ie to "demultiplex"). The spreadsheet listing which barcode/index is which sample is often called a "mapping file."

If your data is already demultiplexed (you have separate R1.fastq and R2.fastq files for each sample), then you don't really need it... however some people use the index sequences in their quality control workflows. For example, this 2016 paper in BMC Genomics by Wright and Vetsigien "Quality filtering of Illumina index reads mitigates sample cross-talk": https://doi.org/10.1186/s12864-016-3217-x

Answer 3

I looked at 10x data a couple of years ago, and at that time, their protocol would use four separate indexes, with balanced index sequences, as a way of introducing diversity into the library. These four indexes were separate from the individual oligos used to mark single cells. For an individual sample, you would need to combine the reads from all four separate indexes, which is how it looks like in your data. There is a brief explanation here: https://support.10xgenomics.com/single-cell-gene-expression/index/doc/specifications-sample-index-sets-for-single-cell-3