How to convert fastq to fast5

Question

fast5 is a variant of HDF5 the native format in which raw data from Oxford Nanopore MinION are provided. You can easily extract the reads in fast5 format into a standard fastq format, using for example poretools.

Say I have aligned these reads in fastq format to an external reference genome, resulting in a SAM file. Say I have then taken a subset of the SAM file, according to the bitwise flag, to include only the reads that map to the reference. With the read ID, I can then grep them out from the file containing the reads in fastq format, generating a subset file in fastq format containing only the IDs that have mapped to the reference.

Now my question is, can we subset reads from the fast5 archive according to the list of mapping reads as taken from the file with reads in fastq format? This is for educational purposes, so that we have a smaller starting archive, and the fast5 -> fastq extraction takes less cpu time.

Answer 1

NOTICE: I have altered my answer slightly from the original as I have turned the original script into a pip installable program (with tests) and have updated the links and code snippets accordingly. The essence of the answer is still exactly the same.

---

This is something I have been meaning to get around to for a while, so thanks for the prompt.

I have created a python program called fast5seek to do what (I think) you're after.

As you have mentioned this is for educational purposes I have added a tonne of comments to the code too so I think you shouldn't have any issues following it.

The docs on the GitHub repo have all the info, but for those reading along at home

pip3 install fast5seek
fast5seek -i /path/to/fast5s -r in.fastq in.bam in.sam -o out.txt

What it does is read in <in.fastq|in.bam|in.sam> and extract the read id from each header. It then goes through all the fast5 files under /path/to/fast5s and checks whether their read id is in the set of read ids from <in.fastq|in.bam|in.sam>. If it is, the path to the file is written to it's own line in out.txt.

If no output (-o) is given, it will write the output to stdout.

So if you wanted to pipe these paths into another program, you could do something like

mkdir subset_dir/
fast5seek -i /path/to/fast5s/ -r in.fastq | xargs cp -t subset_dir/

The above example would copy the fast5 files that are found in your fastq/BAM/SAM to subset_dir/.

Answer 2

I do this very frequently, using the read file name to identify FAST5 files associated with particular reads. If a FASTQ record includes the channel number and read number (and preferably the runID as well), then I use that information to find the associated FAST5 read.

If reads are called with Albacore, then the sequencing_summary.txt file has additional information. The first column is the exact name of the file that was called, the second column is the read ID of the FASTQ sequence, and the third column is the run ID associated with that read ID. This is much easier to work with, but does require calling reads with Albacore (which, admittedly, is what seems to be producing the best results at the moment).