Error while calling bcftools mpileup - Failed to open -: unknown file type

Question

I have sequenced a bacterial genome with a GridIon from ONT. Basically what I want to check is whether or not trimming 50 bps at the beginning of the reads will improve alignment against the reference genome and ultimately the call of a consensus sequence.

For that what I have done so far is:

1. concatenated all the fastqs generated from the Gridion into a single (very large) fastq file: ref494_cat_all.fastq
2. aligned this concatenated fastq against the reference genome ref_genbank.fasta with minimap2: ~/Downloads/minimap2-2.17_x64-linux/minimap2 -ax map-ont ref_genbank/ref_genbank.fasta ref494_cat_all.fastq > ref494_aligned_trim50.sam
3. Performed the general samtools steps to covert sam to bam, index and sort

Then, in order to get the consensus sequence I first try to generate a .vcf.gz file, according to this tutorial

bcftools mpileup -f ref_genbank/ref_genbank.fasta ref494_aligned.sorted.bam | bcftools call -mv -Oz -o calls.vcf.gz

Immediatelly after calling the command above I got:

Note: none of --samples-file, --ploidy or --ploidy-file given, assuming all sites are diploid
[mpileup] 1 samples in 1 input files

Then, after a minute or two, I got:

Failed to open -: unknown file type

I'm pretty sure that there's no issue with .bam file, as I could visualise the alignment with IGV or tablet.

Does anyone know what the issue can be?

Answer 1

The - is usually used to mean standard input when reading data. So the message would suggest that the second bcftools command fails to read from stdin, so it fails to read the output of the first bcftools command:

$ echo "foo bar" | /ref/software/bcftools-1.8/bcftools call -mv -Oz -o calls.vcf.gz
Note: none of --samples-file, --ploidy or --ploidy-file given, assuming all sites are diploid
Failed to open -: unknown file type

So there is something wrong with either your input file or with the output produced when running the first bcftools command on that input file. We can't really help more unless you share the file.

Answer 2

It's possible you're looking at instructions for a recent version of samtools / bcftools, whereas your computer has an older version. Can you try samtools mpileup -g -f ... to output the pileup as BCF format instead of bcftools mpileup -f ...?

Update: it's good to know that the samtools / bcftools pipelines are at least producing consistent results, which suggests that the error is not associated with the output format on the mpileup side of the pipeline, and also good to see that you've identified that this was a memory issue.

The most frequent cause of memory issues I've had with sorted BAM files has been trying to figure out variants in extremely high coverage data (e.g. over 1000X), which would make sense for a bacterial isolate sequenced on a GridION. I use my own script to carry out a reservoir sampling on mapped BAM format files, which reduces coverage down to a more manageable level. Here's an example command to normalise to 100X coverage where possible:

samtools view reads.bam | ./samNormalise.pl -coverage 100 | \
   samtools view -b > reads_normalised.bam