Problem with samtools command with a final hyphen

Question

bwa mem ../01.data/GCF_000146045.2_R64_genomic.fna ../01.data/${sra}_1.fastq.gz ../01.data/${sra}_2.fastq.gz | \
    samtools view -S -b -t ../01.data/GCF_000146045.2_R64_genomic.fna.fai - | \
    samtools sort - >$sra.sorted.bam
samtools index $sra.sorted.bam

I am mapping the downloaded yeast sequences in sra format to the reference genome in a bash shell script. And I am using samtools to change to bam file. Then I try to sort the resulted bam file.

I could follow the steps but I could not understand the use of hyphen after the .fai (index file) and the use of this hyphen before the pipe. Moreover, what is the use of the hyphen after the samtools sort?

Answer 1

It allows you to chain commands together to avoid creating intermediate files and simplify your workflow:

Samtools is designed to work on a stream. It regards an input file `-' as the standard input (stdin) and an output file `-' as the standard output (stdout). Several commands can thus be combined with Unix pipes. Samtools always output warning and error messages to the standard error output (stderr).

This is a common idiom in the *nix world, and is actually part of the POSIX specification:

Guideline 13:

For utilities that use operands to represent files to be opened for either reading or writing, the '-' operand should be used to mean only standard input (or standard output when it is clear from context that an output file is being specified) or a file named -.

Answer 2

The hyphen "-" character is commonly used as a placeholder in Unix commands to represent the standard input (stdin) or standard output (stdout) streams. From the POSIX specification:

Guideline 13:

For utilities that use operands to represent files to be opened for either reading or writing, the '-' operand should be used to mean only standard input (or standard output when it is clear from context that an output file is being specified) or a file named -.

In your pipeline, the hyphen after the index file name indicates that the output of the bwa mem command should be piped to samtools view, which will then read it from the standard input stream. The hyphen after the samtools sort command, is not strictly necessary since an input SAM/BAM/CRAM is entirely optional. Note that recent versions of samtools let you coordinate sort the SAM and write a sorted BAM, so you can do away with the call to samtools view entirely. For example:

set -euo pipefail

bwa mem \
    ../01.data/GCF_000146045.2_R64_genomic.fna \
    "../01.data/${sra}_1.fastq.gz" \
    "../01.data/${sra}_2.fastq.gz" |
samtools sort \
    -o "${sra}.sorted.bam"
samtools index \
    "${sra}.sorted.bam"

If you wanted to, you could replace the hyphens with the /dev/stdin special file, which represents the standard input streams. Sometimes this can make more complex commands a bit more readable, but note that not all Unix systems support the /dev/stdin and /dev/stdout special files. Most modern Unix systems do however.

The above uses set -eo pipefail to ensure pipeline errors are not masked. This exits immediately if any command in the pipeline exits with a non-zero status.