Hot answers tagged

62 votes
Accepted

What is the difference between FASTA, FASTQ, and SAM file formats?

Let’s start with what they have in common: All three formats store sequence data, and sequence metadata. Furthermore, all three formats are text-based. However, beyond that all three formats are ...
Konrad Rudolph's user avatar
23 votes
Accepted

Convert a BAM file from one reference to another?

You're the second person I have ever seen using NCBI "chromosome names" (they're more like supercontig IDs). Normally I would point you to a resource providing mappings between chromosome names, but ...
Devon Ryan's user avatar
  • 19.6k
20 votes

What is the difference between FASTA, FASTQ, and SAM file formats?

In a nutshell, FASTA file format is a DNA sequence format for specifying or representing DNA sequences and was first described by Pearson ...
eastafri's user avatar
  • 329
19 votes
Accepted

Converting a BAM file into VCF

It's not really possible to convert bam to vcf. bam is a mapping file, it does not contain ...
Kamil S Jaron's user avatar
17 votes
Accepted

How can I downsample a BAM file while keeping both reads in pairs?

samtools has a subsampling option: -s FLOAT: Integer part is used to seed the random number generator [0]. Part after the decimal point sets the fraction of templates/pairs to subsample [no ...
rightskewed's user avatar
17 votes
Accepted

Why would someone use a CRAM instead of a BAM?

Whenever you want to save space (this can be a substantial savings). Until quite recently (samtools/htslib 1.7), only CRAM supported long CIGAR strings. If you need to guarantee that any random ...
Devon Ryan's user avatar
  • 19.6k
16 votes

What is the difference between samtools, bamtools, picard, sambamba and biobambam?

The obvious answer is that different people wrote them. It's fairly common in bioinformatics for people with a computer science background to get frustrated with existing tools and create their own ...
gringer's user avatar
  • 13.8k
16 votes
Accepted

Why does this human bam file only have one copy of each chromosome?

The maternal and paternal copies of a chromosome are called haplotypes. Many metazoans (animals) are diploid and have maternal and paternal chromosome contribution during sexual reproduction, not just ...
conchoecia's user avatar
  • 3,141
15 votes
Accepted

Merge hundreds of small BAM files into a single BAM file

samtools merge merged.bam *.bam is efficient enough since the input files are sorted. You can get a bit faster with sambamba and/or biobambam, but they're not ...
Devon Ryan's user avatar
  • 19.6k
15 votes
Accepted

Are soft-clipped bases used for variant calling in samtools + bcftools?

No, samtools (and therefore bcftools) does not use soft-clipped bases. You can quickly confirm this by using either samtools depth or ...
Devon Ryan's user avatar
  • 19.6k
13 votes
Accepted

Obtaining uniquely mapped reads from BWA mem alignment

Update - as of January 2021, samtools can now do filtering based on an expression that includes tag variables. In this case, this expression can be used to exclude any reads that have either an XA or ...
gringer's user avatar
  • 13.8k
12 votes

Convert bam file to highly compressible bam

You can just set the fields you don't need to *: ...
Devon Ryan's user avatar
  • 19.6k
12 votes
Accepted

Why do SAM and BAM use different coordinate systems?

Arithmetic on a zero-based coordinate system is less complicated than that on a one-based system, so it appears zero-based is often (not exclusively) used for binary data formats, like BAM or bigBed, ...
Alex Reynolds's user avatar
12 votes

Good / recommended way to archive fastq and bam files?

EDIT: I am rewriting the answer in response to updates to the original question. TL;DR: use CRAM Background 1: quality binning and FASTQ compression In the old days, base callers outputted base ...
user172818's user avatar
  • 6,485
11 votes
Accepted

BAM to BigWig without intermediary BedGraph

This can be done in R very easily from an indexed .bam file. Given single-end file for sample1. ...
story's user avatar
  • 1,573
11 votes

Random access on a FASTQ file

Arbitrary record access in constant time To get a random record in constant time, it is sufficient to get an arbitrary record in constant time. I have two solutions here: One with ...
winni2k's user avatar
  • 2,236
11 votes

I have 23andme text files and would like to convert to SAM/BAM format

Having done 23andme myself I can tell you that your variant file, which contains SNP genotypes, cannot be converted to a bam file. It does not contain the same information as a bam file. It may be ...
Wouter De Coster's user avatar
11 votes
Accepted

Index a BAM file using pysam

Oh you silly sausage, pysam.index takes a bam file name, not a python object. ...
Kamil S Jaron's user avatar
10 votes
Accepted

How to count reads in bam per bed interval with bedtools

The order of -a and -b switched at some point. You want: ...
Devon Ryan's user avatar
  • 19.6k
10 votes
Accepted

How to count the number of mapped read in 100-bp window from a BAM/SAM file

one-liner Here's a gritty one-liner to count the number of reads in a region if you have just one region that you want to investigate. Change the 1 in ...
conchoecia's user avatar
  • 3,141
9 votes
Accepted

Why most aligners do not output the "X" CIGAR operation?

The SAM format originally had only M, I, D, N, S, H, and P CIGAR operators. See the original SAM specification (if you can view Apple Pages documents) and Table 1 in The Sequence Alignment/Map format ...
John Marshall's user avatar
9 votes
Accepted

Quantifying reads mapping to multiple loci

You almost had the correct python code already, you just need to filter out secondary alignments: ...
Devon Ryan's user avatar
  • 19.6k
9 votes
Accepted

What are the de facto required fields in a SAM/BAM read group?

The sample tag (i.e. SM) was a mandatory tag in the initial SAM spec (see the .pages file; you need a mac to open it). When transitioned to Latex, this requirement ...
user172818's user avatar
  • 6,485
9 votes

How to subset a BAM by a list of QNAMEs?

use picard FilterSamReads http://broadinstitute.github.io/picard/command-line-overview.html#FilterSamReads READ_LIST_FILE (File) Read List File containing reads that will be included or excluded ...
Pierre's user avatar
  • 1,501
9 votes
Accepted

How to validate that BAMs have been downloaded correctly?

samtools quickcheck is all you need. From the manual: Quickly check that input files appear to be intact. Checks that beginning of the file contains a valid header ...
Devon Ryan's user avatar
  • 19.6k
9 votes
Accepted

Extracting all reads from bam file which match read IDs in another file

It is still slow but grep has a -f option to take in a file samtools view inbam.bam | grep -f read_names.txt > read_locs.txt
Bioathlete's user avatar
  • 2,574
8 votes

What is the difference between FASTA, FASTQ, and SAM file formats?

FASTA (officially) just stores the name of a sequence and the sequence, unofficially people also add comment fields after the name of the sequence. FASTQ was invented to store both sequence and ...
BaCh's user avatar
  • 754
8 votes

Merge hundreds of small BAM files into a single BAM file

Merging sorted files is a linear operation, so any well-implemented tools that do it will do it with approximately the same efficiency. So samtools merge (use the ...
John Marshall's user avatar
8 votes
Accepted

True bam file encoding and viewing as text

Why not work only with sam files and we need to convert them to bam? SAM is for human only, a binary format like BAM is smaller and parsed much faster by a program (and we want speed and optimize ...
Pierre's user avatar
  • 1,501
8 votes

ethnicity check either from bam or vcf files

The main difficulty here is the use of GRCh38. Unfortunately, despite the fact that it's more than four years old, the major ethnicity-labeled public datasets (1000 Genomes, gnomAD when allele ...
Christopher Chang's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible