62
votes
Accepted
What is the difference between FASTA, FASTQ, and SAM file formats?
Let’s start with what they have in common: All three formats store
sequence data, and
sequence metadata.
Furthermore, all three formats are text-based.
However, beyond that all three formats are ...
- 4,845
23
votes
Accepted
Convert a BAM file from one reference to another?
You're the second person I have ever seen using NCBI "chromosome names" (they're more like supercontig IDs). Normally I would point you to a resource providing mappings between chromosome names, but ...
- 19.6k
20
votes
What is the difference between FASTA, FASTQ, and SAM file formats?
In a nutshell,
FASTA file format is a DNA sequence format for specifying or representing DNA sequences and was first described by Pearson ...
- 329
19
votes
Accepted
Converting a BAM file into VCF
It's not really possible to convert bam to vcf. bam is a mapping file, it does not contain ...
- 5,517
17
votes
Accepted
How can I downsample a BAM file while keeping both reads in pairs?
samtools has a subsampling option:
-s FLOAT:
Integer part is used to seed the random number generator [0]. Part after the decimal point sets the fraction of templates/pairs to subsample [no ...
- 991
17
votes
Accepted
Why would someone use a CRAM instead of a BAM?
Whenever you want to save space (this can be a substantial savings). Until quite recently (samtools/htslib 1.7), only CRAM supported long CIGAR strings.
If you need to guarantee that any random ...
- 19.6k
16
votes
What is the difference between samtools, bamtools, picard, sambamba and biobambam?
The obvious answer is that different people wrote them. It's fairly common in bioinformatics for people with a computer science background to get frustrated with existing tools and create their own ...
- 13.8k
16
votes
Accepted
Why does this human bam file only have one copy of each chromosome?
The maternal and paternal copies of a chromosome are called haplotypes. Many metazoans (animals) are diploid and have maternal and paternal chromosome contribution during sexual reproduction, not just ...
- 3,141
15
votes
Accepted
Merge hundreds of small BAM files into a single BAM file
samtools merge merged.bam *.bam is efficient enough since the input files are sorted. You can get a bit faster with sambamba and/or biobambam, but they're not ...
- 19.6k
15
votes
Accepted
Are soft-clipped bases used for variant calling in samtools + bcftools?
No, samtools (and therefore bcftools) does not use soft-clipped bases. You can quickly confirm this by using either samtools depth or ...
- 19.6k
13
votes
Accepted
Obtaining uniquely mapped reads from BWA mem alignment
Update - as of January 2021, samtools can now do filtering based on an expression that includes tag variables. In this case, this expression can be used to exclude any reads that have either an XA or ...
- 13.8k
12
votes
Convert bam file to highly compressible bam
You can just set the fields you don't need to *:
...
- 19.6k
12
votes
Accepted
Why do SAM and BAM use different coordinate systems?
Arithmetic on a zero-based coordinate system is less complicated than that on a one-based system, so it appears zero-based is often (not exclusively) used for binary data formats, like BAM or bigBed, ...
- 3,125
12
votes
Good / recommended way to archive fastq and bam files?
EDIT: I am rewriting the answer in response to updates to the original question.
TL;DR: use CRAM
Background 1: quality binning and FASTQ compression
In the old days, base callers outputted base ...
- 6,485
11
votes
Accepted
BAM to BigWig without intermediary BedGraph
This can be done in R very easily from an indexed .bam file.
Given single-end file for sample1.
...
- 1,573
11
votes
Random access on a FASTQ file
Arbitrary record access in constant time
To get a random record in constant time, it is sufficient to get an arbitrary record in constant time.
I have two solutions here: One with ...
- 2,236
11
votes
I have 23andme text files and would like to convert to SAM/BAM format
Having done 23andme myself I can tell you that your variant file, which contains SNP genotypes, cannot be converted to a bam file. It does not contain the same information as a bam file. It may be ...
- 1,314
11
votes
Accepted
Index a BAM file using pysam
Oh you silly sausage, pysam.index takes a bam file name, not a python object.
...
- 5,517
10
votes
Accepted
How to count reads in bam per bed interval with bedtools
The order of -a and -b switched at some point. You want:
...
- 19.6k
10
votes
Accepted
How to count the number of mapped read in 100-bp window from a BAM/SAM file
one-liner
Here's a gritty one-liner to count the number of reads in a region if you have just one region that you want to investigate. Change the 1 in ...
- 3,141
9
votes
Accepted
Why most aligners do not output the "X" CIGAR operation?
The SAM format originally had only M, I, D, N, S, H, and P CIGAR operators. See the original SAM specification (if you can view Apple Pages documents) and Table 1 in The Sequence Alignment/Map format ...
- 956
9
votes
Accepted
Quantifying reads mapping to multiple loci
You almost had the correct python code already, you just need to filter out secondary alignments:
...
- 19.6k
9
votes
Accepted
What are the de facto required fields in a SAM/BAM read group?
The sample tag (i.e. SM) was a mandatory tag in the initial SAM spec (see the .pages file; you need a mac to open it). When transitioned to Latex, this requirement ...
- 6,485
9
votes
How to subset a BAM by a list of QNAMEs?
use picard FilterSamReads http://broadinstitute.github.io/picard/command-line-overview.html#FilterSamReads
READ_LIST_FILE (File) Read List File containing reads that will be included or excluded ...
- 1,501
9
votes
Accepted
How to validate that BAMs have been downloaded correctly?
samtools quickcheck is all you need. From the manual:
Quickly check that input files appear to be intact. Checks that beginning of the file contains a valid header ...
- 19.6k
9
votes
Accepted
Extracting all reads from bam file which match read IDs in another file
It is still slow but grep has a -f option to take in a file
samtools view inbam.bam | grep -f read_names.txt > read_locs.txt
- 2,574
8
votes
What is the difference between FASTA, FASTQ, and SAM file formats?
FASTA (officially) just stores the name of a sequence and the sequence, unofficially people also add comment fields after the name of the sequence. FASTQ was invented to store both sequence and ...
- 754
8
votes
Merge hundreds of small BAM files into a single BAM file
Merging sorted files is a linear operation, so any well-implemented tools that do it will do it with approximately the same efficiency. So samtools merge (use the ...
- 956
8
votes
Accepted
True bam file encoding and viewing as text
Why not work only with sam files and we need to convert them to bam?
SAM is for human only, a binary format like BAM is smaller and parsed much faster by a program (and we want speed and optimize ...
- 1,501
8
votes
ethnicity check either from bam or vcf files
The main difficulty here is the use of GRCh38. Unfortunately, despite the fact that it's more than four years old, the major ethnicity-labeled public datasets (1000 Genomes, gnomAD when allele ...
Only top scored, non community-wiki answers of a minimum length are eligible