57 votes
Accepted

What is the difference between FASTA, FASTQ, and SAM file formats?

Let’s start with what they have in common: All three formats store sequence data, and sequence metadata. Furthermore, all three formats are text-based. However, beyond that all three formats are ...
user avatar
41 votes
Accepted

What's the most efficient file format for the storage of DNA sequences?

Genomes are commonly stored as either fasta files (.fa) or twoBit (.2bit) files. Fasta files store the entire sequence as text and are thus not particularly compressed. twoBit files store each ...
user avatar
  • 880
25 votes

What's the most efficient file format for the storage of DNA sequences?

The standard formats for storing sequence data are fasta and fastq. Fasta is used if you only need the raw sequence data, fastq is used if you want to store the sequence data along with the quality ...
user avatar
  • 841
23 votes
Accepted

Convert a BAM file from one reference to another?

You're the second person I have ever seen using NCBI "chromosome names" (they're more like supercontig IDs). Normally I would point you to a resource providing mappings between chromosome names, but ...
user avatar
  • 19.2k
20 votes

What's the most efficient file format for the storage of DNA sequences?

The standard and the most common sequence format is FASTA for sure. You can compress it with a compressor. For the ~3GB human genome, gzip reduces the size to ~900MB, depending on the option in use. ...
user avatar
  • 5,735
18 votes

What is the difference between FASTA, FASTQ, and SAM file formats?

In a nutshell, FASTA file format is a DNA sequence format for specifying or representing DNA sequences and was first described by Pearson ...
user avatar
  • 309
18 votes
Accepted

How do you write a .gz fastq file with Biopython?

I'm not sure I'm doing it the best way, but here is an example where I read a compressed gzip fastq file and write the records in block gzip fastq: ...
user avatar
  • 2,960
17 votes
Accepted

Why would someone use a CRAM instead of a BAM?

Whenever you want to save space (this can be a substantial savings). Until quite recently (samtools/htslib 1.7), only CRAM supported long CIGAR strings. If you need to guarantee that any random ...
user avatar
  • 19.2k
16 votes
Accepted

What's the difference between VCF spec versions 4.1 and 4.2?

This is easy to check, you can download both specs in .tex format and do diff. Changes to the v4.2 compared to v4.1: Information field format: adding source and ...
user avatar
14 votes

How to convert FASTA to BED

You can do this easily with bioawk, which is a version of awk with added features facilitating bioinformatics: ...
user avatar
  • 2,960
13 votes

What's the most efficient file format for the storage of DNA sequences?

There are several things to consider when asking for "the most efficient" way to store data, it all depends on your use case. Do you just need ACGT, or are there also IUPAC codings for combinations? ...
user avatar
  • 744
12 votes
Accepted

Why do SAM and BAM use different coordinate systems?

Arithmetic on a zero-based coordinate system is less complicated than that on a one-based system, so it appears zero-based is often (not exclusively) used for binary data formats, like BAM or bigBed, ...
user avatar
11 votes
Accepted

BAM to BigWig without intermediary BedGraph

This can be done in R very easily from an indexed .bam file. Given single-end file for sample1. ...
user avatar
  • 1,498
11 votes

How to convert FASTA to BED

It's good practice to have your FASTA indexed, so you can leverage the .fai you are likely to already have. If not, you can just ...
user avatar
11 votes

I have 23andme text files and would like to convert to SAM/BAM format

Having done 23andme myself I can tell you that your variant file, which contains SNP genotypes, cannot be converted to a bam file. It does not contain the same information as a bam file. It may be ...
user avatar
9 votes
Accepted

What are the de facto required fields in a SAM/BAM read group?

The sample tag (i.e. SM) was a mandatory tag in the initial SAM spec (see the .pages file; you need a mac to open it). When transitioned to Latex, this requirement ...
user avatar
  • 5,735
8 votes

What's the most efficient file format for the storage of DNA sequences?

I think the question is a bit ambiguous so please excuse this answer that's a bit redundant from the rest of the ones provided. As others have mentioned, if you want to store a full genome, ...
user avatar
  • 181
8 votes
Accepted

How GFF3 attributes (9th column) varies from one gene prediction algorithm to another

The first place to start is the GFF3 specification. This is the official word on what is and is not allowed in a GFF3 file. For example, users can define arbitrary attribute keys, so long as they do ...
user avatar
7 votes

What's the most efficient file format for the storage of DNA sequences?

It is not yet standardized, but graph format has the potential for being the most space-efficient method for storing genomes. The idea is this: rather than store a genome as a linear string of ...
user avatar
  • 429
7 votes

What is the difference between FASTA, FASTQ, and SAM file formats?

FASTA (officially) just stores the name of a sequence and the sequence, unofficially people also add comment fields after the name of the sequence. FASTQ was invented to store both sequence and ...
user avatar
  • 744
7 votes

Are there any databases of templates for common bioinformatic file formats?

You mention Biopython, which contains tests: https://github.com/biopython/biopython/tree/master/Tests. Some of the tests consist in reading files present in the folders listed in the above link. ...
user avatar
  • 2,960
7 votes
Accepted

What is this 5-column sequencing format from 2009?

This is an early Solexa/Illumina sequencer format. The columns are the identifier to location on the flowcell. I believe the "." was the original placeholder for an unread base, which has ...
user avatar
  • 2,556
7 votes
Accepted

Script to run everything in a loop for extracting tar.gz files into fastq and to bam with alignment?

This should work for you: ...
user avatar
  • 1,806
6 votes

How to convert FASTA to BED

You could adapt this awk one-liner. Note that it assumes that sequence IDs are not longer than 100 characters and that there is no description following the sequence ID on the header line. ...
user avatar
  • 646
6 votes
Accepted

Do variant calls change when you call from CRAM?

By default, a CRAM you create with samtools is lossless. It typically halves the input BAM in terms of file size. If you want to compress more, you can let samtools convert most read names to integers....
user avatar
  • 5,735
6 votes
Accepted

Is there a standard k-mer count file format?

Not as far as I am aware. The Ray assembler used to (and possibly still does) store the kmers as FASTA files where the header was the count of the sequence, which I thought was a pretty neat ...
user avatar
  • 11.7k
6 votes

What format is this? Pretty sure it's not a BED file

It looks like a weird way to represent data. You're right, it does not at all look like a BED. Digging in GEO made me find this information: Supplementary_files_format_and_content: Tab-delimited ...
user avatar
  • 2,644
6 votes
Accepted

BinaryCIF vs MMTF formats, which one to choose?

So binaryCIF is newer and is heavily inspired by MMTF. NGL, the most "loved" JS library (IMO) to show protein fetches proteins as MMTF, while Mol* uses bCIF —Mol* has the official repo of ...
user avatar
  • 3,469
5 votes

Are there any databases of templates for common bioinformatic file formats?

Not that I am aware. It is best to go with format specifications when coding. Also it may be good to look at the example files that come together with various tools performing file conversions and ...
user avatar
  • 481
5 votes
Accepted

What kind of "gff" format does bioawk parse?

I would consider the description there a bug. The filter is actually the strand, strand is the frame, ...
user avatar
  • 19.2k

Only top scored, non community-wiki answers of a minimum length are eligible