61
votes
Accepted
What is the difference between FASTA, FASTQ, and SAM file formats?
Let’s start with what they have in common: All three formats store
sequence data, and
sequence metadata.
Furthermore, all three formats are text-based.
However, beyond that all three formats are ...
- 4,825
42
votes
Accepted
What's the most efficient file format for the storage of DNA sequences?
Genomes are commonly stored as either fasta files (.fa) or twoBit (.2bit) files. Fasta files store the entire sequence as text and are thus not particularly compressed.
twoBit files store each ...
- 890
25
votes
What's the most efficient file format for the storage of DNA sequences?
The standard formats for storing sequence data are fasta and fastq. Fasta is used if you only need the raw sequence data, fastq is used if you want to store the sequence data along with the quality ...
- 841
23
votes
Accepted
Convert a BAM file from one reference to another?
You're the second person I have ever seen using NCBI "chromosome names" (they're more like supercontig IDs). Normally I would point you to a resource providing mappings between chromosome names, but ...
- 19.5k
20
votes
What's the most efficient file format for the storage of DNA sequences?
The standard and the most common sequence format is FASTA for sure. You can compress it with a compressor. For the ~3GB human genome, gzip reduces the size to ~900MB, depending on the option in use.
...
- 6,445
20
votes
What is the difference between FASTA, FASTQ, and SAM file formats?
In a nutshell,
FASTA file format is a DNA sequence format for specifying or representing DNA sequences and was first described by Pearson ...
- 329
19
votes
Accepted
How do you write a .gz fastq file with Biopython?
I'm not sure I'm doing it the best way, but here is an example where I read a compressed gzip fastq file and write the records in block gzip fastq:
...
- 3,090
17
votes
Accepted
Why would someone use a CRAM instead of a BAM?
Whenever you want to save space (this can be a substantial savings). Until quite recently (samtools/htslib 1.7), only CRAM supported long CIGAR strings.
If you need to guarantee that any random ...
- 19.5k
16
votes
Accepted
What's the difference between VCF spec versions 4.1 and 4.2?
This is easy to check, you can download both specs in .tex format and do diff.
Changes to the v4.2 compared to v4.1:
Information field format: adding source and ...
- 2,725
15
votes
How to convert FASTA to BED
You can do this easily with bioawk, which is a version of awk with added features facilitating bioinformatics:
...
- 3,090
13
votes
What's the most efficient file format for the storage of DNA sequences?
There are several things to consider when asking for "the most efficient" way to store data, it all depends on your use case. Do you just need ACGT, or are there also IUPAC codings for combinations? ...
- 754
12
votes
How to convert FASTA to BED
It's good practice to have your FASTA indexed, so you can leverage the .fai you are likely to already have. If not, you can just ...
- 772
12
votes
Accepted
Why do SAM and BAM use different coordinate systems?
Arithmetic on a zero-based coordinate system is less complicated than that on a one-based system, so it appears zero-based is often (not exclusively) used for binary data formats, like BAM or bigBed, ...
- 3,125
11
votes
Accepted
BAM to BigWig without intermediary BedGraph
This can be done in R very easily from an indexed .bam file.
Given single-end file for sample1.
...
- 1,563
11
votes
I have 23andme text files and would like to convert to SAM/BAM format
Having done 23andme myself I can tell you that your variant file, which contains SNP genotypes, cannot be converted to a bam file. It does not contain the same information as a bam file. It may be ...
- 1,314
9
votes
Accepted
What are the de facto required fields in a SAM/BAM read group?
The sample tag (i.e. SM) was a mandatory tag in the initial SAM spec (see the .pages file; you need a mac to open it). When transitioned to Latex, this requirement ...
- 6,445
8
votes
What is the difference between FASTA, FASTQ, and SAM file formats?
FASTA (officially) just stores the name of a sequence and the sequence, unofficially people also add comment fields after the name of the sequence. FASTQ was invented to store both sequence and ...
- 754
8
votes
What's the most efficient file format for the storage of DNA sequences?
I think the question is a bit ambiguous so please excuse this answer that's a bit redundant from the rest of the ones provided.
As others have mentioned, if you want to store a full genome, ...
- 181
8
votes
Accepted
How GFF3 attributes (9th column) varies from one gene prediction algorithm to another
The first place to start is the GFF3 specification. This is the official word on what is and is not allowed in a GFF3 file. For example, users can define arbitrary attribute keys, so long as they do ...
- 5,060
7
votes
What's the most efficient file format for the storage of DNA sequences?
It is not yet standardized, but graph format has the potential for being the most space-efficient method for storing genomes. The idea is this: rather than store a genome as a linear string of ...
- 429
7
votes
Are there any databases of templates for common bioinformatic file formats?
You mention Biopython, which contains tests: https://github.com/biopython/biopython/tree/master/Tests.
Some of the tests consist in reading files present in the folders listed in the above link. ...
- 3,090
7
votes
Accepted
What is this 5-column sequencing format from 2009?
This is an early Solexa/Illumina sequencer format. The columns are the identifier to location on the flowcell. I believe the "." was the original placeholder for an unread base, which has ...
- 2,574
7
votes
Accepted
Script to run everything in a loop for extracting tar.gz files into fastq and to bam with alignment?
This should work for you:
...
- 1,816
7
votes
How is the GT field in a VCF file defined?
0 and 1 is just a way of coding the reference (0) and alternate (1) allele. These could be A/G, C/T etc. It's just a simplified way of expressing the different alleles.
...
- 1,621
7
votes
Accepted
How is the GT field in a VCF file defined?
You can get most of the info from this paper. See Fig. 1 and the surrounding text. Quoting from there, "GT, genotype, encodes alleles as numbers: 0 for the reference allele, 1 for the first ...
- 755
6
votes
Accepted
How to convert FASTA to BED
We have many excellent answers! This will be an excellent reference for future users.
I found what exactly what I was asking in my question:
https://www.biostars.org/p/191052/
...
- 2,739
6
votes
How to convert FASTA to BED
You could adapt this awk one-liner. Note that it assumes that sequence IDs are not longer than 100 characters and that there is no description following the sequence ID on the header line.
...
- 646
6
votes
Accepted
Do variant calls change when you call from CRAM?
By default, a CRAM you create with samtools is lossless. It typically halves the input BAM in terms of file size. If you want to compress more, you can let samtools convert most read names to integers....
- 6,445
6
votes
Accepted
Is there a standard k-mer count file format?
Not as far as I am aware. The Ray assembler used to (and possibly still does) store the kmers as FASTA files where the header was the count of the sequence, which I thought was a pretty neat ...
- 13.5k
6
votes
What format is this? Pretty sure it's not a BED file
It looks like a weird way to represent data. You're right, it does not at all look like a BED.
Digging in GEO made me find this information:
Supplementary_files_format_and_content: Tab-delimited ...
- 2,656
Only top scored, non community-wiki answers of a minimum length are eligible