Skip to main content

Questions tagged [file-formats]

File format are sets of defined rules to organise a specific type of data. Common file format in bioinformatics include FASTA, FASTQ, SAM, BAM, VCF.

Filter by
Sorted by
Tagged with
51 votes
9 answers
7k views

What's the most efficient file format for the storage of DNA sequences?

I'd like to learn which format is most commonly used for storing the full human genome sequence (4 letters without a quality score) and why. I assume that storing it in plain-text format would be ...
kenorb's user avatar
  • 1,293
41 votes
4 answers
61k views

What is the difference between FASTA, FASTQ, and SAM file formats?

I'd like to learn the differences between 3 common formats such as FASTA, FASTQ and SAM. How they are different? Are there any benefits of using one over another? Based on Wikipedia pages, I can't ...
kenorb's user avatar
  • 1,293
18 votes
7 answers
16k views

How to convert FASTA to BED

I have a FASTA file: ...
SmallChess's user avatar
  • 2,699
17 votes
3 answers
5k views

Convert a BAM file from one reference to another?

I have a set of BAM files that are aligned using the NCBI GRCh37 human genome reference (with the chromosome names as NC_000001.10) but I want to analyze it using a BED file that has the UCSC hg19 ...
morgantaschuk's user avatar
17 votes
3 answers
6k views

BAM to BigWig without intermediary BedGraph

I have a pipeline for generating a BigWig file from a BAM file: BAM -> BedGraph -> BigWig Which uses bedtools genomecov ...
Nathan S. Watson-Haigh's user avatar
14 votes
3 answers
6k views

How do you write a .gz fastq file with Biopython?

How do you write a .gz (or .bgz) fastq file using ...
Mark Ebbert's user avatar
  • 1,354
14 votes
1 answer
959 views

Why would someone use a CRAM instead of a BAM?

I had this question from a graduate student yesterday, and I was stuck. What should I say? Why use a CRAM instead of a BAM? When is it a good idea to use a CRAM instead of a BAM? When is it a bad ...
EB2127's user avatar
  • 1,423
14 votes
2 answers
303 views

Is there a standard k-mer count file format?

I am doing a research project involving calculating k-mer frequencies and I am wondering if there is any standard file format for storing k-mer counts.
Jon Deaton's user avatar
13 votes
6 answers
405 views

Are there any databases of templates for common bioinformatic file formats?

I want some templates of different file formats that I can use to test my scripts and identify possible bugs in my code. For example, consider nucleotide FASTA, a simple but often abused format, I ...
Chris_Rands's user avatar
  • 3,948
13 votes
1 answer
4k views

What's the difference between VCF spec versions 4.1 and 4.2?

What are the key differences between VCF versions 4.1 and 4.2? It looks like v4.3 contains a changelog (specs available here) but earlier specifications do not. This biostar post points out one ...
blmoore's user avatar
  • 366
11 votes
2 answers
430 views

Do variant calls change when you call from CRAM?

We're considering switching our storage format from BAM to CRAM. We work with human cancer samples, which may have very low prevalence variants (i.e. not diploid frequency). If we use lossy CRAM to ...
morgantaschuk's user avatar
10 votes
2 answers
291 views

What are the de facto required fields in a SAM/BAM read group?

The SAM specification indicates that each read group must have a unique ID field, but does not mark any other field as required. I have also discovered that htsjdk throws exceptions if the sample (...
mattm's user avatar
  • 754
10 votes
4 answers
280 views

Annotation format design

Bashing file formats is a favorite pastime in bioinformatics, and annotation file formats such as GFF and BED seem to get special attention. A lot of this frustration stems from community's shockingly ...
Daniel Standage's user avatar
9 votes
5 answers
1k views

Converter between PDB or mmCIF and MMTF

I'd like to test MMTF, a new format for storing biomolecular structures which is promoted by RCSB as a more compact alternative to mmCIF and PDB. From MMTF FAQ: How do I convert a PDBx/mmCIF ...
marcin's user avatar
  • 1,261
9 votes
3 answers
248 views

PDB format: remark number for free text

I would like to add a text to PDB files that I'm processing with my tool, rna-pdb-tools. Someone points that the way I'm using it right now it's not correct (https://github.com/mmagnus/rna-pdb-tools/...
Marcin Magnus's user avatar
8 votes
2 answers
7k views

How is the GT field in a VCF file defined?

As my question in SO was closed and asked to be posted in this forum, I am posting it here. I am not from the bioinformatics domain. However, for the sake of analysis, I am trying to pick up certain ...
The Great's user avatar
  • 227
8 votes
3 answers
3k views

How can I read FCS files using open source libraries?

FCS is a patented data format used for storing flow cytometry data. The most recent version is FCS3.1. There is some documentation on the format, but there is no information on how to read these files....
WYSIWYG's user avatar
  • 263
8 votes
1 answer
1k views

How GFF3 attributes (9th column) varies from one gene prediction algorithm to another

GFF3 files are in tabular format with 9 fields per line, separated by tabs. The first 8 fields share almost same data structure, but the 9th field varies a lot depending on feature type and gene ...
Arijit Panda's user avatar
8 votes
3 answers
3k views

How to convert the .vcf (imputed) file with GT:GP format to GT:DS?

I have the genotyped data from impute2 output in .gen format (imputed to 1000G P3). The file has genotype posterior probabilities (GP:3 values per variant). I have converted .gen to .vcf using qctools ...
Nilufer's user avatar
  • 81
6 votes
1 answer
715 views

Why do SAM and BAM use different coordinate systems?

BAM files are, at least as far as I know, simply binary compressed versions of SAM files. They have the exact same information and are used in the same way. Why then does the SAM format use a 1-based ...
terdon's user avatar
  • 10.2k
6 votes
2 answers
255 views

BinaryCIF vs MMTF formats, which one to choose?

Both file formats are advertised as more memory/parsing efficent than old PDB format and its successor mmCIF format, but I can't understand the context (I'm a developer, but pretty new to the field of ...
ashas's user avatar
  • 63
6 votes
3 answers
1k views

Infer missing UTR features in GFF3 file

I am working on a GFF file that is missing the 5'UTR and 3'UTR information. For example: ...
l0110's user avatar
  • 292
6 votes
1 answer
274 views

What is this 5-column sequencing format from 2009?

There is high throughput sequencing data here, and I don't know what format it is in. It was submitted in 2009, and the description says the following: Library strategy: ncRNA-Seq Library source:...
bli's user avatar
  • 3,130
6 votes
1 answer
1k views

How should the SAM MD tag match the CIGAR string?

I am trying to understand how the MD:Z tag is used. The following is from the SAM Optional Fields Specification, which gives an example but is not thorough. The MD field aims to achieve SNP/indel ...
mattm's user avatar
  • 754
6 votes
1 answer
370 views

Run gffread in multi-thread mode

Is there any possibility to run gffread in multi-thread mode? The answer seems to be 'no' from the manual (or gffread -h), as no multi-thread option is mentioned. ...
aechchiki's user avatar
  • 2,676
6 votes
2 answers
405 views

Meaning of the FORMAT fields of the VCF file coming from GIAB project

After reading the GIAB paper in https://www.biorxiv.org/content/early/2018/05/25/281006 and its Figure 1, I am still having trouble understanding the data inside the GIAB VCF file for HG001 (...
Javier's user avatar
  • 161
6 votes
1 answer
77 views

Determine reference for reference-compressed SRA file

Note: this question was also asked on Github. I have 241 SRA files that appear to be reference compressed. I didn't even know this was a thing until I tried to convert them to Fastq files without an ...
Daniel Standage's user avatar
5 votes
2 answers
2k views

bcftools output vs. bgzip

I've got what seems like it should be a simple question, but I can't seem to figure it out from google. bcftools has 4 output options: ...
heathobrien's user avatar
  • 1,816
5 votes
1 answer
126 views

What format is this? Pretty sure it's not a BED file

Time for guess the bioinformatics file format... :) I found this '.bed' file on GEO (GSE84660), but it's clearly not a BED file. Anyone know what it might be? And what can view it? It's from a HiC ...
ithinkiam's user avatar
  • 356
5 votes
1 answer
530 views

Can a data file in VCF format be converted into FASTA?

I'm considering purchasing the 'MyGenome' product by Veritas Genetics to analyze my genome for a project. I'd like the data to be in FASTA format, but Veritas only provides VCF data. Is it possible to ...
WagonWheelWilly's user avatar
5 votes
1 answer
321 views

Why doesn't Biopython AlignIO.read() recognise the 'mauve' format?

On this Biopython tutorial, they describe how to import a multiple sequence alignment in the Mauve (XMFA: extensible multi fasta format). So I imported the AlignIO module: ...
Biomagician's user avatar
  • 2,459
5 votes
3 answers
526 views

How to indicate the END of a haplotype block in VCF?

In VCF I know how to indicate that two genotypes are in the same phase by using consecutive "0|1" and "1|0" genotype fields, for example. However, how do I deal with the case that the first two ...
Dan's user avatar
  • 612
5 votes
1 answer
184 views

Is the optional SAM NM field strictly computable from the MD and CIGAR?

From SAM Optional Fields Specification the NM field is Edit distance to the reference, including ambiguous bases but excluding clipping Assuming both the MD and CIGAR are present, is the edit ...
mattm's user avatar
  • 754
5 votes
1 answer
549 views

Customizing bigWig file

I generate bigWig files using bamCoverage from deeptools, in part for my colleagues to visualize their mapped libraries in the IGV viewer. A problem is that the ...
bli's user avatar
  • 3,130
4 votes
3 answers
6k views

How to convert Bed file to fasta file?

I have Bed file containing start and end of a sequence, and I need to convert it to fasta format, any recommendations?
Nour El-Islam Awad's user avatar
4 votes
1 answer
3k views

How to quickly and robustly convert between mmCIF and PDB?

There is already a question on PDB/CIF to MMTF, however what is a robust way to programmatically go between PDB and CIF files? For example I can use a python script from this gist that relies on ...
James's user avatar
  • 409
4 votes
1 answer
85 views

What does the "X_OS_IND" column mean?

I want to survival analysis using the subset of TCGA LUAD dataset, which identifier are located here. ...
WangShixiang's user avatar
4 votes
1 answer
350 views

What kind of "gff" format does bioawk parse?

I was wondering if I could use the gff parsing capability of bioawk to facilitate the parsing of gtf files, and I looked at the following help message: ...
bli's user avatar
  • 3,130
4 votes
1 answer
448 views

What are the differences between BNGL (BioNetGen Language) and SBML (System Biology Markup Langague) formats?

Per request from meta comment. I am self-learning about whole cell modeling, specifically An introduction to whole-cell modeling and Fundamentals of Systems Biology: From Synthetic Circuits to Whole-...
Guy Coder's user avatar
  • 315
4 votes
1 answer
114 views

How to represent trans-spliced genes in GTF?

For example, see this gene (nad1) in ENA: http://www.ebi.ac.uk/ena/data/view/ABI60879 If you look at the XML for that gene you see the following: ...
Dan's user avatar
  • 612
4 votes
1 answer
1k views

get gene lines from gtf file

I would like to retrieve gene lines from a GTF file for which I only have exons & transcripts lines (output from Cufflinks) and alternative splicing possible. I need gene lines for compatibility ...
aechchiki's user avatar
  • 2,676
3 votes
4 answers
8k views

Can a gff file be converted to a fasta file?

I downloaded an annotated genome file in gff format here. I would like to use it for proteomics. Though I need it in fasta format. Is there any tool that converts gff to annotated fasta? I see the ...
Soerendip's user avatar
  • 1,295
3 votes
2 answers
108 views

How is the "canonical" version (`_entity_poly.pdbx_seq_one_letter_code`) obtained in the PDB?

I encountered the dichotomy in the context of PDBx/mmcif files, say, 6OSQ: each chain has a _entity_poly.pdbx_seq_one_letter_code...
rtviii's user avatar
  • 364
3 votes
1 answer
95 views

How to interpret UniProt allele patterns?

In creating a parser for the UniProt flat file uniprot_sprot.dat (directory) in the manual for the comments POLYMORPHISM there ...
Guy Coder's user avatar
  • 315
3 votes
1 answer
51 views

How can I export a pruned phylogenetic tree in nexus format in R?

I have a phylogenetic tree with a lot of different populations and I wanted to removes most of them and apply new names to the labels. I wrote this code: ...
Nickmofoe's user avatar
  • 339
3 votes
1 answer
159 views

How can I convert my tree to the correct nexus format?

I would like to use the program BayesTraitV4. I have my phylogenetic tree in NEXUS format but the program does not accept it. I checked the example data and while they are also in NEXUS format, they ...
Nickmofoe's user avatar
  • 339
3 votes
1 answer
377 views

Converting convert .svs files into .mrxs format

I wish to convert Aperio AFI microscope .svs files from a pathology lab microscope into 3DHistech .mrxs format. Lab ppl have some pipeline X and they asked me to look into the matter. I can read the ...
pinegulf's user avatar
3 votes
1 answer
3k views

Convert PAF format to SAM/BAM format

I have a bunch of PAF files resulting from the alignments of fastq files on a reference genome with minimap2. I would like to convert them into SAM/BAM format so I can use ...
Paul Endymion's user avatar
2 votes
3 answers
1k views

I have 23andme text files and would like to convert to SAM/BAM format

I would like to convert 23andme text file to NextGen to BAM file for Yfull.com to read. It is difficult to get answers on how to convert to SAM/BAM file for the 23andme text file to be converted to ...
Owance Jackie's user avatar
2 votes
2 answers
220 views

What do these files / annotations mean?

I have no experience in Bioinformatics and I need to understand what the annotations given here mean (I am including the first few lines, please see the link for more): ...
Dendrobium's user avatar