3
$\begingroup$

There's a variety of formats for storing genomic features (e.g. Genbank, GFF, GTF, Bedfile) but all of the ones that I'm familiar with use either a custom format (Genbank) or a CSV with defined column conventions and space for additional data (GFF, GTF, Bedfile).

Is there a feature format that is JSON based and adheres to a defined schema?

$\endgroup$
8
  • $\begingroup$ It is a nice question, but why do you expect JSON schema for this type of data What would be its advantages over other file formats? $\endgroup$
    – llrs
    Sep 12, 2019 at 15:57
  • $\begingroup$ Easy to validate, easy to understand, enforcement of conventions. I find that with bed or gff files I'm constantly either going back to the definition to find out what the columns are or trying to figure them out based on their content. This is especially annoying with a multi-column tsv file that has missing values for columns (as can happen in the genePred format). Some file types (e.g. GFF) allow for additional fields whose names are not enforced. So one file could have geneName and another could have gene_name. $\endgroup$
    – juniper-
    Sep 12, 2019 at 16:03
  • $\begingroup$ Bear in mind that this is a very subjective issue. For example, I find JSON an absolute pain to work with and very hard to understand, while simple, tab-delimited text files are a pleasure to work with. Note that if a field is empty, you have F1\tF2\t\tF4, so empty fields are also easily dealt with. $\endgroup$
    – terdon
    Sep 12, 2019 at 16:23
  • $\begingroup$ @terdon Completely agree on the subjectivity of the issue. I'm not trying to convert anybody. Just curious if such a format exists. $\endgroup$
    – juniper-
    Sep 12, 2019 at 16:26
  • $\begingroup$ Would you be interested in solutions that somehow convert gff to json? I'm sure there are tools or Bio[Python|Perl|R] modules for this. It would help if you could clarify exactly what genomic features you're after though. Since it's such a vague term, you kinda need vague formats for it :/. $\endgroup$
    – terdon
    Sep 12, 2019 at 16:28

3 Answers 3

5
$\begingroup$

JVCF is a JSON schema for describing genetic variants. BioJSON is a JSON schema for multiple sequence alignments. The output of mygene.info's gene annotation web service is JSON-formatted.

You might build one of your own schemas from any or all of these, as inspiration for structure and content.

Consider that most genomics formats are historically tabular, because they are not hierarchical, because they originated prior to the advent of JavaScript as a common programming language (and subsequent use of JSON to serialize data), and because a lot of stream-oriented UNIX tools read in data and process it in ways that are natural to tabular structure.

Some tabular results can be hierarchical, but use inefficiencies in field values to apply hierarchy. JSON is inefficient for storing tabular data, but it is absolutely wonderful for hierarchical data. This difference is perhaps worth considering.

One could think of storing a gene as a master object containing various haplotypes, exons, introns, features, associated variants, associated TF binding sites, gene ontologies, etc. Ordering or prioritization of such items would be possible with the use of sorted lists within a gene object. The format is as extensible as it is a matter of adding a property, list, or object to the schema.

To gain adoption or "buy-in", build tools which publish it to users, and tools that consume what users get in order to do things, along with conversion and validation utilities.

$\endgroup$
1
  • 1
    $\begingroup$ Thanks for the detailed answer! I think you hit the nail on the head with tabular vs. hierarchical data. Formats like GFF, Genbank and genePred store hierarchical data in a tabular format which makes it difficult to parse. Contigs, genes, mRNAs, CDSs, introns and exons have a very natural organization that may be well suited to a JSON-like structure. $\endgroup$
    – juniper-
    Sep 13, 2019 at 16:42
1
$\begingroup$

BioJS has capacity to read and write GFF as streams, it might have something of interest such as this: https://github.com/biojs-io/biojs-io-gff

$\endgroup$
1
  • $\begingroup$ Oh neat! Looks like this implementation converts GFF to its own internal JSON representation. $\endgroup$
    – juniper-
    Sep 13, 2019 at 16:49
1
$\begingroup$

I've written a Python library called PyReference which converts a GTF / GFF (RefSeq and Ensembl) into a gzipped JSON file.

You can just use this if you like but there's also a Python wrapper around the JSON, which allows you to write genomics code more naturally.

Python isn't known for being fast but the library function for reading a JSON file is highly optimised - the following takes less than 4 seconds on my laptop:

import numpy as np
import pyreference

reference = pyreference.Reference()

my_gene_ids = ["MSN", "GATA2", "ZEB1"]
for gene in reference[my_gene_ids]:
    average_length = np.mean([t.length for t in gene.transcripts])
    print("%s average length = %.2f" % (gene, average_length))
    print(gene.iv)
    for transcript in gene.transcripts:
        if transcript.is_coding:
            threep_utr = transcript.get_3putr_sequence()
            print("%s end of 3putr: %s" % (transcript.get_id(), threep_utr[-20:]))
$\endgroup$
1
  • $\begingroup$ This looks promising, thank you! It would be even better if there was an API reference linked to in the GitHub repo ;-) $\endgroup$
    – juniper-
    Jan 16, 2022 at 3:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.