6
$\begingroup$

Both file formats are advertised as more memory/parsing efficent than old PDB format and its successor mmCIF format, but I can't understand the context (I'm a developer, but pretty new to the field of bioinformatics).

Is BinaryCIF a later iteration of MMTF concepts/ideas (so it's a new competing format) or is the purpose of the two formats different ?

Refs :

$\endgroup$
6
$\begingroup$

So binaryCIF is newer and is heavily inspired by MMTF.

NGL, the most "loved" JS library (IMO) to show protein fetches proteins as MMTF, while Mol* uses bCIF —Mol* has the official repo of the bCIF format. Alex Rose, developed the former at the RCSB PDB, and the latter in collaboration with the PDBe. He also worked with Antony Bradley, the main author on the MMTF format, who left the RCSB PDB, some 4+ years ago. So there is definitely no competition there.

  • The MMTF download of the RCSB works well: https://mmtf.rcsb.org/download.html
  • The MMTF format did not catch on —okay, mmCIF is only slowly replacing PDB, which is still the only accepted format for analyses—, hence the need for a bCIF format
  • The libraries to do the conversion aren't without glitches
  • In the distant future bCIF will be the sole binary format
  • mmCIF <--> bCIF conversion is easier

However, I really ought to stress that mmCIF is a better choice to bCIF or mmTF. And even then many tools still hate mmCIFs and use PDB format. If memory footprint is not really a problem, then stick with mmCIF. If you are running a webserver, you may be sending data as gzipped making this a moot point.

Last year for my site, michelanglo, I was faced with a dilemma: Mol* has dotted lines for gaps and does not crash the browser with a virus or someone wanting to show hyperball representation on all residues. However, it was not documented (and then glitchy) so I firmly stuck with NGL, which is excellent. For file formats, I stick with PDB and a metadata dictionary as I know the format well and is despite having the jankiest syntax and no real metadata support (REMARK... ha!), does not break isopetides, disulfides or for no apparent reason.

$\endgroup$
2
  • $\begingroup$ Good post. As stated, pdb (from the age of Fortran) remains a strong favourite $\endgroup$
    – M__
    Nov 7 '20 at 21:15
  • $\begingroup$ Thanks a lot for this useful and detailed response : a lot of context, exactly what I needed. We'll use MMTF essentially to send files to NGL (as you have guessed ;) ), internally our tools will work with PDB and mmCIF. I'll keep an eye on bCIF, as Mol* has impressive tools (but not needed at our current stage). Have a nice day and thanks again ! $\endgroup$
    – ashas
    Nov 9 '20 at 8:20
2
$\begingroup$

Consider mmJSON.

Each of the three PDB sites had the same problem: using mmCIF files in web-based viewers is inefficient. To solve this problem three new file formats were introduced:

  • MMTF at RCSB,
  • BinaryCIF at PDBe,
  • mmJSON at PDBj.

MMTF has the smallest files – it's aggressively optimized for size. It is based on MessagePack, but on top of it uses a number of custom compression schemes, which make the format rather complex. Usually it's not a problem – MMTF developers made an effort to provide libraries for reading MMTF files in the most popular programming languages. But you need to install these libraries. To make the file smaller, MMTF doesn't contain all the metadata from mmCIF, which may be a blocker in some cases. What is nice is that the MMTF developers reached out to other developers in this field, organized some gathering and they were taking comments and suggestions from the outside. As the result, a few popular programs such as PyMOL and JMol can also read MMTF.

BinaryCIF is clearly based on (or at least inspired by) MMTF. So it's also built on top of MessagePack with a number of custom compression schemes. It includes all the mmCIF metadata, which makes the files slightly bigger, but I think it's a good trade-off. Initially there was only TypeScript code for reading it, now I see that there are also implementations in Java in Python.

mmJSON is 1:1 equivalent of mmCIF, but uses JSON instead of CIF. There is no extra compression added, but having the data from mmCIF re-ordered column-wise makes gzip compression much more efficient (this was also noted by MMTF developers). If the file is stored as json.gz or downloaded in web browser (with the usual compression between the server and the client) its size is 30-40% of gzipped mmCIF. It's not as small as MMTF and BinaryCIF (which are several times smaller than mmCIF) and this is the only downside.

The advantage is that you can read in it any language like all other JSON files. No need to install/update MessagePack + MMTF/BinaryCIF libraries.

A few years ago I compared reading MMTF and mmJSON in Python – reading mmJSON was order of magnitude faster. It may be very different with libraries in other languages, but generally JSON parsers are fast. The fastest ones parse GBs per second. Reading binary formats can be faster, but decompression schemes can also make it slower. Anyway, all the three formats are fast enough to read.

Sadly, I don't think that mmJSON is as popular as it should be. Perhaps I'm the only person using it outside of PDBj. That's why I'm writing this answer.

The mmJSON files are available at: https://pdbj.org/help/mmjson

$\endgroup$
2
  • $\begingroup$ Thanks ! Really useful added context and experience. Your POV on mmJSON is really interesting if one want to read/parse it in a JS or Python based stack. Do you know if pdbj.org is in good sync with rcsb.org (on which we sync regularly)? Is the transformation mmCIF => mmJSON reliable (Matteo said above that some conversions are glitchy)? $\endgroup$
    – ashas
    Sep 29 at 14:33
  • 1
    $\begingroup$ @ashas all the wwPDB sites are updated on Wednesdays. I suppose the conversion is reliable, I don't know about any problems. $\endgroup$
    – marcin
    Sep 29 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.