Google released their variant caller DeepVariant which won the highest SNP performance award in the Precision FDA Truth challenge (99.999% accuracy).

From the linked github repo, we see that DeepVariant is a CNN, we provide images of the reads as an input and train the neural network with a stochastic gradient descent algorithm.

To me this is unintuitive. Why use images of reads when you already have the reads data?

  • 2
    $\begingroup$ en.wikipedia.org/wiki/Law_of_the_instrument $\endgroup$ Apr 19, 2018 at 14:31
  • 1
    $\begingroup$ Where did you see they need images? The documentation says the input is a bam file, just like any other caller. $\endgroup$
    – terdon
    Apr 19, 2018 at 14:38
  • 1
    $\begingroup$ @terdon Images (of pileups, of course) are mentioned in the preprint. I think they're also using the Inception CNN, which is heavily used in image processing. I think many people forget that images are just matrices. $\endgroup$
    – Devon Ryan
    Apr 19, 2018 at 14:50
  • 1
    $\begingroup$ @terdon It seems that the first version was on images: We have made a number of improvements to the methodology as well. The biggest change was to move away from RGB-encoded (3-channel) pileup images and instead represent the aligned read data using a multi-channel tensor data layout. and now they use something else. Thank you! $\endgroup$ Apr 19, 2018 at 14:50
  • $\begingroup$ As @DevonRyan has already said, these are not images per se. From a machine's point of view an image is just a bunch of matrices (one per colour channel). I'd say this is just an example of bad/confusing use of terminology. $\endgroup$ Apr 21, 2018 at 20:14

2 Answers 2


It's more "images of reads" rather than actual images of reads. In reality, they're feeding in a pileup, which is just a matrix of letters or numbers (you can visualize this as an image, of course). This is more apparent if you look at one of their example python notebooks, CNNs are popular in the image processing field, but at the end of the day images are just large matrices of values so any time you can convert your input into that sort of format then a CNN can be useful (provided you have sufficient training data and some GPUs to throw at things).

  • 1
    $\begingroup$ I find the answer confusing: are you saying that the CNN is not fed images? Because the way google describes DeepVariant, they are indeed creating pileup image data and putting that through their CNN. $\endgroup$ Apr 30, 2018 at 17:05
  • $\begingroup$ It's not an image, it's an "image". The DeepVariant are a bit ambiguous on that point, though at the end of the day it's a multidimensional matrix of values (the relation of that to an image being obvious). $\endgroup$
    – Devon Ryan
    Apr 30, 2018 at 18:05

Why use images of reads when you already have the reads data?

Neural networks take one or multiple N-dimension arrays (n-d arrays) as input; they can't take arbitrary forms of data. To feed reads and their alignment into the network, you have to encode them as n-d arrays. If you ignore "colors" for now, a straightforward choice is a (windowLength, readDepth) 2D-array as you see in an alignment browser – it is an image. The "colors" of this image are often called "channels" or "features" in deep learning (DL). They represent base types and base quality in deepVariant. In the end, you get a (windowLength, readDepth, nFeatures) 3D array as input. This is the most natural encoding to DL researchers.

Why chose CNN for a variant caller

That is because CNN has been repeatedly proved to be the best architecture for such (height, width, nChannels) 3D arrays as input. Read alignments are exactly that.

On a historical note, deepVariant was first built on top of google's now-deprecated DisBelief framework that only takes RGB images as input. They had to squeeze all types of information into three color channels. Now with TensorFlow, an image can have an arbitrary number of color channels. This simplifies implementation and improves accuracy. In addition to this image, deepVariant also takes other information as input.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.