# Why chose CNN for a variant caller

Google released their variant caller DeepVariant which won the highest SNP performance award in the Precision FDA Truth challenge (99.999% accuracy).

From the linked github repo, we see that DeepVariant is a CNN, we provide images of the reads as an input and train the neural network with a stochastic gradient descent algorithm.

To me this is unintuitive. Why use images of reads when you already have the reads data?

• en.wikipedia.org/wiki/Law_of_the_instrument – heathobrien Apr 19 '18 at 14:31
• Where did you see they need images? The documentation says the input is a bam file, just like any other caller. – terdon Apr 19 '18 at 14:38
• @terdon Images (of pileups, of course) are mentioned in the preprint. I think they're also using the Inception CNN, which is heavily used in image processing. I think many people forget that images are just matrices. – Devon Ryan Apr 19 '18 at 14:50
• @terdon It seems that the first version was on images: We have made a number of improvements to the methodology as well. The biggest change was to move away from RGB-encoded (3-channel) pileup images and instead represent the aligned read data using a multi-channel tensor data layout. and now they use something else. Thank you! – Claudiu Creanga Apr 19 '18 at 14:50
• As @DevonRyan has already said, these are not images per se. From a machine's point of view an image is just a bunch of matrices (one per colour channel). I'd say this is just an example of bad/confusing use of terminology. – Eli Korvigo Apr 21 '18 at 20:14

It's more "images of reads" rather than actual images of reads. In reality, they're feeding in a pileup, which is just a matrix of letters or numbers (you can visualize this as an image, of course). This is more apparent if you look at one of their example python notebooks, CNNs are popular in the image processing field, but at the end of the day images are just large matrices of values so any time you can convert your input into that sort of format then a CNN can be useful (provided you have sufficient training data and some GPUs to throw at things).

• I find the answer confusing: are you saying that the CNN is not fed images? Because the way google describes DeepVariant, they are indeed creating pileup image data and putting that through their CNN. – Konrad Rudolph Apr 30 '18 at 17:05
• It's not an image, it's an "image". The DeepVariant are a bit ambiguous on that point, though at the end of the day it's a multidimensional matrix of values (the relation of that to an image being obvious). – Devon Ryan Apr 30 '18 at 18:05