Filtering VEP annotation file

Question

I am filtering a VEP annotated vcf, trying to maintain just those variants classified as deleterious by SIFTand as damaging (probably or possibly included) by PolyPhen.

I am using:

filter_vep -i "$input" -o "$output" -filter "SIFT = deleterious and PolyPhen match damaging"

and I also tried:

filter_vep -i "$input" -o "$output" -filter "SIFT match deleterious and PolyPhen match damaging"

The output contains just variants with some value in these two fields, but some of the variants are classified as tolerated by SIFT and as benign by PolyPhen, so do you know why the filter is not working?

As an example: this two variants shouldn't exist in my filtered VCF, because the first one is classified as benign by PolyPhen and the second is classified as benign by both fields:

    CHROM  POS  REF  ALT  FILTER  INFO
    1       1934441 .       G       T       .       PASS    deleterious(0.01)|benign(0.138)
    1       207085100       .       G       A       .       PASS   tolerated(0.16)|benign(0.037)

As an example of a variant that has pass properly those filters:

    CHROM  POS  REF  ALT  FILTER  INFO
    2       3938449 .       T       A       .       PASS    deleterious(0.01)|probably damaging(0.67)

Those are 3 variants among those that I have in my filtered VCF, but I should be getting just those variants which are classified as deleterious and damaging by both SIFT and PolyPhen, as the third variant I've shown.

Answer 1

I think what you're trying to do could be achieved by adding parenthesis to the conditions and by specifying the format of the input file by using the flag --format vcf, as specified in the documentation. The final command should look like this:

filter_vep -i "$input" --format vcf --filter "(SIFT = deleterious) and (PolyPhen match damaging)" -o "$output"

Answer 2

Here is another way to achieve your goal using the fuc vcf_vep command I wrote:

$ fuc vcf_vep in.vcf 'SIFT.str.contains("deleterious") and PolyPhen.str.contains("damaging")' > out.vcf

For getting help:

$ fuc vcf_vep -h
usage: fuc vcf_vep [-h] [--opposite] [--as_zero] vcf expr

This command will filter a VCF file annotated by Ensemble VEP. It essentially wraps the `pandas.DataFrame.query` method. For details on query expression, please visit the method's documentation page (https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.query.html#pandas-dataframe-query).

examples:
  $ fuc vcf_vep in.vcf 'SYMBOL == "TP53"' > out.vcf
  $ fuc vcf_vep in.vcf 'SYMBOL != "TP53"' > out.vcf
  $ fuc vcf_vep in.vcf 'SYMBOL == "TP53"' --opposite > out.vcf
  $ fuc vcf_vep in.vcf 'Consequence in ["splice_donor_variant", "stop_gained"]' > out.vcf
  $ fuc vcf_vep in.vcf '(SYMBOL == "TP53") and (Consequence.str.contains("stop_gained"))' > out.vcf
  $ fuc vcf_vep in.vcf 'gnomAD_AF < 0.001' > out.vcf
  $ fuc vcf_vep in.vcf 'gnomAD_AF < 0.001' --as_zero > out.vcf

positional arguments:
  vcf         Ensemble VEP-annotated VCF file
  expr        query expression to evaluate

optional arguments:
  -h, --help  show this help message and exit
  --opposite  use this flag to return records that don’t meet the said criteria
  --as_zero   use this flag to treat missing values as zero instead of NaN

You can also do above with Python:

from fuc import pyvcf, pyvep
vf = pyvcf.VcfFrame.from_file('in.vcf')
expr = 'SIFT.str.contains("deleterious") and PolyPhen.str.contains("damaging")'
filtered_vf = pyvep.filter_query(vf, expr)
filtered_vf.to_file('out.vcf')

Answer 3

If you are working under a Linux environment, then you may easily use a piped grep search. For example:

grep deleterious file.vcf |grep damaging >filtered_file.vcf