0
$\begingroup$

This question has also been asked on Biostars

I have 2 directories that contains multiple VCF files. dir

I want to write code in python that will process all vcf files and create an AFD (allele frequency difference) plot for each VCF file present in a folder

I have started with this code, which attempts to convert the VCF files into a pandas dataframe:

# Import Module
import os
import pandas as pd
# Folder Path
path1 = "C://Users//USER//Desktop//isiah/VCFs_1/"
path2 = "C://Users//USER//Desktop//isiah/VCFs_2/"
#os.chdir(path1)

# Read text File


def read_text_file(file_path1,file_path2):
    with open(file_path, 'r') as f:
        print(f.read())


# iterate through all file
for file in os.listdir():
    # Check whether file is in text format or not
    if file.endswith(".vcf"):
        file_path1 = f"{path1}\{file}"
        file_path2 = f"{path2}\{file}"
        print(file_path1,"\n\n",file_path2)

# call read text file function
#data = read_text_file(path1,path2)
#print(read_text_file(path1,path2))
Final= pd.DataFrame(read_text_file(path1,path2))
print(Final)

and its giving me output without calling function and also gives me empty dataframe. Also i am not sure if result is showing all vcf files record or not.

Kindly help me to modify this code. I need my all result in dataframe so i can make plot easily

Folder 1 contains 3 test vcfs whereas folder 2 contains 4 test vcfs:

directories

This is test 1 vcf file:

##fileformat=VCFv4.2
##ALT=<ID=NON_REF,Description="Represents any possible alternative allele at this location">
##FILTER=<ID=FS,Description="FS>60.0">
##FILTER=<ID=HS,Description="HaplotypeScore<13.0">
##FILTER=<ID=LowQual,Description="Low quality">
##FILTER=<ID=MQ,Description="MQ<40.0">
##FILTER=<ID=MQRS,Description="MQRankSum < -12.5">
##FILTER=<ID=Mask,Description="Doesn't overlap a user-input mask">
##FILTER=<ID=PASS,Description="All filters passed">
##FILTER=<ID=QD,Description="QD<2.0">
##FILTER=<ID=RPRS,Description="ReadPosRankSum < -8.0">
##FORMAT=<ID=AD,Number=R,Type=Integer,Description="Allelic depths for the ref and alt alleles in the order listed">
##FORMAT=<ID=DP,Number=1,Type=Integer,Description="Approximate read depth (reads with MQ=255 or with bad mates are filtered)">
##FORMAT=<ID=GQ,Number=1,Type=Integer,Description="Genotype Quality">
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
##FORMAT=<ID=PGT,Number=1,Type=String,Description="Physical phasing haplotype information, describing how the alternate alleles are phased in relation to one another">
##FORMAT=<ID=PID,Number=1,Type=String,Description="Physical phasing ID information, where each unique ID within a given sample (but not across samples) connects records within a phasing group">
##FORMAT=<ID=PL,Number=G,Type=Integer,Description="Normalized, Phred-scaled likelihoods for genotypes as defined in the VCF specification">
##FORMAT=<ID=RGQ,Number=1,Type=Integer,Description="Unconditional reference genotype confidence, encoded as a phred quality -10*log10 p(genotype call is wrong)">
##FORMAT=<ID=SB,Number=4,Type=Integer,Description="Per-sample component statistics which comprise the Fisher's Exact Test to detect strand bias.">
##INFO=<ID=AC,Number=A,Type=Integer,Description="Allele count in genotypes, for each ALT allele, in the same order as listed">
##INFO=<ID=AF,Number=A,Type=Float,Description="Allele Frequency, for each ALT allele, in the same order as listed">
##INFO=<ID=AN,Number=1,Type=Integer,Description="Total number of alleles in called genotypes">
##INFO=<ID=BaseQRankSum,Number=1,Type=Float,Description="Z-score from Wilcoxon rank sum test of Alt Vs. Ref base qualities">
##INFO=<ID=ClippingRankSum,Number=1,Type=Float,Description="Z-score From Wilcoxon rank sum test of Alt vs. Ref number of hard clipped bases">
##INFO=<ID=DP,Number=1,Type=Integer,Description="Approximate read depth; some reads may have been filtered">
##INFO=<ID=DS,Number=0,Type=Flag,Description="Were any of the samples downsampled?">
##INFO=<ID=ExcessHet,Number=1,Type=Float,Description="Phred-scaled p-value for exact test of excess heterozygosity">
##INFO=<ID=FS,Number=1,Type=Float,Description="Phred-scaled p-value using Fisher's exact test to detect strand bias">
##INFO=<ID=HaplotypeScore,Number=1,Type=Float,Description="Consistency of the site with at most two segregating haplotypes">
##INFO=<ID=InbreedingCoeff,Number=1,Type=Float,Description="Inbreeding coefficient as estimated from the genotype likelihoods per-sample when compared against the Hardy-Weinberg expectation">
##INFO=<ID=MLEAC,Number=A,Type=Integer,Description="Maximum likelihood expectation (MLE) for the allele counts (not necessarily the same as the AC), for each ALT allele, in the same order as listed">
##INFO=<ID=MLEAF,Number=A,Type=Float,Description="Maximum likelihood expectation (MLE) for the allele frequency (not necessarily the same as the AF), for each ALT allele, in the same order as listed">
##INFO=<ID=MQ,Number=1,Type=Float,Description="RMS Mapping Quality">
##INFO=<ID=MQRankSum,Number=1,Type=Float,Description="Z-score From Wilcoxon rank sum test of Alt vs. Ref read mapping qualities">
##INFO=<ID=QD,Number=1,Type=Float,Description="Variant Confidence/Quality by Depth">
##INFO=<ID=RAW_MQ,Number=1,Type=Float,Description="Raw data for RMS Mapping Quality">
##INFO=<ID=ReadPosRankSum,Number=1,Type=Float,Description="Z-score from Wilcoxon rank sum test of Alt vs. Ref read position bias">
##INFO=<ID=SOR,Number=1,Type=Float,Description="Symmetric Odds Ratio of 2x2 contingency table to detect strand bias">
##reference=file:///Users/sian_bray/Dropbox/Yant/Cochlearia/02_Data/References/pseudohap_cochlearia_135M_3kb.fasta
##source=SelectVariants
#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  ALO_006 ALO_007 ALO_013 ALO_017 BEA_002 BEA_004 BEA_010 BRI_002 BRI_005 BRI_006 BRI_009 ELI_001 ELI_002 ELI_003 ELI_004 FRE_013 JON_001 KVA_002 KVA_003 KVA_009 KVA_010 LAB_004 LAB_300 LAB_400 LAB_500 LNL_001 LNL_002 LNL_003 LNL_008 MEL_001 MEL_002 NEN_001 NEN_003 NEN_200 NEN_300 ROT_004 ROT_006 ROT_007 ROT_013 RUZ_001 SAL_003 SAL_004 SAL_006 SKF_002 SKF_003 SKF_005 SKF_009 SKI_004 SKI_005 SKI_008 SKN_001 SKN_002 SKN_005 SKN_008 SPU_006 SPU_008 SPU_009 SPU_010 TET_002 TET_004 TET_006 TET_008 TRO_001 TRO_003 TRO_005 TRO_009 VAG_003 VAG_004 VAG_007 VAG_102 VEG_003 VEG_004 WOL_002 WOL_006 WOL_009 WOL_010
237 50041   .   G   T   4099.26 PASS    AC=35;AF=0.133;AN=264;BaseQRankSum=0.00;ClippingRankSum=0.00;DP=1228;ExcessHet=4.3562;FS=0.644;InbreedingCoeff=-0.0953;MQ=57.84;MQRankSum=0.00;QD=25.94;ReadPosRankSum=-1.570e-01;SOR=0.768 GT:AD:DP:GQ:PL  0/0/0/0:11,0:11:14:0,14,33,66,425   0/0/0/0:11,1:12:0:0,0,0,27,369  0/0/0/0:13,3:16:0:0,0,0,0,355   0/0/0/0:41,0:41:50:0,50,120,241,1800    0/0/1/1:5,8:13:1:293,15,0,1,170 0/0/0/1:5,3:8:0:104,0,0,10,191  0/0/0/0:8,0:8:10:0,10,24,48,272 0/1/1/1:1,8:9:11:318,33,11,0,26 ./././.:0,0 1/1/1/1:0,2:2:2:87,12,6,2,0 0/0/0/0:5,2:7:0:0,0,0,0,120 0/0/0/0:16,2:18:0:0,0,0,21,515  0/0/0/0:8,1:9:0:0,0,0,13,270    0/0/0/0:15,3:18:0:0,0,0,0,459   0/0/0/0:10,1:11:0:0,0,0,21,363  0/0/0/0/0/0:15,0:15:12:0,12,26,45,71,116,589    0/0/0/0/0/0:7,2:9:0:0,0,0,0,0,0,151 ./././.:0,0 0/0/0/0:14,0:14:14:0,14,33,66,495   0/0/0/0:1,0:1:1:0,1,3,6,42  0/0/0/0:9,0:9:10:0,10,24,48,360 0/0:21,2:23:14:0,14,726 0/0:14,2:16:11:0,11,452 0/0:14,2:16:0:0,0,467   0/0:23,9:32:0:0,0,527   0/0/0/0:10,0:10:12:0,12,30,60,359   0/0/0/0:11,1:12:0:0,0,0,27,404  0/0/0/0:13,0:13:16:0,16,39,78,526   0/0/0/0:20,3:23:0:0,0,0,2,655   1/1/1/1:0,4:4:5:155,24,12,5,0   0/0/0/0:2,3:5:0:0,0,0,0,8   0/0:23,2:25:0:0,0,798   0/0:12,2:14:0:0,0,381   0/0:14,0:14:36:0,36,540 0/0:24,0:24:63:0,63,945 0/0/0/0:8,1:9:0:0,0,0,12,245    0/0/0/0:27,0:27:32:0,32,78,157,1170 0/0/0/0:18,5:23:0:0,0,0,0,474   0/0/0/0:22,1:23:15:0,15,50,114,736  0/0:10,0:10:27:0,27,405 0/0:8,0:8:24:0,24,322   0/0:13,0:13:36:0,36,540 0/0:4,0:4:12:0,12,155   0/0/0/0/0/0:5,2:7:0:0,0,0,0,0,0,125 0/0/0/0/0/0:19,1:20:0:0,0,0,18,50,106,731   0/0/0/0/0/0:28,3:31:0:0,0,0,0,1,82,978  0/0/0/0/0/0:53,10:63:0:0,0,0,0,0,0,1843 0/0/0/0:1,1:2:0:0,0,0,0,4   1/1/1/1:0,22:22:27:917,132,66,27,0  ./././.:0,0 0/0/0/0:15,0:15:17:0,17,42,84,630   0/0/0/0:14,2:16:0:0,0,0,27,421  0/0/0/0:17,0:17:21:0,21,51,102,670  0/0/0/0:17,7:24:0:0,0,0,0,323   0/0:11,2:13:0:0,0,380
$\endgroup$
0

1 Answer 1

2
$\begingroup$

Answer from @user324810, converted from comment:

There is an alternative than doing it with Python unless it is absolutely necessary. If you want to merge multiple samples into one, use bcftools merge. If you want simply to concatenate multiple VCFs, use bcftools concatenate. Once you obtain the output file, you can load the dataframe and plot it as you wish.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.