# Remove repeated ALT allele from the REF field of a vcf-like file

I have 200 .txt files on a Linux system. In each file, I have the reference allele in the REF column and INDELs in the ALT column. Let's say I have A in the REF column and AC in the ALT. I want to remove the A from the ALT column because that is a repetition of the A reference allele. For example, if I have C in the REF column and CCAT in the ALT column, I only want CAT to remain in the ALT column.

This is what my files look like

CHROM   REF ALT
chr1    C   CT
chr1    C   CT
chr1    C   CA
chr1    C   CT


Solution from @Terdon

This is my original file

#CHROM  POS ID  REF ALT
chr4    5708160 .   G   GATAAC
chr4    30280292    .   C   CACGTAT
chr6    166253422   .   C   CCTTTCT
chr11   21931340    .   A   AACACACGC


This is what happened after running your kindly suggestion

[fi1d18@cyan01 indel]$$awk -F"\t" -vOFS="\t" '{ if(substr(5,1,1) == 4){5=substr($$5,4); }}1' indels.vcf > out.vcf

#CHROM  POS ID  REF ALT
chr4    5708160 .   G   AAC
chr4    30280292    .   C   GTAT
chr6    166253422   .   C   TTCT


I think more than one allele being removed from ALT column, is not it?

I am making a couple of assumptions here:

1. the delimiter is tab
2. the reference allele is always present in the ALT column
3. the reference allele in the ALT column is always in the first position

With those in mind, you can do this using awk as follows:

awk 'BEGIN{FS="\t";OFS="\t"}{$3=substr($3,2)}{print $0}'  Update 1: Using the new VCF style file that @Angel provided, here's how I would do it. This is not significantly different from what @terdon proposed below. The code below will work even if the reference allele is >1 nt. $ cat test.vcf
#CHROM  POS     ID      REF     ALT
chr4    5708160 .       G       GATAAC
chr4    30280292        .       C       CACGTAT
chr6    166253422       .       C       CCTTTCT
chr11   21931340        .       A       AACACACGC

$$awk 'BEGIN{FS="\t";OFS="\t"}{l=length(4); if (4==substr(5,1,l)) {5=substr(5,l+1); print 0} else {print$$0}}' test.vcf
#CHROM  POS     ID      REF     ALT
chr4    5708160 .       G       ATAAC
chr4    30280292        .       C       ACGTAT
chr6    166253422       .       C       CTTTCT
chr11   21931340        .       A       ACACACGC

• Ooh, of course it's different to mine! You are being clever and looking for the entire ref allele instead of assuming it is one residue. Nice! You might want to add an example with a multi-nt ref allele so people can see that your solution deals with those. Dec 4 '19 at 17:55

Another awk approach. This should work on a normal VCF file (it will print the headers) as well as your example and only changes the ALT field if its first character is the same as the 2nd field:

$$awk -F"\t" -vOFS="\t" '{ if(substr(5,1,1) == 4){5=substr($$5,2); }}1' ff.vcf
#CHROM  POS ID  REF ALT
chr4    5708160 .   G   ATAAC
chr4    30280292    .   C   ACGTAT
chr6    166253422   .   C   CTTTCT
chr11   21931340    .   A   ACACACGC


To run this on all of your files, assuming they are the only files in the current directory with a .txt extension, you can do:

for file in *txt; do
awk -F"\t" -vOFS="\t" '{ if(substr($$5,1,1) ==$$4){$$5=substr($$5,2); }}1' "$$file" > "fixed.$$file"
done


This will loop over all *.txt files in the current directory. For each foo.txt, it will create fixed.foo.txt with the modified data.