How to concatenate "by chromosome"-VCFs?

Question

I have a several VCFs which are VCF which only contain information by chromosome. That is, there's a chromosome 1 VCF (with only chr1), a chromosome 2 VCF (with only chr2), etc.

I checked to make sure that these VCFs were valid via VCFtools, i.e.

$ vcf-validator chr1.vcf

which works---these are valid VCFs I was given.

Now, I would like to combine these VCFs into one VCF.

I naïvely tried the following cat operation:

$ cat chr1.vcf chr2.vcf chr3.vcf ... chrX.vcf > total_chroms.vcf

This doesn't work properly though

$ vcf-validator total_chroms.vcf
The header tag 'contig' not present for CHROM=chr1. (Not required but highly recommended.)
Could not parse the line, wrong number of columns: [##fileformat=VCFv4.2\n]
 at /path/vcftools-0.1.14/perl/Vcf.pm line 172, <__ANONIO__> line 191016.
	    Vcf::throw('Vcf4_2=HASH(0x1ae7208)', 'Could not parse the line, wrong number of columns: [##filefor...') called at  /path/vcftools-0.1.14/perl/Vcf.pm line 335
	    Vcf::next_data_array('Vcf4_2=HASH(0x1ae7208)') called at  /path/vcftools-0.1.14/perl/Vcf.pm line 3457
	    Vcf4_1::next_data_array('Vcf4_2=HASH(0x1ae7208)') called at  /path/vcftools-0.1.14/perl/Vcf.pm line 2574
	    VcfReader::run_validation('Vcf4_2=HASH(0x1ae7208)') called at  /path/vcftools-0.1.14//bin/vcf-validator line 60
	    main::do_validation('HASH(0x16ada68)') called at  /path/vcftools-0.1.14//bin/vcf-validator line 14
$

What tools are available to merge these VCF's together into a total VCF?

Answer 1

I would recommend bcftools concat. You can't just cat them together because each file has a header section. The bcftools command will handle all that for you. Each vcf file must be sorted prior to calling concat

bcftools concat -o total_chroms.vcf chr1.vcf chr2.vcf chr3.vcf ... chrX.vcf

Answer 2

The best tool for the job is probably bcftools as suggested by Bioathlete, but you can also do this manually. You just need to collect all header lines from all vcf files, remove any duplicates, and then print all the headers + the actual data into the new one:

grep '^##' chr*vcf | sort | uniq > all.vcf
grep -m1 '^#CHR' chr1.vcf >> all.vcf  ## Get the chr header line 
grep -v '^#' chr*vcf >> all.vcf

Answer 3

use picard GatherVcf: http://broadinstitute.github.io/picard/command-line-overview.html#GatherVcfs

Gathers multiple VCF files from a scatter operation into a single VCF file. Input files must be supplied in genomic order and must not have events at overlapping positions.