0
$\begingroup$

I'm trying to use vcf-merge to combine 2 exome capture vcf files (~250K and ~330K in size) before trying it on all 96 samples. I'd appreciate any advice on the best way to do that! I've detailed what I've tried below. My issue seems to be with using tabix to convert the files to .tbi format.

Step 1: BGZIP

So far, I've zipped the files without issue:

bgzip sample1.vcf
bgzip sample2.vcf

which produces:

sample1.vcf.gz
sample2.vcf.gz

Step 2: TABIX

When I try to use this command:

tabix -h -p vcf sample1.vcf.gz

the stderr is:

Region 536999277..536999278 cannot be stored in a tbi index. Try using a csi index with min_shift = 14, n_lvls >= 6. tbx_index_build failed: sample1.vcf.gz

Using the -C option which works:

tabix -C -h -p vcf sample1.vcf.gz
tabix -C -h -p vcf sample2.vcf.gz

which produces: sample1.vcf.gz.csi sample2.vcf.gz.csi

Step 3: VCF-MERGE

When I use this command:

vcf-merge sample1.vcf.gz.csi sample2.vcf.gz.csi > out.vcf.gz

the merge fails, and I get this stderr:

Broken VCF header, no column names? at /usr/share/perl5/Vcf.pm line 172, <ANONIO> line 1. Vcf::throw(Vcf4_2=HASH(0x5645a760bc38), "Broken VCF header, no column names?") called at /usr/share/perl5/Vcf.pm line 867 VcfReader::_read_column_names(Vcf4_2=HASH(0x5645a760bc38)) called at /usr/share/perl5/Vcf.pm line 602 VcfReader::parse_header(Vcf4_2=HASH(0x5645a760bc38)) called at /usr/bin/vcf-merge line 183 main::init_cols(HASH(0x5645a761f438), Vcf4_2=HASH(0x5645a760b248)) called at /usr/bin/vcf-merge line 279 main::merge_vcf_files(HASH(0x5645a761f438)) called at /usr/bin/vcf-merge line 12

If I exclude the tabix files, the merge still fails and the stderr says:

The column names not tab-separated? Could not load .tbi index of sample1.vcf.gz. The command exited with an error. Is the file tabix indexed?

Any suggestions on how I can correctly merge these files?

Improve this question
$\endgroup$
3
  • $\begingroup$ Cross-posted on Biostars: biostars.org/p/437957 $\endgroup$
    – Ram RS
    May 13, 2020 at 23:28
  • $\begingroup$ Would it be possible to share the files with us? Do you really have a chromosome that's 536999278 bases long? What species are you working on? $\endgroup$
    – terdon ♦
    May 14, 2020 at 8:55
  • $\begingroup$ Hi, I'm working with conifers which have very long chromosomes. As discusses below, the tbi cutoff is ~536,870,912 bp, so csi is needed. $\endgroup$
    – Brian
    May 14, 2020 at 23:55

1 Answer 1

Reset to default
2
$\begingroup$

You should go with the CSI index as it seems like the BAI indexes cannot handle chromosomes longer than 2^29.

Your problem is that you're trying to merge the index files using vcf-merge. You should merge the VCF files, not the .csi index files.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Do you have a reference for this limitation? Is it mentioned in the docs somewhere? $\endgroup$
    – terdon ♦
    May 14, 2020 at 8:55
  • 1
    $\begingroup$ Yeah, I found it in a GitHub issue for the mosdepth/samtools repos: github.com/brentp/mosdepth/issues/41 github.com/samtools/samtools/issues/241 $\endgroup$
    – Ram RS
    May 14, 2020 at 17:39
  • 2
    $\begingroup$ This has come up on the mailing lists and forums often enough over the years to be part of the common lore. It's spelt out in the BAI spec in SAMv1.pdf §5.1.1 and will soon be mentioned in the samtools index manual page (PR #1248). $\endgroup$
    – John Marshall
    May 14, 2020 at 18:14
  • $\begingroup$ Thanks for the advice, and sorry for the cross-post. You're right, CSI must be used if indexing is necessary since all of my chromosomes are larger than 2^29. $\endgroup$
    – Brian
    May 14, 2020 at 23:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.