I am running bcftools merge
to create a single .vcf.gz
file out of a collection of chromosomal .vcf.gz
files from the 1000g project.
The merge process has been running for a couple of days, and I want to estimate how much longer it will take. The file is a growing .vcf.gz
that is being bgzip -c
compressed from a Linux pipe:
bcftools merge [...] | some processing and filtering | bgzip -c > final.file.vcf.gz
How can I know at which point is the file currently, to predict how much longer it will take to complete the merge?
Is there a way to do something like:
tail --with-some-magic final.file.vcf.gz | bgzip -d -c
So that I can read at which chromosome is currently on?
Is there a way to guesstimate the final size of the merge file and compare from the sum of sizes of the original files? E.g. the sum of compressed sizes originally (before filtering and munging) is 20G:
find $PWD -name "ALL.chr*.vcf.gz" | xargs ls -l | awk '{print $5}' | csvtk summary -H -f 1:sum | numfmt --to-si
This gives 20G, but the merge file is at 27G and still running.
Any ideas?