I have a script that extracts the set of unique kmer pairs from a kmer dump file (& then using them downstream to guess ploidy).
The script seems to work correctly (it's reporting a meaningful set of kmer pairs), but since we have added a parallelization it does not quit when the job is done for really big data sets. The problem is that I never managed to reproduce the behavior with a small reproducible example and it takes around 400G of memory and a day of computation to get to the problem. My imagination is quite limited, I have no idea how should I debug such a problem.
I don't really get it; the script actually prints the last line
*_families_2.tsv and *_coverages_2.tsv files saved.
, but then it runs further (& forever). I thought it could be related to longer time to free the memory as discussed here, but it does not seem like the case because it does not quit even after a really long time (I tried to just leave it for a day or so and when i kill it the memory is freed in like 20s).
Does anybody an idea where the problem could be? Or how to figure out?
The reported problem on GitHub.