I have [a script][1] that extracts the set of unique kmer pairs from a kmer dump file (& then using them downstream to guess ploidy).

The script seems to work correctly (it's reporting a meaningful set of kmer pairs), but since we have added a parallelization it does not quit when the job is done for really big data sets. The problem is that I never managed to reproduce the behavior with a small reproducible example and it takes around 400G of memory and a day of computation to get to the problem. My imagination is quite limited, I have no idea how should I debug such a problem.

I don't really get it; the script actually prints the last line 

>*_families_2.tsv and *_coverages_2.tsv files saved.

, but then it runs further (& forever). I thought it could be related to longer time to free the memory as discussed [here][2], but it does not seem like the case because it does not quit even after a really long time (I tried to just leave it for a day or so and when i kill it the memory is freed in like 20s).

Does anybody an idea where the problem could be? Or how to figure out?

The reported problem on [GitHub][3].


  [1]: https://github.com/tbenavi1/smudgeplot/blob/master/hetkmers.py
  [2]: https://stackoverflow.com/questions/2412715/python-program-doesnt-quit-when-finished
  [3]: https://github.com/tbenavi1/smudgeplot/issues/18