# Installing HiC-Pro issues

I am setting up a HiChIPseq pipeline, but installing software necessary for the HiC side of things has been unbecoming.

Both my PI and I have tried to install Hi-C Pro onto our Mac Pro that we use as our compute server (though in a couple of weeks we hope to migrate to an EPYC 64 Core computer server of our own!).

There are three approaches that I have taken to install Hi-C Pro.

To install from source, you need to supply the paths to its dependencies. Believing we did a good job, we run it and find problems accessing bx-python.

No matter how many times we update pip and try to install bx-python and specify that we want it installed for python2.

1. From singularity

This is the one I tried. From what I understand this creates a VM with the necessary dependencies installed, and allows you to use the program without building. However, it appears to abort when conda tries to install numpy.

+ conda install -y -c anaconda numpy=1.16.3
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): / Killed
ABORT: Aborting with RETVAL=255
Cleaning up...


I have seen the failed solving environment bug many times, but have no idea how to solve it. I have tried updating conda, but that doesn't solve it.

Running conda.sh give me this:

Solving environment: failed

ResolvePackageNotFound:
<Lots of packages...>


Sorry if this is sloppy.

• Is the conda environment running under python2 or 3? – ATpoint Sep 14 at 15:57
• @ATpoint python2 – Jeff Sep 14 at 16:24

I could install HiC-Pro using conda when creating a separate environment (which I always do for side-projects), and then pull the conda build of HiC-Pro:

conda create --name HiCPro python=2.7
conda activate HiCPro
conda install -c davebx hicpro


Not sure what conda.sh is that you describe, sometimes tricky to get random code from GitHub running. Give it a go with a standard conda install as I described.

If you're reading this, I can only assume that's because you had trouble installing HiC-Pro too. I was finally able to install from source and run the software, so here is how I did it, giving detail to the error that I was hit with and the clues I used to solve the puzzles and get through installing this program:

From Source To install from source, you need to supply the paths to its dependencies. Believing we did a good job, we run it and find problems accessing bx-python.

No matter how many times we update pip and try to install bx-python and specify that we want it installed for p

# Setup

I downloaded from source once again using git clone.

The I cated the config file to find the programs I needed to find the path of, which I did using which. I ensured that I was using python2.

I also used pip2 to install pysam, numpy, and scipy. I needed to use sudo to install pysam, though. I also installed something called bx-python.

# Make Configure

Here is the error I get when I run make configure

Checking Python libraries ...
Traceback (most recent call last):
File "../scripts/install/check_pythonlib.py", line 24, in <module>
import bx.intervals
ModuleNotFoundError: No module named 'bx'

Traceback (most recent call last):
File "../scripts/install/check_pythonlib.py", line 28, in <module>
raise ImportError('Error - bx-python cannot be imported')
ImportError: Error - bx-python cannot be imported
Can not proceed without the required Python libraries, please install them and re-run
make[1]: *** [configure] Error 1
make: *** [configure] Error 2


Looking at this error for the fifth time, I realize that the first part, and not the last part, is the important one. It's yelling at me that I don't have the bx module installed. It also gives me the path to the file that threw the exception, so I read it and saw that this was what was being run:

## bx
try:
import bx.intervals
if vcmp(bx.__version__, '0.5.0') == -1:
raise ValueError('bx-python '+bx.__version__+' detected. Version >= 0.5.0 required')
except ImportError:
raise ImportError('Error - bx-python cannot be imported')


I notice that there's nothing fancy, the thing just needs to be imported, and its actual name is bx. Wonderful, I pip2 it. But... I hit another error in pip:

pyrsistent requires Python '>=3.5' but the running Python is 2.7.16


Apparently, pyrsistent dropped support for python2. Wonderful. I get around it by:

sudo pip2 install awsebcli pyrsistent==0.16.0


In the middle of this installation, Im reading the output and see:

bx-python 0.8.9 has requirement six>=1.13.0, but you'll have six 1.11.0 which is incompatible.

So:

sudo pip2 install six==1.13.0

At this point, I, without any more problems, sudo pip2 install bx.

I run make configure again, and I run into...

Checking Python libraries ...
Traceback (most recent call last):
File "../scripts/install/check_pythonlib.py", line 24, in <module>
import bx.intervals
ModuleNotFoundError: No module named 'bx'

Traceback (most recent call last):
File "../scripts/install/check_pythonlib.py", line 28, in <module>
raise ImportError('Error - bx-python cannot be imported')
ImportError: Error - bx-python cannot be imported
Can not proceed without the required Python libraries, please install them and re-run
make[1]: *** [configure] Error 1
make: *** [configure] Error 2


huh? Didn't I fix this?? Something seems fishy. For whatever reason, my mind then wanders to the Makefile to see if anything there pops out, cause... what else would be calling that python file? In the main Makefile I see:

# Leftovers

I installed it, but how about when I actually run it?

Error: The 'ice' command is not in your path. Please check where the 'iced' python package has been installed and update your PATH !

Ok:

$sudo pip2 install iced Requirement already satisfied: iced in /Users/bosslab/Library/Python/2.7/lib/python/site-packages (0.4.2)  ...so, I set up a test to see whether it actually is installed: $ python2
Python 2.7.16 (default, Nov  9 2019, 05:55:08)
[GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.32.4) (-macos10.15-objc-s on darwin
>>> iimport iced
>>> quit()


\$ python3
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
>>> ipomport iced
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'iced'
>>> quit()


Aha! Somewhere (i don't know where though) python3 is being called. So I just installed that module for python3 with sudo pip3 install iced.

Then I had some trouble with pointing to the right bowtie index (just give the directory where the index exists) and then I hit my next error dealing with HiC-Pro itself:

Pairing of R1 and R2 tags ...
Logs: logs/dixon_2M/mergeSAM.log
make: *** [bowtie_pairing] Error 1


Checking the log file named in the error message:

/usr/local/bin/python /Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4/scripts/mergeSAM. py -q 0 -t -v -f bowtie_results/bwt2/dixon_2M/SRR400264_00_R1_hg19.bwt2merged.bam -r bowtie_re sults/bwt2/dixon_2M/SRR400264_00_R2_hg19.bwt2merged.bam -o bowtie_results/bwt2/dixon_2M/SRR400 264_00_hg19.bwt2pairs.bam

Missing parentheses? Oh my god, it's using python3 for some reason again. I grep the installation directory for a file that calls merge_sam.py, and find config-system.txt. I go in there and just need to change the python path from /usr/bin/local/ where python3 is installed to /usr/bin/ where python2 is installed.