0
$\begingroup$

I am conducting phylogeny analyses using BEAST 2 as software tool. It is said that it can be accelerated by BEAGLE-library. I have a personal laptop with Windows & a cluster with CentOS.

I installed both BEAST 2 & BEAGLE onto them. This gives me a boost by 3.6 times. It was said too that using OpenCL with BEAGLE gives me more speed-up. Nonetheless, my calculations takes from twelve hours to 3 days so I got really interested in this. Installing it on Windows were very simple. When I ran beast -beagle_info, I saw:

                        BEAST v2.6.6, 2002-2021
             Bayesian Evolutionary Analysis Sampling Trees
                       Designed and developed by
 Remco Bouckaert, Alexei J. Drummond, Andrew Rambaut & Marc A. Suchard

                   Centre for Computational Evolution
                         University of Auckland
                       [email protected]
                        [email protected]

                   Institute of Evolutionary Biology
                        University of Edinburgh
                           [email protected]

                    David Geffen School of Medicine
                 University of California, Los Angeles
                           [email protected]

                      Downloads, Help & Resources:
                           http://beast2.org/

  Source code distributed under the GNU Lesser General Public License:
                   http://github.com/CompEvol/beast2

                           BEAST developers:
   Alex Alekseyenko, Trevor Bedford, Erik Bloomquist, Joseph Heled,
 Sebastian Hoehna, Denise Kuehnert, Philippe Lemey, Wai Lok Sibon Li,
Gerton Lunter, Sidney Markowitz, Vladimir Minin, Michael Defoin Platel,
          Oliver Pybus, Tim Vaughan, Chieh-Hsi Wu, Walter Xie

                               Thanks to:
          Roald Forsberg, Beth Shapiro and Korbinian Strimmer


--- BEAGLE RESOURCES ---

0 : CPU
    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_SSE VECTOR_NONE THREADING_NONE PROCESSOR_CPU FRAMEWORK_CPU

1 : NVIDIA GeForce 940MX
    Global memory (MB): 2048
    Clock speed (Ghz): 1.19
    Number of cores: 384
    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH COMPUTATION_ASYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_NONE THREADING_NONE PROCESSOR_GPU FRAMEWORK_CUDA

2 : Intel(R) HD Graphics 620 (OpenCL 2.1 )
    Global memory (MB): 3219
    Clock speed (Ghz): 1.00
    Number of compute units: 24
    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH COMPUTATION_ASYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_NONE THREADING_NONE PROCESSOR_GPU FRAMEWORK_OPENCL

3 : Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz (OpenCL 2.1 (Build 10))
    Global memory (MB): 8067
    Clock speed (Ghz): 2.50
    Number of compute units: 4
    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH COMPUTATION_ASYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_NONE THREADING_NONE PROCESSOR_CPU FRAMEWORK_OPENCL

However, it still uses only resourse #0, even through forced beast -beagle_order 3 -threads 1 filename.xml. Although, in comparison to Linux case, it was more successful.

I have downloaded intel_sdk_for_opencl_applications_2020.3.494.tar.gz from here. Then I unzipped it & run install.sh. Installation was successful. After that, I have decided to go to build folder in beagle-lib & run cmake -DCMAKE_INSTALL_PREFIX:PATH=$HOME .. in order to go on to run make install but I get the next message:

-- JAVA_HOME=
-- JNI_INCLUDE_DIRS=/usr/lib/jvm/java/include;/usr/lib/jvm/java/include/linux;/usr/lib/jvm/java/include
-- JNI_LIBRARIES=/usr/lib/jvm/java/lib/libjawt.so;/usr/lib/jvm/java/lib/server/libjvm.so
-- Not using libtools for plugins
-- Could NOT find OpenCL (missing: OpenCL_LIBRARY OpenCL_INCLUDE_DIR)
CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)
-- Configuring done
-- Generating done
-- Build files have been written to: /export/home/rinkman/beagle-lib/build

It was solved through manual specification of directories cmake -DOpenCL_INCLUDE_DIR=~/intel/system_studio_2020/opencl/SDK/include/ -DOpenCL_LIBRARY=~/intel/system_studio_2020/opencl/SDK/lib64/libOpenCL.so.1.2 -DCMAKE_INSTALL_PREFIX:PATH=$HOME ... The compiling was successful after that. make test was executed without any problems. I detected the file libhmsbeagle-opencl.so.40.0.0 in ~/lib folder. But beast -beafle_info did not show OpenCL. As there are older versions of BEAST & BEAGLE on the root, I tried to change very roughly startup script to run only my local BEAGLE but nothing changed:

#!/bin/sh

if [ -z "$BEAST" ]; then
    ## resolve links - $0 may be a link to application
    PRG="$0"

    # need this for relative symlinks
    while [ -h "$PRG" ] ; do
        ls=`ls -ld "$PRG"`
        link=`expr "$ls" : '.*-> \(.*\)$'`
        if expr "$link" : '/.*' > /dev/null; then
        PRG="$link"
        else
        PRG="`dirname "$PRG"`/$link"
        fi
    done

    # make it fully qualified
    saveddir=`pwd`
    BEAST0=`dirname "$PRG"`/..
    BEAST=`cd "$BEAST0" && pwd`
    cd "$saveddir"
fi

BEAST_LIB="$BEAST/lib"

if [ -z "$JAVA_HOME" ]; then
  JAVA=java
else
  JAVA="$JAVA_HOME"/bin/java
fi


# use BEAGLE_LIB if the BEAGLE library is not in a standard location
if [ -n "$BEAGLE_LIB" ]; then
    if [ -n "$BEAST_EXTRA_LIBS" ]; then
        BEAST_EXTRA_LIBS="$BEAST_EXTRA_LIBS:$BEAGLE_LIB"
    else
        BEAST_EXTRA_LIBS="$BEAGLE_LIB"
    fi
fi

# Explicitly add /usr/local/lib to library search path to ensure 
# beast continues to find beagle when installed here.  (This is
# necessary due to known problems with certain JREs.)
if [ -z "$LD_LIBRARY_PATH" ]; then
    export LD_LIBRARY_PATH=/usr/local/lib
else
    export LD_LIBRARY_PATH="$LD_LIBRARY_PATH":/usr/local/lib
fi

BEAST_EXTRA_LIBS="$HOME/lib/"

# use BEAST_EXTRA_LIBS variable to load BEAGLE and other libraries from non-default locations 
# this assumes that the library path contains all these libraries (or are set through LD_LIBRARY_PATH)
#if [ -n "$BEAST_EXTRA_LIBS" ]; then 
#  "$JAVA" -Dlauncher.wait.for.exit=true -Xms256m -Xmx1g -Djava.library.path="$BEAST_EXTRA_LIBS" -Duser.language=en -cp "$BEAST_LIB/launcher.jar" beast.app.beastapp.BeastLauncher $*
#else   
#  "$JAVA" -Dlauncher.wait.for.exit=true -Xms256m -Xmx1g -Duser.language=en -cp "$BEAST_LIB/launcher.jar" beast.app.beastapp.BeastLauncher $*
#fi
"$JAVA" -Dlauncher.wait.for.exit=true -Xms256m -Xmx1g -Djava.library.path="$BEAST_EXTRA_LIBS" -Duser.language=en -cp "$BEAST_LIB/launcher.jar" beast.app.beastapp.BeastLauncher $*

Please, could anyone help with this?

P.S. I am a noviciate in unix-systems.

$\endgroup$
2
  • $\begingroup$ @M__ I installed BEAGLE successfully per se & it does work but I would like to run it with OpenCL. $\endgroup$
    – Vovin
    Mar 17, 2022 at 17:55
  • $\begingroup$ @M__ Do you mean beast-users? $\endgroup$
    – Vovin
    Mar 17, 2022 at 20:53

1 Answer 1

1
$\begingroup$

I better explain this style of parameter rich MCMC takes ages and the issues are how speed this up.


Having given this some thought my suggestion is to install miniconda or anaconda and then install the port to OpenCL, then install beast and beagle.

Anaconda is here: https://www.anaconda.com/products/individual

Then install openCL via conda:

 conda create -n beasty python=3.9
 conda activate beasty
 conda install -c conda-forge pyopencl 
 conda install -c bioconda beagle
 conda install -c bioconda beast2

My idea was you'd access OpenCL via python described here. This wouldn't work ... its complicated.

Traditionally OpenCL and Beagle didn't work ... on an Intel Mac

2 : Intel(R) HD Graphics 620 (OpenCL 2.1 )

However, if this is your chip set - I wouldn't give up. In any other scenario I wouldn't pursue this - personally, below.

One alternative is NVIDA e.g. Cloud computing used via CUDA and Beagle. CUDA will easily install and Beagle stacks straight on top.

Any heavy duty MCMC is very amenable to traditional CPU parallelisation - if you think at it - it would be and thats why GPU - CUDA will work. Personally, looking at your resources I would parallelise under you cluster using a MINIMUM of 8-cores per calculation outside CUDA/OpenCL. If that has not already been done, that is the solution in context in my opinion. Unless you're a Google developer few will have access to a NVIDA cluster in any case sufficient CPU parallelisation should be able to achieve the runtime close to that of a good GPU (its gonna be a lot of CPUs though).

Conclusion Honestly, my personal advise is that phylogenetics sump clusters no matter what the calculation is being used and that has always been the case. The best solution is to max out on available cores. When eukaryote genome guys complain about 1000 hour runtimes (obviously thats the total CPU parallelised), thats a routine calculation in phylogenetics. Buying in NVIDA doesn't necessarily solve this, because whilst MCMC is massively accelerated for maximum likelihood isn't accelerated to that extent by comparison.

Extra information In case the OP is not aware, the enormous advantage of Beast2 is the ability to resume an MCMC at the point it ended (or some checkpoint therein), you don't need to run the whole thing again. You should seek to exploit that, so a calculation started on a laptop can be finished on a cluster. I can't state sufficiently how powerful that is, it stands to reason because the random number stream is declared and every sample thereafter. Keeping all calculations is standard practice in any case, but especially with a Beast2 run and don't every discard it (hard disk is cheap right?). You might find you have to return to it and its a whole lot better than needing to do the whole thing again.

Its time to move on.

$\endgroup$
8
  • $\begingroup$ I have gotten Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abor\ failed UnsatisfiableError: The following specifications were found to be incompatible with each other: Output in format: Requested package -> Available versions Package libstdcxx-ng conflicts for: python=3.10 -> libffi[version='>=3.4.2,<3.5.0a0'] -> libstdcxx-ng[version='>=7.3.0|>=7.5.0'] $\endgroup$
    – Vovin
    Mar 28, 2022 at 16:00
  • $\begingroup$ beast2 -> beagle-lib -> libstdcxx-ng[version='>=10.3.0|>=9.3.0|>=7.5.0|>=7.3.0']The following specifications were found to be incompatible with your system: - feature:/linux-64::__glibc==2.17=0 - python=3.10 -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17'] Your installed version is: 2.17 What does this mean? Which packages should I delete? Of what is 2.17 a version? Please could you help? $\endgroup$
    – Vovin
    Mar 28, 2022 at 16:02
  • $\begingroup$ I get the same conflict with a spanking new environment :( $\endgroup$
    – Vovin
    Mar 29, 2022 at 9:39
  • $\begingroup$ beast2 can be installed with beagle & opencl only through adding conda-forge & bioconda channels, & creating new environment from scratch as conda create -n env_name opencl beagle beast2. Nonetheless, conda run beast -beagle_info shows only 1 resource: CPU without OpenCL $\endgroup$
    – Vovin
    Mar 29, 2022 at 10:45
  • $\begingroup$ There is a way to save states in BEAST 1. Should it be left here as a comment or as the answer correction? $\endgroup$
    – Vovin
    Jun 20, 2022 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.