How to fix RepeatMasker.lib.nsq missing error when running RepeatModelor?

Question

I have run RepeatModelor and I don't see the classified output file. I got this error.

Missing /home/mudith/.conda/envs/braker/share/RepeatMasker/Libraries/RepeatMasker.lib.nsq!
Please rerun the configure program in the RepeatModeler directory before running this script.

I think I have to run configure file. I found it. But it doesn't run when I just try perl ./configure Can someone please explain me how to run this? I'm new to this. Also I found this in github. But I don't understand.

cd $(dirname $(which RepeatMasker))/../share/RepeatMasker
# ./configure downloads required databases
echo -e "\n2\n$(dirname $(which rmblastn))\n\n5\n" > tmp && ./configure < tmp

I don't understand the third line.

Answer 1

What that code snippet is doing:

That snippet you found on github (where is that from exactly?) looks like it tries to run RepeatMasker's configure script, feeding it some options that are needed by RepeatModeler. The cryptic third line generates and dumps a block of text to a file named tmp:

2
path/to/bin/rmblastn

5

And then passes that in to RepeatMasker's own configure script as though it were entered on the keyboard. It's the equivalent of hitting enter to accept a default, entering "2" to select the RMBlast option, typing in the path to rmblastn, hitting enter to set RMBlast as the default, and then typing "5" to save and quit.

That all looks fine, but I think this might be a problem with your (and mine, see below) RepeatMasker install rather than RepeatModeler itself. These tools are new to me but I'm interesting in seeing this work too, so I gave a shot at troubleshooting it here.

The way I was able to get it working (note I assume you're working with conda packages here) was:

1. pushd $CONDA_PREFIX/share/RepeatMasker
2. ./util/buildRMLibFromEMBL.pl Libraries/RepeatMaskerLib.embl > Libraries/RepeatMasker.lib to "automatically create the old RepeatMasker.lib FASTA style database" as the script's comments put it. NOTE: you might want to first delete or move the old empty file rather than alter it because of how conda shares data between environments. If you update it in place (which > will still do), you'll alter the file for all conda environments using the package, and conda may complaint about unexpected file sizes when checking packages.
3. perl configure to do RepeatMasker's setup including some makeblastdb commands using RepeatMasker.lib. (Note this is RepeatMasker's configure script and not RepeatModeler's.) You can type in the options manually or you could probably use that code snippet you found.
4. popd and run RepeatModeler/RepeatClassifier and hopefully it finds the BLAST library files now

All the gritty detail:

I see a conda path in your logs. Are you using the bioconda packages named repeatmodeler and repeatmasker? It looks to me like the repeatmasker package in particular includes an empty RepeatMasker.lib file and that's the cause of the trouble. I don't see any way that RepeatMasker.lib is set up automatically via the configure script, and the BLAST database files like RepeatMasker.lib.nsq that RepeatModeler expects are built from that (empty) file.

The Please rerun the configure program in the RepeatModeler directory message is a red herring, I think, in this case because the $CONDA_PREFIX/share/RepeatMasker/Libraries/RepeatMasker.lib file does exist, just not the other files for a database it assumes should exist. If I instead run configure for RepeatMasker and update its script to not hide error messages, I see this coming from makeblastdb:

Building a new DB, current time: 10/26/2020 16:10:07
New DB name:   /home/jesse/miniconda3/envs/example-repeatmodeler/share/RepeatMasker/Libraries/RepeatMasker.lib
New DB title:  /home/jesse/miniconda3/envs/example-repeatmodeler/share/RepeatMasker/Libraries/RepeatMasker.lib
Sequence type: Nucleotide
Keep MBits: T
Maximum file size: 1000000000B
BLAST options error: File /home/jesse/miniconda3/envs/example-repeatmodeler/share/RepeatMasker/Libraries/RepeatMasker.lib is empty

And it's true, that's an empty file provided by the conda package (the metadata in $CONDA_PREFIX/conda-meta/repeatmasker-4.1.0-pl526_0.json shows size_in_bytes: 0 for that one so that's apparently "correct" for the package.)

I saw a reference to that filename in a buildRMLibFromEMBL.pl script in the RepeatMasker share directory, and running that produces a FASTA file:

./util/buildRMLibFromEMBL.pl Libraries/RepeatMaskerLib.embl > Libraries/RepeatMasker.lib

Then ./configure sees a non-empty .lib file and builds the database, and RepeatClassifier finally sees the resulting database files.

But why is this needed?

Shouldn't RepeatMasker come with a non-empty FASTA file if it's needed by its own configure script? I can see bioconda's build.sh doesn't do anything in particular to handle that file, but, maybe it should? This is probably where somebody who knows these tools better can chime in.)