2
$\begingroup$

I am using the tool seqret from emboss to transform an annotation file in GFF3 format and a FASTA file into an EMBL file because Wormbase does not supply an EMBL file with annotation and sequence together. I am running it on Mac OS X. I got the gff3 and fasta files from Wormbase. I installed emboss using conda from the bioconda channel. I use the following command:

seqret -sequence c_elegans.PRJNA13758.WS263.genomic.fa -feature -fformat gff -fopenfile c_elegans.PRJNA13758.WS263.annotations.gff3 -osformat embl -auto -outseq c_elegans.PRJNA13758.WS263.embl

It fails with the following error:

/Users/me/anaconda/bin/seqret`: line 7: 54774 Killed: 9               _seqret "$@"

/Users/me/anaconda/bin/seqret is a short file that indeed contains _seqret "$@" on line 7:

#!/bin/sh
BIN_DIR=$(dirname $(which seqret))
export EMBOSS_ACDROOT=$BIN_DIR/../share/EMBOSS/acd/
export EMBOSS_DATA=$BIN_DIR/../share/EMBOSS/data/
export PLPLOT_LIB=$BIN_DIR/../share/EMBOSS/
_seqret "$@"

I tried to erase it but that did not help.

I also tried EMBLmyGFF3 in a Python 2 environment:

EMBLmyGFF3 c_elegans.PRJNA13758.WS263.annotations.gff3 c_elegans.PRJNA13758.WS263.genomic.fa --topology linear --molecule_type 'genomic DNA' --transl_table 1 --species 'Caenorhabditis elegans' --locus_tag CELE --project_id PRJNA13758 -o c_elegans.PRJNA13758.WS263.annotations.embl

but I just had this message on display for hours:

#############################################################################
# NBIS 2016 - Sweden                                                        #
# Authors: Martin Norling, Niclas Jareborg, Jacques Dainat                  #
# Please visit https://github.com/NBISweden/EMBLmyGFF3 for more information #
#############################################################################

Update 11th March:

I started EMBLmyGFF3 yesterday at 5 pm and this morning it started outputting messages such as:

08:22:08 ERROR feature: >>inverted_repeat<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:22:08 ERROR feature: >>assembly_component<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:22:08 WARNING feature: Unknown qualifier 'genbank' - skipped
08:22:08 ERROR feature: >>tandem_duplication<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:22:08 WARNING feature: Unknown qualifier 'public_name' - skipped
08:22:08 WARNING feature: Unknown qualifier 'polymorphism' - skipped
08:22:08 WARNING feature: Unknown qualifier 'other_name' - skipped
08:22:08 WARNING feature: Unknown qualifier 'consequence' - skipped
08:22:08 WARNING feature: Unknown qualifier 'variation' - skipped
08:22:08 ERROR feature: >>biological_region<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:22:08 WARNING feature: Unknown qualifier 'balancer_type' - skipped
08:22:08 WARNING feature: Unknown qualifier 'balancer' - skipped
08:22:08 ERROR feature: >>protein_match<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:22:08 ERROR feature: >>transcript_region<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:22:08 ERROR feature: >>transcribed_fragment<< is not a valid EMBL feature type. You can ignore this message if you don't need it.

...

Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:45:36 ERROR feature: >>piRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:46:00 ERROR feature: >>pseudogenic_tRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:47:59 ERROR feature: >>antisense_RNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:47:59 ERROR feature: >>lincRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:48:03 ERROR feature: >>pre_miRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:48:03 ERROR feature: >>miRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
08:50:37 ERROR feature: >>snRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
09:04:00 ERROR feature: >>miRNA_primary_transcript<< is not a valid EMBL feature type. You can ignore this message if you don't need it.

For some of these features, it seems OK to not report them. But for others, such as piRNA, I need to find a solution. C. elegans has got 1000s of piRNAs, it would be nice to have them in the EMBL file. I need to find a list of valid EMBL feature types.

Update + ~ 24h computation:

...

Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
12:21:19 ERROR feature: >>scRNA<< is not a valid EMBL feature type. You can ignore this message if you don't need it.
Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.
Conversion done

My EMBL file is only 1.19 GB compared to the GFF3 3.34 GB, so I probably lost a lot of annotation that is not supported by EMBL.

$\endgroup$
  • $\begingroup$ Please always mention your operating system and hardware. I am guessing that you are running this on a mac laptop, but it would be much easier if I didn't need to guess. $\endgroup$ – terdon Mar 10 '18 at 0:40
2
$\begingroup$

That error is telling you that the script you were running was killed by signal 9, also known as SIGKILL, while executing the command _seqret "$@". This is the way the operating system will kill processes that are asking for more memory than it can provide.

Note that removing _secret "$@" is not the solution since that's what's actually doing the job. $@ is a bash array that contains all the arguments passed to a bash script. The /Users/me/anaconda/bin/seqret file is just a simple script that sets up a few variables and then launches _seqret, passing it the arguments you gave to /Users/me/anaconda/bin/seqret as $@. So deleting that lines just makes the script useless.

Based on your question, I am guessing you're running macOS on a laptop, so it isn't unreasonable that something that needs to read an entire genome and what looks like a full set of annotations will run out of memory on a laptop. Of course, this is a pretty small genome, but the annotations are still 396MB compressed, so depending on your machine's specs and what kind of internal processing the command is doing, it isn't too surprising that it fails.

So, either run this on a more powerful machine, or try splitting the annotations file into smaller files and running it sequentially on each of those.

$\endgroup$
3
$\begingroup$

I'm one of the EMBLmyGFF3 developers. If the compressed annotation is 396MB as mentioned by @terdon it is not a small annotation... In the worse case we have tested we never waited more than few minutes. We haven't done any particular speed test. It would be valuable if you could tell us how long it has taken in your case (If you didn't kill the process before the end...).

At least we could try to implement some nice features, like to see that the tool is working , or/and see which percentage of the file has been already processed.

If it is really slow we could think to try to improve its speed.

$\endgroup$
  • $\begingroup$ Does EMBLmyGFF3 accept .gz compressed files? Is it better? I have started the transformation at 5 pm today. The size of the uncompressed annotation file is 3.1 GB. The genome file is about 100 MB uncompressed. I'll let you know if/when it finishes. It would definitely be useful to have a progress bar and a sign that the tool is working properly. Thanks a lot! $\endgroup$ – charlesdarwin Mar 10 '18 at 17:13
  • $\begingroup$ Still going and using 50 GB of memory (on a 16 GB physical RAM machine) but less than 100% of a CPU. So, it's using the hard disk to store some stuff. $\endgroup$ – charlesdarwin Mar 10 '18 at 17:46
  • $\begingroup$ The process has taken about 24 hours in total on 16 GB physical RAM and 8 cores. $\endgroup$ – charlesdarwin Mar 12 '18 at 1:09
  • 1
    $\begingroup$ Ok it is obviously really slow... Using .gz compressed file will not change anything here. We will investigate how to improve the speedness. $\endgroup$ – juke34 Mar 12 '18 at 9:36
  • 1
    $\begingroup$ You will find the complete list of features and qualifiers accepted by EBI here: ebi.ac.uk/ena/WebFeat $\endgroup$ – juke34 Jul 16 '18 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.