What is & how to solve File error: my.xml.state (Remote I/O error)?

Question

I caught the next exception during my phylogeographical analysis in BEAST 2 with GEO_SPHERE. What could be the reason? & how to evade this in the future?

      ...
      856000000     -3662.2647      5969.5577     -9631.8225 42m16s/Msamples
      857000000     -3522.3540      6105.9849     -9628.3389 42m5s/Msamples
      858000000     -3621.7922      6022.9359     -9644.7282 41m54s/Msamples
      859000000     -3463.1050      6107.9766     -9571.0817 41m43s/Msamples
      860000000     -3470.6562      6160.7143     -9631.3706 41m35s/Msamples
File error: VP1_test.xml.state (Remote I/O error)
java.lang.RuntimeException: An error was encounted. Terminating BEAST
        at beast.app.util.ErrorLogHandler.publish(Unknown Source)
        at java.logging/java.util.logging.Logger.log(Logger.java:979)
        at java.logging/java.util.logging.Logger.doLog(Logger.java:1006)
        at java.logging/java.util.logging.Logger.log(Logger.java:1029)
        at java.logging/java.util.logging.Logger.severe(Logger.java:1776)
        at beast.app.beastapp.BeastMain.<init>(Unknown Source)
        at beast.app.beastapp.BeastMain.main(Unknown Source)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at beast.app.beastapp.BeastLauncher.run(Unknown Source)
        at beast.app.beastapp.BeastLauncher.main(Unknown Source)

Unfortunately, googling this error with "BEAST" gave me nothing. If without it, there are a lot of irrelevant problems.

---

Update

I have just found out that this error happened simultaneously onto all my nodes (different processors, RAM, & common hards & SSDs). All logs of BEAST 2 analyses have it. What is stranger, some of the analyses pressed on after this & someones were broken. To the present I got more interested not in the broken ones but could this exception spoil those which continued their work?

Answer 1

You can restart this at the place you left 'cause is Beast2.

  860000000     -3470.6562      6160.7143     -9631.3706 41m35s/Msamples

However, this is almost 1 billion replications and should be sufficient for convergence. The fact it ends on a round number might infer a algorithm limit is in place. If the Beast2 was restarted from this point and it abruptly ended this would be a very likely reason.

The alternative is an error external to the algorithm such as a RAM error - it has been running a long time.

Whatever the reason I would simply perform ESS statistics and convergence diagnostics on what you have now. There's no real reason to continue if it has converged for 10e6 replications maximum and this is much, much bigger than that.

---

The terminated processes have not spoiled the work again you can simply input the terminated chains into the restart for Beast2. This is because Beast2 uses checkpoints. Brief information on how to resume from the last checkpoint is highlighted here https://www.beast2.org/2014/04/07/checkpointing-tricks.html. Convergence checks are important.

The situation is only is only catastrophic for Beast1 where you would have needed to start from the beginning.