breakpoint for long time running

[Zhichao Zhou]

Hi,

I have running for 3 weeks for a large dataset of viruses. Our server regularly reboots for 4 weeks. So I have to re-start it in the next coming week. But it will be too time-consuming. This is my current status:

Looks like everything is now set up, we will first clean up the input file, and then we will start the host prediction steps themselves
[1/1/Run] Running blastn against genomes...
[1/3/Run] Get relevant blast matches...
[2/1/Run] Running blastn against CRISPR...
[2/2/Run] Get relevant crispr matches...
[3/1/Run] Running WIsH...
[3/1/Run] Running WIsH extra database...
[3/2/Run] Get relevant WIsH hits...
[4/1/Run] Running VHM s2 similarities...

Please see the time log for the generation time for each file and folder.

Do you have a way to use the old results or a breakpoint-checking method for re-run?

Or I can just use the partial results, for example, I want only “blastn“, “CRISPR“, and “WIsH“ results and get the confidence scores and results based on these?

I really do not want to wait for another 3-4 weeks to see the primary results (perhaps needing curation or re-checking in the future, that will take more time).

‌

Looking forward to your guidance!

Best regards!

‌

‌

‌

‌

[Simon Roux]

Hi Zhichao,
A few things here:
- iPHoP checkpoints by default. If your job is cancelled because of the 4 weeks limit, you can go into the output folder (“Wdir”), remove the last file (to make sure the last step is re-run if ever it was incomplete), and re-run the same command line as you did here. This would skip all the first steps (Blast, CRISPR, WIsH, etc)
- For this kind of cases, we typically recommend splitting the input datasets into smaller batches (and we provide a small utility to do so). Each input sequence is handled separately in iPHoP, so processing by batch does not change the final results, but can speed things up significantly if you can use multiple nodes / jobs.

Best,

Simon

[Zhichao Zhou]

Hi Simon,

Thank you very much!

Best,

Chao