1. Casey Dunn
  2. agalma
Issue #15 resolved

How to restart transcriptome pipeline?

Sergey Naumenko
created an issue

agalma transcriptome -p 48 --stage 22 -i gam15_1

48 = previous run number gam15_1 = specie's id

biolite.pipeline.run_stage: error: argument 'assemblies' is required

Comments (11)

  1. Felipe Zapata

    Hi Sergey,

    You can restart any pipeline in agalma using the following syntax:

    • agalma COMMAND --id CATALOG_ID --restart

    If you need to restart at certain stage, you would also need the flag --stage STAGE_NUMBER

  2. Sergey Naumenko reporter

    Sorry, it does not work: cd /mnt/local/snaumenko/agalma/scratch/transcriptome-48 agalma transcriptome --id gam15_1 --restart --stage 22

    output

    biolite.config.init: override: threads=10 biolite.config.init: override: memory=400G biolite.pipeline.restart: Restarting from checkpoint 'transcriptome.chk'

    errors:

    Traceback (most recent call last): File "/home/tools/agalma-0.3.1/lib/python2.7/site-packages/agalma/transcriptome.py", line 69, in <module> pipe.run() File "/mnt/home/tools/biolite-0.3.1/lib/python2.7/site-packages/biolite/pipeline.py", line 510, in run self.parse_args() File "/mnt/home/tools/biolite-0.3.1/lib/python2.7/site-packages/biolite/pipeline.py", line 280, in parse_args self.restart(chkfile) File "/mnt/home/tools/biolite-0.3.1/lib/python2.7/site-packages/biolite/pipeline.py", line 337, in restart self.checkpoints = cPickle.load(f) EOFError

  3. Felipe Zapata

    Please make sure you are restarting from the same directory you initially started this job:

    /mnt/local/snaumenko/agalma/scratch/

    You should have a file called transcriptome.chk in that directory (please, double check that is the case).

  4. Sergey Naumenko reporter

    Restarting form agalma/scratch:

    agalma transcriptome --id gam15_1 --restart

    biolite.pipeline.restart: Restarting from checkpoint 'transcriptome.chk' Traceback (most recent call last): File "/home/tools/agalma-0.3.1/lib/python2.7/site-packages/agalma/transcriptome.py", line 69, in <module> pipe.run() File "/mnt/home/tools/biolite-0.3.1/lib/python2.7/site-packages/biolite/pipeline.py", line 510, in run self.parse_args() File "/mnt/home/tools/biolite-0.3.1/lib/python2.7/site-packages/biolite/pipeline.py", line 280, in parse_args self.restart(chkfile) File "/mnt/home/tools/biolite-0.3.1/lib/python2.7/site-packages/biolite/pipeline.py", line 337, in restart self.checkpoints = cPickle.load(f) EOFError

    restarting from agalma/scratch/transcriptome-48

    same result

  5. Sergey Naumenko reporter

    Hi, Felipe!

    the solution is to go to the run folder

    cd agalma/scratch/transcriptome-88

    and

    agalma transcriptome --restart

    I would expect:

    1)go to agalma/scratch

    2)agalma transcriptome -p 88 --restart

    agalma transcriptome --id gam15_1 --restart

    is not intuitive, because there are might be many runs for gam15_1 data

    SN

  6. Felipe Zapata

    Thanks for your suggestion. -p is used when one needs the output of one pipeline to be the input of another pipeline. Because we use checkpoints, --restart will automatically restart at the latest checkpoint available in your *.chk file. We offer the flexibility to restart at a specific stage within a pipeline, but the user has to make sure the pipeline will restart at a point where the intermediate files are available (ie, make sure to restart at or before the stage when the pipeline died). You are right there could be other runs for a particular ID, but if the pipeline died and that's the reason why you are restarting the run, it is unlikely you had overwrote intermediate files.

  7. Log in to comment