divide by zero error when an "all" target depends on a checkpoint

Create issue
Issue #1113 duplicate
Brendan Furneaux created an issue

Description

I wanted to test that my checkpoint was working as intended, so I make a target "all" which depended on the outputs of the checkpoint, but had no output. This results in a divide-by-zero error.

Snakemake version

5.4.2

Minimal example

import glob

checkpoint demultiplex:
    input: "test.txt"
    output: directory("test")
    shell:
        # in reality the number of output files is not known
        "mkdir -p {output} &&"
        "touch {output}/test-1.txt &&"
        "touch {output}/test-2.txt &&"
        "touch {output}/test-3.txt"

def find_outputs(wildcards):
        checkpoints.demultiplex.get(**wildcards)
        return glob.glob("test/test-*.txt")

rule all:
    input:
        find_outputs

on running snakemake all, I get the following:

Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
    count   jobs
    1   all
    1   demultiplex
    2

[Mon Feb 18 09:09:05 2019]
checkpoint demultiplex:
    input: test.txt
    output: test
    jobid: 1
Downstream jobs will be updated after completion.

Updating job 0.
[Mon Feb 18 09:09:05 2019]
Finished job 1.
[Mon Feb 18 09:09:05 2019]
Error in rule demultiplex:
    jobid: 1
    output: test

Traceback (most recent call last):
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/executors.py", line 357, in _callback
    callback(job)
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/scheduler.py", line 338, in _proceed
    self.progress()
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/scheduler.py", line 477, in progress
    logger.progress(done=self.finished_jobs, total=len(self.dag))
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/logging.py", line 160, in progress
    self.handler(dict(level="progress", done=done, total=total))
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/logging.py", line 125, in handler
    handler(msg)
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/logging.py", line 314, in text_handler
    p = done / total
ZeroDivisionError: division by zero
Removing output files of failed job demultiplex since they might be corrupted:
test
exception calling callback for <Future at 0x7f44d86e6550 state=finished returned NoneType>
Traceback (most recent call last):
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/executors.py", line 357, in _callback
    callback(job)
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/scheduler.py", line 338, in _proceed
    self.progress()
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/scheduler.py", line 477, in progress
    logger.progress(done=self.finished_jobs, total=len(self.dag))
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/logging.py", line 160, in progress
    self.handler(dict(level="progress", done=done, total=total))
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/logging.py", line 125, in handler
    handler(msg)
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/logging.py", line 314, in text_handler
    p = done / total
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 297, in _invoke_callbacks
    callback(self)
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/executors.py", line 367, in _callback
    error_callback(job)
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/scheduler.py", line 346, in _error
    self._handle_error(job)
  File "/home/brendan/.local/lib/python3.5/site-packages/snakemake/scheduler.py", line 356, in _handle_error
    self.running.remove(job)
KeyError: demultiplex

and then snakemake hangs.

It appears that the issue is in the logging code, where snakemake is trying to tell me "x of y (z%)" and getting a divide-by-zero error. The build works properly if I give it the --quiet flag to disable logging, or if the all target has output.

Maybe this is related to #1102

Comments (2)

  1. Log in to comment