When jobs are scheduled using snakemake with the --cluster command, snakemake waits until the output files are created before continuing with the next job. The scheduler that we use at our cluster (sbatch) allows one to set dependencies on other jobs when launching them. The advantage of scheduling jobs this way is that your job gets run immediately after the job it is dependent on. With snakemake this kind of scheduling is not possible because it waits for the output files and the next job is scheduled at the end of the current queue again. Is it possible to have some option to make snakemake with --cluster not wait on output of inbetween jobs but run the cluster script immediately? I can keep track myself of which files are generated by what jobid to perform the proper scheduling.
Issue #28 resolved